Introduction to robotics and the advantages
- 1. Robotics Lecture 1: Introduction to Robotics See course website http://www.doc.ic.ac.uk/~ajd/Robotics/ for up to date information. Andrew Davison Department of Computing Imperial College London
- 2. Lecture Plan The regular schedule will be a 1 hour face to face lecture (Tuesday 2pm, Lecture Room 308) and a compulsory 3 hour practical session (Tuesday 3–4pm, and Thursday 11am–1pm, Teaching Labs 219). There may some variations from week to week which will be fully detailed on the course website and announced in lectures, email, and on EdStem. This week only there will be a two hour face to face lecture today (Tuesday 2–4pm), and a one hour live remote lecture on Thursday (11am) on MS Teams. From next week we will start the regular schedule. 1. Introduction to Mobile Robotics and the Course 2. Robot Motion 3. Sensors 4. Probabilistic Robotics 5. Monte Carlo Localisation 6. Advanced Sensing 7. Simultaneous Localisation and Mapping
- 3. Course Organisation • All instructions, the schedule and materials are on the course website at http://www.doc.ic.ac.uk/~ajd/Robotics/. • Live lectures will be recorded and made available soon afterwards on PANOPTO. • Starting next week, practical exercises will be set every week. You will work in fixed groups of 4–5 students using robotics kits based on Raspberry Pi and Lego Mindstorms NXT. • Some practicals will be assessed via demonstration, and this forms the only coursework element of Robotics. • The exam at the end of term will test everything in the course, but will be especially closely tied to the practical exercises. • General support outside of live hours will be given on EdStem. • We will have a competition at the end of term!
- 4. Robotics: An Inter-Disciplinary Field Robotics integrates science and engineering, and overlaps with many disciplines: • Artificial Intelligence • Computer Vision / Perception • Machine Learning / Estimation / Inference • Neuroscience • Electronic / Mechanical Engineering In this course the emphasis will be largely pragmatic: elements of practical algorithms for mobile robotics that have been proven to work in real applications.
- 5. What is a Robot? A physically-embodied, artificially intelligent device with sensing and actuation. • It can sense. It can act. • It must think, or process information, to connect sensing and action. • Pixels to torques. . .
- 6. What is a Robot? • Is a washing machine a robot? Most people would call it an appliance instead, but it does have sensing, actuation and processing. • A possible distinction between appliance and robot (David Bisset): whether the workspace is physically inside or outside the device. • The cognitive ability required of a robot is much higher: the outside world is complex, and harder to understand and control. • What about a modern car? Or smartphone? Are they becoming robots?
- 7. The Classical Robot Industry: Robot Arms • The most widely and successfully used robots up until now are industrial robot ‘arms’, mounted on fixed bases and used for instance in manufacturing. • Most operate in highly controlled environments, and carry out repetitive movements.
- 8. Robots for the Wider World • Experimental mobile robots are now being tested in a wide range of challenging application scenarios. • They need perception which gives them a suitable level of understanding of their complex and changing surroundings.
- 9. A Fully Autonomous Robot for the Home? • There is just as much challenge in developing generally capable robots for the home as there is in those outdoor environments. • There is a new wave of advanced mobile robots now aiming at much more flexible robots which can interact with the world in human-like ways. Over recent years this has again become the current goal of significant research teams. See the videos at http://personalrobotics.stanford.edu/ from Stanford’s Personal Robotics Program.
- 10. Our Focus: Mobile Robots • What are the general principles of how robots move? Why and how can they use sensors to understand their environments sufficiently to navigate usefully and safely? • Required competences include: • Movement control • Obstacle avoidance • Localisation • Mapping • Path planning • . . . as well as whatever specialised task the robot is actually trying to achieve! • Real world robots must deal with the noisy nature of real sensors and actuators, and that is what leads us down the path to probabilistic methods.
- 11. Mobile Robotics Applications Field Robotics • Exploration (planetary, undersea, polar). • Inpection (factories, bridges, etc.) • Search and rescue (earthquake rescue; demining). • Mining and heavy transport; container handling. • Military (unmanned aircraft, land-based pack-bots, insect robots). Service Robotics • Domestic (Vacuum cleaning, lawnmowing, laundry, more general clearing and cleaning. . . ?). • Medical (remote doctor, hospital delivery, helping the elderly). • Transport (Autonomous cars, parcel delivery). • Entertainment (Robot pets/companions, robot building kits, robot competitions, Personal drones, many others).
- 12. Levels of Autonomy for Mobile Robots Mobile robots will ideally be untethered and self-contained, with power source, sensing and processing on-board, but other levels of autonomy are possible. Level of autonomy: 1. Teleoperation (Remotely-Operated Vehicle ROV, e.g. Robot Wars, mine clearing, surgical robots, some delivery robots). 2. Semi-autonomous/Supervised (e.g. Mars rovers, driver assist systems). 3. Fully autonomous (e.g. robot vacuum cleaners, autonomous cars(?)).
- 13. Mobile Robots: State of the Art Mars Rovers Spirit and Opportunity (NASA) • Both had successful missions on Mars in starting in late 2004. Spirit went ‘silent’ in March 2010, and Opportunity finally in 2018. • 1.6m long; 180kg. 9 cameras (Hazcams, Navcams, Pancams, microscopic). • Remote human planning combined with local autonomy. • Increased autonomy as mission progressed. • Currently active: Curiosity Rover (2012), up to 90m/hour, has travelled about 30km; Perserverance (2021); Zhurong (2021).
- 14. Mobile Robots: State of the Art DARPA Grand Challenge 2005 winner “Stanley” (Stanford University, USA). • Completed 175 mile desert course autonomously in 6 hours 54 minutes. • Guided along rough ‘corridor’ by GPS. • Road-following and obstacle avoidance using laser range-finders and vision. https://www.youtube.com/watch?v=FLi_IQgCxbo
- 15. Mobile Robots: State of the Art DARPA Urban Challenge 2007 winner ‘Boss’ (Carnegie Mellon University) • Robots had to achieve extended missions in a mocked-up urban area, obeying traffic laws and avoiding other robots and cars. • Much more sophisticated sensor suites than in desert challenge (lasers, cameras, radars) to achieve all-around awareness. • Current technology: e.g. Google car (now Waymo) https://www.youtube.com/watch?v=B8R148hFxPw • Most car companies now have major autonomous driving projects. Other companies are developing ‘autonomous taxi’ services.
- 16. Mobile Robots: State of the Art • Spot, from Boston Dynamics https://youtu.be/wlkCQXHEgjA. • Five active stereo sensors give all around perception for localisation and mapping. • Product officially launched in 2019 and aimed at industrial inspection applications.
- 17. Mobile Robots: State of the Art Skydio: ‘The Self-Flying Camera’ • Visual-inertial navigation and obstacle avoidance using multiple stereo camera pairs. • Mobile NVidia processor onboard. • https://www.youtube.com/watch?v=gsfkGlSajHQ
- 18. Mobile Robots: State of the Art iRobot ‘Roomba’ Robot Vacuum Cleaner, first launched in 2002 • ‘Random bounce’ movement style with short-range IR sensing. • Over 10 million units sold! • Later generation and competing products are now achieving precise navigation. • http://www.youtube.com/watch?v=OMUhSBeIm40
- 19. Dyson 360 Eye • On sale in Japan in 2015; around the world in 2016. • Uses omnidirectional vision to build a map of its environment automatically (SLAM). This permits accurate, repeatable localisation, and therefore precise coverage and cleaning. • https://youtu.be/gZ3VV0OlAxo
- 20. The Dyson Robotics Laboratory at Imperial College • Founded in 2014, funded by Dyson and led by Andrew Davison, our lab researches the vision and robotics technology that we hope will open up new categories in robotic products for the home. http://www.imperial.ac.uk/dyson-robotics-lab. • Part of a thriving robotics research community across Imperial College: http://www.imperial.ac.uk/robotics.
- 21. NodeSLAM: Making maps using Neural Object Descriptors • Sucar, Wada, Davison, Dyson Robotics Lab at Imperial College, 3DV 2020. • Volumetric coded object models for four categories. • Optimised against multi-view depth images using differentiable rendering. • https://youtu.be/zPzMtXU-0JE
- 22. Robotics: Requirements 1. Essential geometry (vectors, rotations, trigonometry). 2. Essential probability theory. 3. Programming: you will write a lot of code. 4. Willingness to work with (build, adjust, calibrate, test) robotics elements from scratch, and to deal with the frustrations of real robotics hardware. 5. The practicals are a big commitment and a lot of work.
- 23. Robotics: Learning Outcomes By the end of the course you should understand: 1. The defining properties of a robot: sensing and action, linked by processing. 2. An overview of the practical issues of modern-day mobile robotics. 3. Robot locomotion methods, particularly wheel control, encoders, configurations and uncertainty in motion. 4. 2D coordinate frames for pose estimation and planning. 5. The use of sensors in reactive, behavioural programming. 6. Calibrating and robustifying sensors. 7. The essentials of probabilistic techniques in robotics; probabilistic localisation and mapping. 8. Local and global planning for wheeled robots. 9. Techniques for real-time robot programming.
- 24. Robotics: A Practical Course In the first practical, in groups you will be given a robotics kit which you will keep throughout term to work on practical exercises every week. We will use these kits to build mobile robots and implement most of the methods you are taught in lectures.
- 25. Robotics: Coursework and Assessment The coursework component is based on cumulative assessment of achievement in the practical sessions and there will be no submission of written reports. You will be set a practical task each week, several of which (and each practical sheet will very clearly say which) will be ASSESSED. • For the first practical session we will ask you all to organise yourself into practical groups for the term; we need people to commit to the course at this point. • Each assessed practical exercise will have a number of well-defined objectives with a specified number of marks for each. Most of these objectives involve practical demonstration of your robots, showing us some results on paper or on screen, or oral explanation of results. • We will mark these exercises by visiting all groups at the start of the next week’s practical session, where each group must demonstrate their robot and discuss with me or a lab assistant. • We will check attendance in each group at the assessments and will ask questions to make sure each group member has been involved.
- 26. Robotics: Coursework and Assessment • The total marks from the assessed practicals will form your overall coursework mark for Robotics. • No extra written coursework will be set. • All members of a group will receive the same mark by default (unless we have a strong reason to believe that certain members are not doing their share of work). • Coursework marks in Robotics are worth 33% of the total marks available for the whole course, to reflect the fact that it is a lot of work. Also, the exam will be designed to tie in closely with the coursework, and those members of groups that have made a good effort during term do very well on the exam. • Previous years’ exam papers are a good guide to seeing what the style of questions will be, but every year the exam will change somewhat to reflect the current lecture and practical content of the course.
- 27. Robotics: Competition In the final week of the course, we will have a competition between the groups, testing the performance of the robots developed for the final practical exercise. See the course website for pictures and videos from previous years’ competitions . . . but this year’s challenge will be different again! See previous years at http://www.doc.ic.ac.uk/~ajd/Robotics/.
- 28. Extra Information • Robotics course web page (will carry course timetable, notes, practical sheets, extra handouts and other information): http://www.doc.ic.ac.uk/~ajd/Robotics/ • You should not need to buy any books, but if you want some more detail on probabilistic robotics in particular we can recommend the following: • ‘Probabilistic Robotics’, Sebastian Thrun, Wolfram Burgard and Dieter Fox
- 29. Robot Floor Cleaner Case Study • If you are interested in some good motivation for mobile robotics and product thinking, see our case study tutorial on the website (in the Additional Handouts section lower down). • Dyson DC06: almost released in 2004 but never went on sale. • In many ways floor cleaning presents an unusual mobile robot navigation problem; rather than just get from A to B it has to visit everywhere in a domain.
- 30. Some Robot Floor Cleaners on the Market Roomba Navibot Mint Neato Roomba (iRobot), floor coverage http://www.youtube.com/watch?v=OMUhSBeIm40 Mint Floor Cleaner (Evolution Robotics) http://www.youtube.com/watch?v=6Cf55mIaNGw
- 31. Evaluating Robot Floor Cleaners • Dyson ‘GTS’ open source ground truth evaluation system based on ceiling-mounted cameras.
- 32. Getting More From Omnidirectional Vision • Fitting ‘box’ room models to omnidirectional depth reconstruction. • Lukierski, Leutenegger and Davison, 2016.
- 33. Real-Time Height Map Fusion from a Single Camera Tvc (Zienkiewicz, Davison, Leutenegger, 2015/2016) • Dense fusion from a moving robot-mounted camera to identify free space and obstacles. • https://youtu.be/3NQqeRcSsCw • Real-time multi-scale fusion using adaptive level of detail, 3DV 2016.