"AI, Please Test My App!" - by Gil Tayar
0 likes2,623 views
The document discusses the evolution and classification of autonomous driving, detailing five levels of automation recognized by the Society of Automotive Engineers and the National Highway Traffic Safety Administration. It also highlights the role of machine learning, AI, and visual testing in developing and testing autonomous vehicles. The speaker, Gil Tayar, emphasizes the importance of AI as a tool to aid rather than replace human workers in the tech industry.
"AI, Please Test My App!" - by Gil Tayar
- 1. @giltayar Not Only Cars: "AI, Please Test My App" Gil Tayar (@giltayar) December, 2018 1
- 2. @giltayar@giltayar Driverless Cars Are Here! 1st May 2012 Google passes Nevada’s self-driving car test @giltayar 2
- 3. @giltayar@giltayar And It’s War! ● Between Ford, GM, Mercedes, Volvo, BMW, Tesla, … ● But also between Google, Apple, Intel, Uber, … 3
- 4. @giltayar@giltayar Why? Because it’s boring 4
- 6. @giltayar@giltayar We can start concentrating on the what 6
- 7. @giltayar@giltayar Just Like With Testing ? 7
- 8. @giltayar@giltayar About Me ● My developer experience goes all the way back to the ‘80s. ● Am, was, and always will be a developer ● Testing the code I write is my passion ● Currently evangelist and architect @ Applitools ● We deliver Visual Testing tools: If you’re serious about visual testing, checkout Applitools Eyes ● Sometimes my arms bend back ● But the gum I like is coming back in style @giltayar 8
- 10. @giltayar Driverless Cars Are Here! 10
- 11. @giltayar Driverless Cars Are Here! Who’s Winning? Google or Tesla? 11
- 12. @giltayar 5 Levels of Autonomous Driving ● Classification system of self-driving car ● Developed by the Society of Automotive Engineers (SAE) ● Adopted by the National Highway Traffic Safety Administration (NHTSA) 12
- 15. @giltayar Technological Advances for Level 1 Driving Assistance Cars ● It’s all about vision: ○ LIDAR: Light Detection And Ranging ■ Pulses of laser light that bounce off objects and help measure ranges ■ Similar to radar ○ Stereoscopic cameras ○ Radar ● And the algorithms to understand what is being seen 15
- 17. @giltayar Technological Advances for Level 2 Cars ● It’s all about AI: ○ Machine Learning ○ Neural Networks ○ Deep Learning ● It’s all about the algorithms to understand what is being seen and to act upon them ● Actually, not algorithms... 17
- 18. @giltayar How Do Neural Networks Work? 18
- 20. @giltayar How Do Neural Networks Work? 20
- 22. @giltayar Each Individual Neuron But who decides the weights of each individual neuron? 22
- 23. @giltayar Training and Backpropagation ● Start with random weights ● Feed network with inputs that have known output ● If the network’s output is correct, good ○ But try again! You can do better. ● If not, fix the weights, and loop ● This is the “learning” part of Machine Learning Inputs to outputs Errors to weight adjustment 23
- 24. @giltayar Training and Backpropagation ● Start with random weights ● Feed network with inputs that have known output ● If the network’s output is correct, good ○ But try again! You can do better. ● If not, fix the weights, and loop Inputs to outputs Errors to weight adjustment This is why mileage numbers are important. Deep learning needs lots of data! 24
- 25. @giltayar Machine vs Deep Learning ● Deep Learning is a subset of Machine Learning ● Machine Learning is not just with Neural Networks ○ There are other methods ● Machine Learning started with simple Neural Networks ○ Few inputs, few neurons ● Deep Learning uses high computational powers to take Neural Networks one step forward ○ Lots of inputs (full images) ○ Tons of neurons and layers 25
- 26. @giltayar So how does this work in cars? In: image, out: traffic signals 26
- 27. @giltayar So how does this work in cars? In: image, out: steering actions 27
- 28. @giltayar Level 3: Conditional Automation ● Much more autonomy ● Human need not monitor ● Car will notify when it can’t handle situation, and let the driver take over 28
- 29. @giltayar Level 4: High Automation ● No steering wheel! ● Driver never needs to intervene ● Some conditions do not work ○ Snow storm ○ Fog ● Google cars @giltayar 29
- 30. @giltayar Level 5: Full Automation ● The holy grail of autonomous cars ● Works driverless under any condition! @giltayar 30
- 32. @giltayar Level 0: “No Autonomy” @giltayar 32
- 33. @giltayar Level 1: Drive Assistance What was the differentiator for Level 1 driving? 33
- 34. @giltayar Level 1: Drive Assistance What was the differentiator for Level 1 driving? 34
- 35. @giltayar How Can Visuals Help? ● Action ● Action ● Validation ● Action ● Validation ● Action ● Action ● Action ● Validation If the AI can see the page, can it assist in validation? 35
- 36. @giltayar Looking at the page can check everything at once 36
- 37. @giltayar It’s Not Just Pixels ● Comparing pages visually is not just pixel comparison ● Complex algorithms are involved ● But they sometimes get things wrong ● We need better understanding of what we see 37
- 38. @giltayar Level 2: Partial Automation What was the differentiator for Level 2 driving? 38
- 39. Machine Learning Enhancements ● Detect patterns and structures in a page as a human (Dynamic data) ● Grouping similar changes 39
- 40. @giltayar Level 3: Conditional Automation Not only checking against a baseline Checking any page for design and data problems @giltayar 40
- 41. @giltayar How? ● Machine Learning ● Looking at hundred of test results and seeing how things change over time ● If there is an anomaly, send it over to a human to check 41
- 42. @giltayar But a Human Still Drives the Tests 42
- 43. @giltayar Level 4: Partial Automation ● The AI drives the tests ● Observes users using the application, observes interactions over time ● From those observations, generates tests ○ Using reinforcement learning techniques 43
- 44. @giltayar Level 5: Full Automation 44
- 45. @giltayar Fly on the wings of fantasy ● The AI will be able to converse with the product manager, understand the application, and fully drive the tests itself! ● Unfortunately, given that no human has been able to intelligently understand a product manager’s description of an application, AI-s will need to be much smarter than humans to reach this level! 45
- 47. @giltayar Are we there yet? Should we go looking for another job? @giltayar 47
- 48. @giltayar AI Doesn’t Really Understand 48
- 49. @giltayar Answer These Questions ● Does your job require thinking, or is it rote and repetitive? ○ If thinking, you’re good. AI won’t be replacing you. ● Otherwise... ● If a tool took away the repetitive and boring parts, would you be doing thinking at work? ○ If you will, you’re good. AI will be that tool. It will be helping you! ● Otherwise... ● Can you start thinking? ○ If you can, you’re good. AI won’t be replacing you. It will be helping you! ● Otherwise: ○ Sorry, AI will be replacing you. 49
- 50. @giltayar AI Is A Tool ● Use it ● Benefit from it ● Don’t worry about it @giltayar 50
- 52. @giltayar Thank You Gil Tayar (@giltayar) 52