TechEvent 2019: Artificial Intelligence in Dev & Ops; Martin Luckow - Trivadis
1 like•524 views
This document provides an overview of artificial intelligence trends and applications in development and operations. It discusses how AI is being used for rapid prototyping, intelligent programming assistants, automatic error handling and code refactoring, and strategic decision making. Examples are given of AI tools from Microsoft, Facebook, and Codota. The document also discusses challenges like interpretability of neural networks and outlines a vision of "Software 2.0" where programs are generated automatically to satisfy goals. It emphasizes that AI will transform software development over the next 10 years.
![=2018
“High-level machine intelligence” (HLMI)
is achieved when unaided machines can
accomplish every task better and more
cheaply than human workers.
arXiv:1705.08807v3 [cs.AI] 3 May 2018
Yale/Oxford-study
(2016/18)
352 researchers in AI,
worldwide
3-Point-Estimation for
“HLMI arriving in x
years”
➔ 50% chance for HLMI
within 45 years
➔ 10% chance within 9
years
Asian researchers see it in 30y, north americans in 74y ☺
Time Until Machines Outperform Humans](https://image.slidesharecdn.com/artificialintelligenceindevopsmartinluckowtrivadistechevent2019-191002122204/85/TechEvent-2019-Artificial-Intelligence-in-Dev-Ops-Martin-Luckow-Trivadis-5-320.jpg)
![The next 15
Years of ML
2016 (2018) =
arXiv:1705.08807v3 [cs.AI] 3 May 2018
50%
50% 75%
“Physically assemble any LEGO set given the pieces
and instructions, using non- specialized robotics
hardware.”
“Defeat the best Go players,
training only on as many
games as the best Go
players have played.”
Problem solved ☺
Company bankrupt (Laundroid)
https://aibirds.org
Angry Birds AI Competition](https://image.slidesharecdn.com/artificialintelligenceindevopsmartinluckowtrivadistechevent2019-191002122204/85/TechEvent-2019-Artificial-Intelligence-in-Dev-Ops-Martin-Luckow-Trivadis-6-320.jpg)
![The Future: towards General AI
2016 (2018) =
arXiv:1705.08807v3 [cs.AI] 3 May 2018](https://image.slidesharecdn.com/artificialintelligenceindevopsmartinluckowtrivadistechevent2019-191002122204/85/TechEvent-2019-Artificial-Intelligence-in-Dev-Ops-Martin-Luckow-Trivadis-7-320.jpg)
![References
• Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans: „When Will AI Exceed Human
Performance? Evidence from AI Experts“, arXiv:1705.08807v3 [cs.AI] 3 May 2018
• Toby Walsh: „Expert and Non-Expert Opinion about Technological Unemployment“, arXiv:1706.06906
[cs.AI] 21 June 2017
• Gartner: “Top 10 Strategic Technology Trends for 2019”, Published 15 October 2018
• Gartner: “Top 10 Strategic Technology Trends for 2019: AI-Driven Development”, Published 13 March 2019
• Oleksii Kharkovyna: “Why AI & ML Will Shake Software Testing up in 2019”, Hackernoon, Published 7 March
2019
• Diego Lo Giudice et al: “How AI Will Change Software Development And Applications”, Forrester Report,
Updated Nov. 2, 2016
• Andrej Karpathy: “Software 2.0”, Medium, Nov. 11, 2017
• Microsoft: “Sketch 2 Code”, https://sketch2code.azurewebsites.net
• Codota: https://www.codota.com](https://image.slidesharecdn.com/artificialintelligenceindevopsmartinluckowtrivadistechevent2019-191002122204/85/TechEvent-2019-Artificial-Intelligence-in-Dev-Ops-Martin-Luckow-Trivadis-39-320.jpg)
TechEvent 2019: Artificial Intelligence in Dev & Ops; Martin Luckow - Trivadis
- 1. Trivadis Blog Artificial Intelligence in Dev & Ops a teaser… ☺ Dr. Martin Luckow
- 2. Martin Luckow • Since 2011@Trivadis • Transformation Architect • Solution Development • Trend Radar Responsible https://www.linkedin.com/in/martin-luckow/
- 3. Round Up The past, the state and the future of AI
- 4. AlphaGo Zero IBM Watson AI Timeline Eugene GoostmanELIZA eurequa HeurekaBlocksworld Source: Gartner (High Level Machine Intelligence) VA‘s (Siri…) Google DuplexDeep Blue
- 5. =2018 “High-level machine intelligence” (HLMI) is achieved when unaided machines can accomplish every task better and more cheaply than human workers. arXiv:1705.08807v3 [cs.AI] 3 May 2018 Yale/Oxford-study (2016/18) 352 researchers in AI, worldwide 3-Point-Estimation for “HLMI arriving in x years” ➔ 50% chance for HLMI within 45 years ➔ 10% chance within 9 years Asian researchers see it in 30y, north americans in 74y ☺ Time Until Machines Outperform Humans
- 6. The next 15 Years of ML 2016 (2018) = arXiv:1705.08807v3 [cs.AI] 3 May 2018 50% 50% 75% “Physically assemble any LEGO set given the pieces and instructions, using non- specialized robotics hardware.” “Defeat the best Go players, training only on as many games as the best Go players have played.” Problem solved ☺ Company bankrupt (Laundroid) https://aibirds.org Angry Birds AI Competition
- 7. The Future: towards General AI 2016 (2018) = arXiv:1705.08807v3 [cs.AI] 3 May 2018
- 8. AI in Development & Operations… The ultimate Goal • to enable users to describe their own user stories or business requirements • to an AI-based platform that in turn • models, • tests and • executes them automatically
- 9. Enthusiasts and Pragmatists Rapid Prototyping Intelligent Programming Assistants Automatic Analytics & Error Handling Automatic Code Refactoring Project Management Decision Making „AI is everything“ „AI is a tool“ „Software 2.0“
- 10. “Software 2.0” in 10 Secs 1. Specify some goal on the behavior of a desirable program (e.g., “satisfy a dataset of input output pairs of examples”, or “win a game of Go”) 2. Write a rough skeleton of the code (e.g. a neural net architecture), that identifies a subset of program space to search, and use the computational resources at to search this space for a program that works. 3. In the specific case of neural networks, we restrict the search to a continuous subset of the program space where the search process can be made efficient with backpropagation and stochastic gradient descent. Andrej Karpathy Director of AI at Tesla. Previously Research Scientist at OpenAI and PhD student at Stanford.
- 11. “Software 2.0” Challenges “At the end of the optimization we’re left with large networks that work well, but it’s very hard to tell how.” “Across many applications areas, we’ll be left with a choice of using a 90% accurate model we understand, or 99% accurate model we don’t.” “… can fail in unintuitive and embarrassing ways ,or worse, can “silently fail” (By silently adopting biases in their training data, which are very difficult to properly analyze and examine when their sizes are easily in the millions in most cases.)” … Result of the fundamental properties of neural networks
- 12. AI & Development Process Source: Gartner
- 13. Sources of AI? Azure https://azure.microsoft.com/en-gb/services/cognitive-services/ Where to get AI services? Try AWS, Azure, Google, …
- 14. AI & Fit For Purpose Apps App D Narrow, workflow- oriented apps are suitable for todays narrow AI Professional, cloud-based AI- services are already available Embedded and on-premise AI available starting in 2019
- 15. AIaaS Embedded AIaaS AIaaS AI & Meshed App & Service Architecture
- 16. Back to AI in Development & Operations
- 17. •ML is shortening running business requirements into technology by enabling less technical domain experts to develop technologies using either natural language or visual interfaces. Rapid Prototyping •Smart programming assistants can reduce reading documentation and debugging code by offering JIT support and recommendations, such as relevant document, best practices, and code examples. Intelligent Programming Assistants •Programming assistants can also learn from past experience to identify common errors and flag them automatically during the development phase. Automatic Analytics & Error Handling •Machine learning can be used to analyze code and automatically optimize it for interpretability and performance. Automatic Code Refactoring •Machine learning can train on data from past projects – such as user stories, feature definitions, estimates, and actuals – to predict effort and budget more accurately. Precise Estimates •An AI trained on past development projects and business factors can assess the performance of existing applications and features and help to identify efforts that would maximize impact and minimize risk. Strategic Decision-Making
- 18. Microsoft’s Sketch2Code Use Case • Transform any hands-drawn design into HTML code User Workflow 1. Take a picture from the whiteboard 2. Provide it to the service 3. Get the HTML-Code
- 19. Microsoft’s Sketch2Code Try it on https://sketch2code.azurewebsites.net/ • API interface in Serverless implementation (FaaS) • Open Source on GitHub • Evolving
- 21. Codota: AI line completion & more • Uses learned open- source code models to suggest relevant code • Intelligent code snippets search • Supports Java & Scala, JavaScript (soon) • Works with IntelliJ, Android Studio and Eclipse. https://www.codota.com
- 22. Microsoft’s IntelliCode Try it in Visual Studio 2019… enable it in Tools > Options > IntelliCode Training base: thousands of open source projects on GitHub each with over 100 stars • Assisted IntelliSense • Interactive training: AI-assisted recommendations for your own code • Issue detection (preview) • Focused Code Review • For C#, C++, Java, Phyton, TypeScript/JavaScript, XAML
- 24. Facebook’s Sapienz Multi-objective Automated Testing for Android Applications (… and iOS)
- 25. Use Case • Facebook launched on the web • If a bug is found on the web, an update can be rolled out immediately • Mobile apps require the user to physically update their app to get a fix, which makes it all the more important to find bugs well before the app ships. • Facebook’s android has more than 1b installations… https://github.com/Rhapsod/sapienz
- 26. Search-Based Software Engineering Converts a software engineering problem into a computational search problem and involves • defining a (huge!) search space, or the set of possible solutions • defining a metric (fitness | objective | cost | quality function) to measure the quality of potential solutions • finding (local) extremas of the function by (tricky) searches in the space. Sapienz applies this to software testing, including • automatic generation of test cases (test data) • test case minimization and • test case prioritization https://github.com/Rhapsod/sapienz
- 27. Setup - null pointer exceptions - resource leaks - annotation reachability - missing lock guards & concurrency race conditions Static analysis on code ? ? ? ? ? Dynamic analysis on thousands of emulators Suspects Confirmed
- 28. Facts • According to Facebook, 75% of reported crashes are fixed… which means that Sapienz indicates real problems in the code, mostly • Rollout to FB’s main iOS Apps https://github.com/Rhapsod/sapienz
- 30. SapFix / Developer Interaction • In early stages of deployment in the Facebook Android app • Automatically generates fixes for specific bugs • Final call on whether to accept the fix is made by a human engineer.
- 31. SapFix
- 32. SapFix / Developer Interaction
- 33. To sum things up We are still at the beginning, but...
- 34. “There will be a long ramp-up as knowledge diffuses through the developer community, but in ten years I predict most software jobs won’t involve programming.” (Pete Warden, Google, Tensorflow) Deep Learning is Eating Software
- 35. What should you do? •Attract, retain and motivate developers to build their application development skills by investing in AI fundamentals training today. Attract, retain and motivate •Explore the AI-augmented development tools, by building evaluation criteria with your development team, to address both near-and longer-term requirements. Explore •Build a roadmap for advancing your AI-augmented development strategy, leveraging both team practices (such as TDD and FDD) and cloud services. Build
- 36. Skills, Staffing, Organisation Financial Planning Measurements Use Cases Roadmaps Continuous Stakeholder-Management How can Trend Radar be used in your company? Business Strategy Digital Strategy Unit-Level Strategies Ideation Incubation-Projects & Labs Realisation Operations Ideation Understand Business Context Ideation 1 2 3
- 37. Trend Radar Iteration Workflow Trend identification Trend / use case analysis and preparation Trend presentation & documentation handover Trend communication and processingSource analysis Review Actions POCs New Work Connectivity Virtual Assistants Deep Learning Digital Workplace Mobile BI Silver Society … Rating 17:10
- 38. Qualified use cases are derived from relevant trends. Inspired from these use cases, concrete actions and projects are derived and implemented. How do we structure trends for you? 1. Name 2. Definition 3. Use Case Examples (General/Industry Specific) 4. Market Context e.g.: “Automotive”, “Banking”, “Healthcare”, “Communication”, “Retail” 5. Business Impact 6. Benefit Rating e.g.: “Low”, “Moderate”, “High”, “Transformational” 6. Position in Hype-Cycle 7. Market penetration 8. Market maturity 9. Time to Mainstream e.g.: <2y, 2-5y, 5-10y (TTA) 10. User Advice
- 39. References • Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans: „When Will AI Exceed Human Performance? Evidence from AI Experts“, arXiv:1705.08807v3 [cs.AI] 3 May 2018 • Toby Walsh: „Expert and Non-Expert Opinion about Technological Unemployment“, arXiv:1706.06906 [cs.AI] 21 June 2017 • Gartner: “Top 10 Strategic Technology Trends for 2019”, Published 15 October 2018 • Gartner: “Top 10 Strategic Technology Trends for 2019: AI-Driven Development”, Published 13 March 2019 • Oleksii Kharkovyna: “Why AI & ML Will Shake Software Testing up in 2019”, Hackernoon, Published 7 March 2019 • Diego Lo Giudice et al: “How AI Will Change Software Development And Applications”, Forrester Report, Updated Nov. 2, 2016 • Andrej Karpathy: “Software 2.0”, Medium, Nov. 11, 2017 • Microsoft: “Sketch 2 Code”, https://sketch2code.azurewebsites.net • Codota: https://www.codota.com
- 40. References • Ke Mao, Mark Harman, Yue Jia : „Sapienz: Multi-objective Automated Testing for Android Applications“, http://www0.cs.ucl.ac.uk/staff/K.Mao/archive/p_issta16_sapienz.pdf, 2016