Se research update
0 likes128 views
1) The document summarizes a research update presentation on software engineering and artificial intelligence given by Assistant Professor Nacha Chondamrongkul. 2) It discusses how software engineering research tackles different stages of software production to minimize costs, efforts, and failures. It also examines how AI can be applied to enhance software engineering processes and how software engineering principles are needed to develop AI systems. 3) Key challenges discussed include how to specify requirements for intelligent systems, test AI systems given their unpredictability, and address issues around reliability, fairness, and deployment when integrating machine learning models into complex software.
![Example Research in Software Analytics
• StackMine [ICSE'12] - performance debugging
• Data source: performance call stack traces from windows end user
• Analytics output – ranked cluster of call stack traces
• Impact: Deployed/Used in daily practice of Windows performance
analysis team
• Service Analysis Tool (ASE'13) - service incident management
• Data source: Transaction logs, past incident reports
• Output: Healing suggestions/likely root causes of given incidents
• Impact: deployed and used by important microsoft service](https://image.slidesharecdn.com/seresearchupdate-220112015847/85/Se-research-update-18-320.jpg)
Se research update
- 1. Software Engineering Research Update Assistant Professor Nacha Chondamrongkul, PhD. School of Information Technology Mae Fah Luang University
- 2. Agenda Software Engineering Research Landscape AI & Software Engineering
- 3. Research Talk • Projects-based learning for Software EngineeringEducation Aj.Tan • Online Learning Ecosystem Aj. Pam • RespiratoryMonitoring in Sleeping Environment Aj. Oil
- 4. What is Software Engineering "Software engineering is an engineering discipline that is concerned with all aspects of software production from the early stages of system specification through to maintaining the system after it has gone into use" Ian Sommerville (Software Engineering, 9th Edition, 2011)
- 6. Software Engineering Research Researches tackle different stages of ware production to minimise: "cost / effort / failure"
- 7. Software Quality • Researches tackle the quality of software:
- 8. Software Engineering Venues • General venue: - ICSE (A*), FASE (A), ASE (A), APSEC(B), SEKE(B) • Requirement Engineering - venue: RE(A) • Software Design – venue: ICSA(A), ECSA(A) • Code Development - venue: OOPSLA (A), POPL(A*) • Testing venue: ICST(A) • Maintenance & Evolution – venue: SANER(A), ICSME(A) • Software Engineering Education – venue: ICSEET(A)
- 9. AI and SE Artificial Intelligence Software Engineering AI for SE, apply AI to enhance software production SE for AI, how to engineer AI-based software
- 10. AI for SE This Photo by Unknown author is licensed under CC BY.
- 11. Artificial Intelligence CLASSIFICATION OPTIMISATION SCHEDULING REASONING
- 12. Intelligent Test Generation • Pex – Microsoft Research (ASE'14) • Automated Test Generation for .Net code • Use dynamic symbolic execution (Path exploration) • Shipped in Visual Studio 2015/2017 as "IntelliTest"
- 13. Intelligent Test Generation (Cont.) • Tencent's automatic test generation tool for WeChat (FSE'16) (ICSE'17) • Test coverage is a challenge. • Guided random generation of test input using Google Monkey • Deployed in daily wechat practices • Active 1 billion users (2018)
- 14. Software Evolution Planning • Automated planning of evolution path (ECSA 2020/TOSEM) • Evolution planning is a challenge as software need to evolve as small incremental steps support the modernisation • Apply formal model to enable automated verification • Apply AI Planning to find safe path to the target design
- 15. Security analysis through automated reasoning • Apply automated reasoning to identify security characteristics (ICSA'20) • Focus on blockchain & microservices • Use OWL & ontology model to represent software architecturedesign • Define ontology class to represent security such as attack surface, least privilege, etc. • Model checking to simulate the attack that could happen
- 16. Software Analytics • Software analytic is to enable software engineers to perform data exploration and analysis in order to obtain insightful and actionable information for data-driven task around software and services. *Dongmei Zhang, Yingnong Dang, Jian-Guang Lou, Shi Han, Haidong Zhang, and Tao Xie. 2011. Software analytics as a learning case in practice: approaches and experiences. In Proceedings of the International Workshop on Machine Learning Technologies in Software Engineering (MALETS '11).
- 17. Datasource
- 18. Example Research in Software Analytics • StackMine [ICSE'12] - performance debugging • Data source: performance call stack traces from windows end user • Analytics output – ranked cluster of call stack traces • Impact: Deployed/Used in daily practice of Windows performance analysis team • Service Analysis Tool (ASE'13) - service incident management • Data source: Transaction logs, past incident reports • Output: Healing suggestions/likely root causes of given incidents • Impact: deployed and used by important microsoft service
- 19. Make machine recognise code • Code2Vec (POPL'2019) • Convert source code into Abstract Syntax Tree (AST) and use ML to learn. • Let the ML model predict what the source code does. • https://code2vec.org • Can be utilised for code searching by given human language, e.g. GitHub Co-pilot
- 20. GitHub Co-pilot
- 21. Open Topics in AI for SE • How to define or determine level of intelligence in software engineering solution? • Automation vs. Intelligence • How to attain high-quantity and high-quality labelled software data? • How to transfer research results into industrial practice?
- 22. SE for AI This Photo by Unknown author is licensed under CC BY-SA-NC.
- 23. Challenges in AI/Autonomous System • How to build such systems by combining technique from software engineering with AI and ML? • Develop & Test • How to specify autonomous system behaviour in the face of unpredictability, in environment: human, physical objects, other systems *In search of foundation of next-generation autonomous system,Proc, Natl Acad, USA, 2020
- 24. AI system is complex • March 18, 2018, self-driving Uber car struck and killed a victim. • "Was this algorithmic tragedy inevitable? And such incidents should we prepare for? • Software is released into a code universe which no one can fully understand.
- 25. Machine Learning (ML) ML is a black box. ML has no test oracle. Regulation require computer software to be transparency ML systems needto be trust- worthy
- 26. Adversarial ML Testing • Generate examples that derives from legitimate examples with slight modification to induce misclassification Tian et al, Deep Test, ICSE 2018
- 27. DeepBillboard • DeepBillboard contributes to the systematic generation of adversarial examples for misleading steering angle when perturbations are added to roadside billboards. • An attacker may easily construct a physical world billboard to affect the steering decision H. Zhou et al., "DeepBillboard: SystematicPhysical-World Testing of Autonomous Driving Systems," 2020 IEEE/ACM 42nd International Conferenceon Software Engineering (ICSE),2020,pp. 347-358.
- 28. Requirement and Testing of AI system is a challenge • In 2014, Amazon tried building AI tool to help with recruiting but the tool was discriminating against women; in 2017, Amazon abandoned the tools • Data source: predominantly male resumes submitted to Amazon over 10- year period as candidates • Fairness testing: testing software for discrimination (Galhotra et al. ESEC/FST'17) This Photo by Unknown author is licensed under CC BY.
- 29. Challenges in ML Model Deployment
- 30. Open Challenges in Deployment
- 31. Reliability in ML-related library • Detecting Numerical Bugs in NN Architecture Yuhao Zhang, Luyao Ren, Liqian Chen, Yingfei Xiong, Shing-Chi Cheung, and Tao Xie. 2020. Detecting numerical bugs in neural network architectures. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020).
- 32. SE for AI - Open Challenges • How to specify requirement for intelligence software • How to tackle the complexity of integrating software with the rest of the system. • How to define/verify test oracle (or properties) for intelligence software. • How to design high-quality library for developing intelligence software. • How to transfer research results into industrial practice.
- 33. Thank you Q&A