Software Modeling and Artificial Intelligence: friends or foes?
Download as PPTX, PDF•3 likes•2,279 views
(1) Modeling and AI can be both friends and foes, depending on how they are used together. (2) Model-driven engineering (MDE) approaches can help make AI systems like chatbots and machine learning pipelines more rigorous, robust, and interoperable by applying modeling principles. (3) AI techniques like machine learning and deep learning also have the potential to enhance MDE, for example by enabling automated model transformations and smarter modeling tools with features like autocomplete.
![Execution Language
Import library Example
Import platform Slack
Import platform Github
Listen to Slack
on intent OpenNewIssue do
Slack.reply(‘’Sure, I’ll help you to write
your issue! Which repository is for?’’)
on intent SpecifyRepository do
Slack.reply(‘Great, I’ll open an issue
related to repository {$Repository.name}.’’)
[… a few intents later with all the info …]
Github.openIssue({$Repository.name},
{$Issue.title}, {$Issue.content})](https://image.slidesharecdn.com/modelingia-190614061122/85/Software-Modeling-and-Artificial-Intelligence-friends-or-foes-38-320.jpg)
Software Modeling and Artificial Intelligence: friends or foes?
- 1. Modeling and AI: Friends or foes Jordi Cabot @softmodeling – jordicabot.com June 2019
- 2. About
- 3. SOM Research Lab Software runs the world. Models run the software
- 4. Our mission We are interested in the broad area of systems and software engineering, especially promoting the rigorous use of models and engineering principles while keeping an eye on the most unpredictable element in any project: the people. Flickr/clement127
- 10. To model, or not to model, this is the WRONG question - Shakespeare
- 12. Cognification: The application of knowledge to boost the performance and impact of a process
- 14. © Prof. Dr. Andreas Vogelsang – TU Berlin
- 15. Why should you care?
- 19. Kai-Fu Lee
- 20. AI Coder
- 21. Modeler
- 23. How can we be friends: (1) MDE for AI
- 24. • Grady Booch – history of softwre engineering The entire history of software engineering is that of the rise in levels of abstraction - Grady Booch Also true for AI -> MDE is the best tool for this evolution
- 25. AI is software, too! (and rather immature)
- 27. Modeling ML pipelines (data preparation is more challenging than the learning itself)
- 28. As expected, all players are moving in this direction • Key to get more users that will then become users of their underlying infrastructure Azure ML Studio Orange RapidMiner IBM SPSS Modeler Nvidia digits Knime
- 30. Model-Driven Bot Development: Xatkit https://github.com/xatkit-bot-platform Gwendal Daniel, Jordi Cabot, Laurent Deruelle, Mustapha Derras: Multi-platform Chatbot Modeling and Deployment with the Jarvis Framework. CAiSE 2019: 177-193
- 31. Let’s create a chatbot to help newcomers to write issues on Github! Alright! It’s just a set of questions & answers, this will be pretty simple! Narrator It wasn’t. Once upon a time
- 32. What went wrong? if(‘’ I have an issue’’) { return ‘’In which repository?’’ }
- 33. Chatbots are complex systems Conversation Logic Text Processing External Services Messaging Platforms Deployment Evolution Maintenance Tests
- 35. MDD of (chat)bots with Xatkit • Raise the level of abstraction at what chatbots are defined – Focus on the core aspects of the chatbot • Conversation and user interactions • Action computations – Independent from specific implementation technologies • Automate the deployment and execution of the modeled chatbot over multiple platforms
- 36. Chatbot dev – Example: Bots for OSS Slack bot to help users open a well-described new issue on a GitHub project If the User Wants To Open Issue Reply « In which repository? » on Slack User Intent Action Parameters Platform Conversation step
- 37. Intent language Library example intent OpenNewIssue { inputs { ‘’I want to create an issue’’ ‘’Open an issue’’ } } intent SpecifyRepository { inputs { ‘’In repository MyRepo’’ } outContext Repository { name <- ‘’MyRepo’’ (@any) } }
- 38. Execution Language Import library Example Import platform Slack Import platform Github Listen to Slack on intent OpenNewIssue do Slack.reply(‘’Sure, I’ll help you to write your issue! Which repository is for?’’) on intent SpecifyRepository do Slack.reply(‘Great, I’ll open an issue related to repository {$Repository.name}.’’) [… a few intents later with all the info …] Github.openIssue({$Repository.name}, {$Issue.title}, {$Issue.content})
- 39. Platform Language Abstract platform Chat actions { PostMessage(message, channel) Reply(message) } } Platform Slack extends Chat Platform Github action { openIssue(repository, title, content) } }
- 40. Xatkit Framework Chatbot Designer Intent Recognition Providers (platform-specific) Platorm Package Intent Package Xatkit Modeling Language Chatbot User Instant Messaging Platforms Xatkit Runtime Execution Package uses uses Platform-independent chatbot definition External Services Deployment Configuration Platform Designer
- 41. How can we be friends: (2) AI for MDE
- 42. AI to simplify development
- 45. Also in MDE land.... Jordi Cabot, Robert Clarisó, Marco Brambilla, Sébastien Gérard: Cognifying Model-Driven Software Engineering. STAF Workshops 2017: 154-160
- 46. Mendix Assist
- 47. OutSystems.ai
- 48. Many other useful applications to explore Modeling bot as virtual assistant (kind of an “expert system”) Model inferencer to discover schema of unstructured data A code generator that mimicks a company programming style A real-time model reviewer ( “pair-modeling” ) A morphing modeling tool that adapts to the user profile (expertise, typical usage)
- 49. AI to skip modeling: An ML-based approach to learn model transformations Loli Burgueño, Jordi Cabot and Sebastien Gerard. An LSTM-Based Neural Network Architecture for Model Transformations
- 50. Model transformations chain Original model … CODE Software code1 refinementst n refinementth Model-to-model Transformation Model-to-text Transformation CODE CODE • Requires learning a new language (the MT Language) • Time consuming • Error prone
- 51. Let’s try to learn the MTs automatically Input Output Training Transforming ML Input OutputML Machine Learning Artificial Neural Networks Deep Networks Recurrent networks LSTM BPMN Petri nets BPMN Petri Net
- 52. MTs ≈ sequence-to-sequence arch • Combine two LSTM for better results • Avoids fixed size input and output constraints
- 53. Tree-to-Tree Encoder-decoder arquitecture InputTree EmbeddingLayer Encoder LSTM network OutputTree ExtractionLayer Decoder LSTM network AttentionLayer InputModel OutputModel
- 54. • Pre- and post-processing required to… • represent models as trees • reduce the size of the training dataset by using a canonical form • rename variables to avoid the “dictionary problem” InputModel (preprocessed) InputTree EmbeddingLayer Encoder LSTM network OutputTree ExtractionLayer OutputModel (non-postprocessed) Decoder LSTM network AttentionLayer InputModel OutputModel Preprocessing Postprocessing Complete picture ML is NOT a fast learner!
- 55. AI for model management: Model similarity with Graph Kernels Robert Clarisó, Jordi Cabot: Applying graph kernels to model-driven engineering problems. MASES@ASE 2018: 1-5
- 57. Model diversity for better testing
- 58. If I ask a solver for 25 solutions, I only get three REALLY different ones!
- 59. Graph-based ML for more diverse results
- 60. Feature-based methods • Graphs summarized as feature vectors • The choice of the features is key • E.g. of features: number of vertices, edges, avg/min/max degree… Easy to use BUT loss of precision
- 61. Kernel-based methods • Goal – Polynomial times – Capture relevant topological information • How – Compare graph structures: path, subtrees, cycles – Different Kernel methods. E.g. random walks Best Kernel depends on the problem!
- 62. Let’s revisit this problem Kernels to cluster the solutions and choose representative examples
- 64. The leader of a dept works in a different dept (error?) Might be missed by feature vector encoding
- 66. A model- based bot used to build models
- 67. Also for BPMN models Joint work with Josep Carmona’s team (go to see From Process models to chatbots on Wed)
- 69. Open Challenges
- 70. All this is very fancy and fashionable BUT
- 72. Where is my data? Every few years, a new initiative to collect models in a repo pops up but in the end… Data augmentation techniques? (mutation, GANs,…)
- 73. • Size of the training dataset • Diversity in the training set • Computational limitations of ANNs – i.e., mathematical operations • Generalization problem – predicting output solutions for input models very different from the training distribution it has learnt from Limits of Neural Networks
- 74. Models Community
- 75. Modeling needs to adapt - AI is here to stay - Let’s see it as an opportunity - To showcase the benefits of modeling - To improve the modeling experience and broaden its user base
- 76. Models are not enough We need (smart) processing capabilities to get the most out of themhttps://modelia.eu/
- 77. Summary
- 78. Exciting times to be a modeler!!
- 80. Back-up slides
- 81. Model representation MODEL ASSOCOBJ c Class ATTS isAbstract name false family OBJ a Attribute ATTS multivalued name false surname OBJ d t Datatype ATTS name String at t c a ASSOC typ e a d t
- 82. Preliminary results • Correctness – Measured through the accuracy and validation loss
- 83. Preliminary results • Performance 1. How long does it take for the training phase to complete?
- 84. Preliminary results • Performance 1. How long does it take for the training phase to complete? 2. How long it takes to transform an input model when the network is trained?
Editor's Notes
- #2: Thank you for the invitation. It’s an honour to be here. I’ll talk about the intersection between these two concepts and how they can help each other I’ll try to cover the state of the art but also to focused on some specific applications we’re working on in my team
- #5: Similar to many of you
- #6: We have a soft side
- #7: They do the work, I take care of the marketing
- #8: Before we talk about the relationship between modeling and AI, we should make sure we have a common understanding of these concepts
- #10: I’ll skip the introduction to MDE since you’re all experts on this here.
- #11: There is always a model and we all model, at least in our head, the real question is whether to make the models explícit And this is linked to a trade-off analysis of the cost of these models vs the benefits of having them explícit. As we’ll see, AI can reduce the cost and maximize these benefits
- #13: We typically use the word AI or even just ML as these are the most fashionable concepts. And we’ll use those in this talk. But I prefer the word cognification, taken from these book.
- #14: Todo tendrá la opción de funcionar con o sin cognificación.
- #15: More than determining the exact limits of each circle the point is make clear that not everything can be reduced to Deep learning
- #20: So we may even wonder whether there is a future for us
- #21: Relationship between humans and AI depending on the type of tasks depending on two dimensions: compassion/empathy and creativity/optimization Best results always in the category where both have some kind of coooperation
- #23: We need to have a better perspective of what AI is (and is not) and how we can use it as a valuable tool
- #24: In fact they are very Good Friends that can help each other a lot. I’ll spend the rest of the talk describing how they can help each ohter.
- #28: Plenty of complexity and heterogeneity. This is the typical scenario where modeling abstractions can help a lot. The AI landscape could beneffit from DSLs, PIM, transformations, …. We’re very Good at this. Huge opportunity for us to contribute to the AI world.
- #29: The
- #30: A way to facilitate the creation of ML models. The easier to use the tool, the more users they get for their infrastructure
- #31: Still, typicall rather low-level and not integrated with the rest of the company Information System.
- #36: Plenty of complexity and heterogeneity. This is the typical scenario where modeling abstractions can help a lot. The AI landscape could beneffit from DSLs, PIM, transformations, …. We’re very Good at this. Huge opportunity for us to contribute to the AI world.
- #38: We need to define the potential intentions of the users, The actions / executions to be done in response And the platform (and services
- #44: All of you old enough may have a feeling of deja vu
- #45: A lot of initiatives going on in the programming IDEs domain
- #46: It shows a customary usage of InputStream in decoding bitmaps. Furthermore, this recommendation demonstrates a good practice to catch the potential IOException when opening the InputStream. If this exception occurs in runtime and is not caught, the app will crash immediately. A responsible engineer should extend the code using this recommendation and handle this exception properly.
- #47: Even powerpoint has an asssitant
- #50: Beyond an autocomplete similar to the Mendix Assist, they also offer another interesting type of suggestions for data models.
- #51: We all can imagine several potential applications And many of these ideas are not really new. It’s just that they are now posible thanks to better data and tools With Antoni we had already a Project 2011 on this topic
- #53: Typical MDD process
- #54: In the end we choose LSTM (Long short Term memory networks) since they are able to remember more context (RNN already remember due to the backpropagation but LSTM remember more long distance) When transforming models you need to have this capacity of remembering parts of the models you’ve seen and matched before
- #55: A typical sequence-to-sequence arch combines two LSTMs for better results especially regarding constraints on the input and output sizes.
- #56: The encoder transform the sequence into a fixed size numeric vector. The decoder reverses the process
- #57: We follow a variation that uses tree embeddings as this is closer to models
- #59: AI can also help in managing and sorting models
- #62: Which is clearly inefficient If we can somehow cluster the solutions and just take one of each type, we would get much more meaningful information!
- #65: And you may even need to define your own
- #66: Which is clearly inefficient
- #67: And you may even need to define your own
- #68: And you may even need to define your own
- #72: I haven’t yet seen this combination much. E.g. apps that include facial recognition components
- #73: More and more mixed. Following up on what we said before, we need lowcode tools to include ML component that help users include inteligent components in their apps
- #74: Possibilites are endless but only if we manage to fix some important challenges first.
- #75: More and more mixed. Following up on what we said before, we need lowcode tools to include ML component that help users include inteligent components in their apps
- #76: We should talk about the elephant in the room
- #77: There is no GitHub for models This makes any initiative related to ML difficult to implement with Good results. Also transfer learning could help here
- #79: But if we only focus on the techniques we’re missing the most important aspect: the people building AND using those models - > social aspects of AI
- #80: One way or the other, modeling needs to adapt to stay relevant
- #81: If we want to move MDE forward we need more than just keep improving modeling techniques
- #83: Need to understand a new community, difficulties in publishing…
- #84: We’re always looking for collaborations. So get involved in our research or stop complaining about useless research done in academia!!!!
- #87: The correctness of ANNs is studied through its accuracy and overfitting (being the latter measured through the validation loss). The accuracy should be as close as 1 as possible while the validation loss as close to 0 as possible. The accuracy is calculated comparing for each input model in the test dataset whether the output of the network corresponds with the expected output. If it does, the network was able to successfully predict the target model for the given input model. The accuracy grows and the loss decreases with the size of the dataset, i.e., the more input-output pairs we provide for training, the better our software learns and predicts (transforms). In this concrete case, with a dataset with 1000 models, the accuracy is 1 and the loss 0 (meaning that no overfitting was taking place), which means that the ANNs are perfectly trained and ready to use. Note that we show the size of the complete dataset but, we have split it using an 80% of the pairs for training, a 10% for validation and another 10% for testing.