Experiments on Design Pattern Discovery
Download as PPT, PDF•1 like•818 views
The document describes experiments conducted to discover design patterns from source code. It outlines the approach taken by DP-Miner tool, presents experiment data on four Java systems, and evaluates results by calculating precision and recall values. Benchmarks are lacking for accurately evaluating design pattern discovery techniques.
![Instances Missed Instances manually found but missed by DP-Miner Recall: 89.23% DrawingView SelectionTool [7] Locator LocatorConnector [6] Locator LocatorHandle [5] Locator PolygonHandle [4] Connector ConnectionTool [3] Connector ChangeConnectionHandle [2] Connector LineConnection [1] STRATEGY CONTEXT](https://image.slidesharecdn.com/experiments-on-design-pattern-discovery-52485-26440/85/Experiments-on-Design-Pattern-Discovery-17-320.jpg)
Experiments on Design Pattern Discovery
- 1. Experiments on Design Pattern Discovery Jing Dong and Yajing Zhao Department of Computer Science The University of Texas at Dallas {jdong, yxz045100}@utdallas.edu
- 2. Outline Introduction Related Work Our Approach Experiment Data Result and Discussion Precision and Recall Benchmark Conclusion and Future Work
- 3. Introduction Difficult to understand large computer-based systems No original architecture and design Patterns help on understanding systems Patterns embed future change of systems Need of design pattern discovery since patterns are lost in source code
- 4. Introduction (Cont’d) DP-Miner Experiment data: System source code JUnit JEdit JHotDraw Java.AWT Such experiments lead to benchmarks
- 5. Related Work X Flyweight X Facade X X Chain of Responsibility X X Builder X X X Abstract Factory X X X X X X X X X X Balanyi 2003 X X Command X X X Proxy X X X X X X Composite X X X X X Decorator X X X X Factory Method X X X X Observer X X Prototype X X X X Singleton X X X X Strategy X X X Template Method X X X Visitor X X X X Bridge X X X X Adapter Gueheneuc 2006 Shi 2006 Blewitt 2001 Niere 2002 Heuzeroth 2003 Antoniol 1998 Tsantalis 2006 Authors Tools
- 6. Our Approach – DP-Miner Matrix and Weight Product of Prime Numbers Structural, Behavioral, and Semantic Analysis
- 7. Experiment Data JUnit: A regression test framework that helps developers to implement unit tests in Java. JEdit: A mature and easy-to-use text editor. JHotDraw: A two dimensional graphics frameworks for technical and structured drawing editors written in Java. Java.AWT: A library for developing graphical user interfaces for Java programs.
- 8. Why Those Four Systems as Data? At the time they were developed, the idea of design patterns is already mature and widely applied. Other works on design pattern discovery use one or more of these systems to evaluate their approaches, which allows us to compare and evaluate our experiment results.
- 9. Experiment Data System information Version Number of Classes Number of Files 484 530 6.0 beta 1 JHotDraw 394 1001 4.2 JEdit 93 126 3.8.2 JUnit 345 570 JDK1.4.2 Java.AWT File # Class # Version Systems
- 10. Experiment Result 64 24 6 76 Strategy 0 58 4 JHotDraw 0 24 17 JEdit 3 6 3 JUnit 3 65 21 Java.AWT Composite Bridge Adapter Systems
- 11. Experiment Result – Java.AWT Results for Each Analysis Phase N/A 76 65 N/A Semantic Analysis 3 3 92 Composite 76 76 100 Strategy 65 76 100 Bridge 21 21 57 Adapter Final Result Behavioral Analysis Structural Analysis Systems Java.AWT
- 12. Experiment Result – JUnit Results for Each Analysis Phase N/A 6 6 N/A Semantic Analysis 3 6 6 3 Final Result 3 9 Composite 6 6 Strategy 6 6 Bridge 3 15 Adapter Behavioral Analysis Structural Analysis Systems JUnit
- 13. Experiment Result – JEdit Results for Each Analysis Phase N/A 24 24 N/A Semantic Analysis 0 0 0 Composite 24 24 33 Strategy 24 24 33 Bridge 17 17 80 Adapter Final Result Behavioral Analysis Structural Analysis Systems JEdit
- 14. Experiment Result – JHotDraw Results for Each Analysis Phase N/A 64 58 N/A Semantic Analysis 0 0 0 Composite 64 64 74 Strategy 58 64 74 Bridge 4 4 27 Adapter Final Result Behavioral Analysis Structural Analysis Systems JHotDraw
- 15. Observations Behavioral analysis phase only deals with a small number of classes, since it’s based on the result of structural analysis phase. Significant reduction of the candidates between structural and behavioral analysis of Adapter and Composite pattern than those of Bridge and Strategy patterns, since the former have more behavioral characteristics than the latter do. Result of structural and behavioral analysis of the Bridge and Strategy patterns are the same, since they have same structural and behavioral characteristics and only differ in their intents.
- 16. Recovery Precisions for JHotDraw Manually checked the results generated by our tool and see whether they are real pattern instances. TP: True Positive FP: False Positive 100% 0 0 Composite 90.63% 6 58 Strategy 91.38% 5 53 Bridge 100% 0 4 Adapter Precision FP TP JHotDraw
- 17. Instances Missed Instances manually found but missed by DP-Miner Recall: 89.23% DrawingView SelectionTool [7] Locator LocatorConnector [6] Locator LocatorHandle [5] Locator PolygonHandle [4] Connector ConnectionTool [3] Connector ChangeConnectionHandle [2] Connector LineConnection [1] STRATEGY CONTEXT
- 18. Reasons of Discrepancy Flexibility of design pattern Composite pattern: Collapse both the Component and Composite classes into a single class or not. Adapter pattern: Full matching all three roles, i.e. Target, Adapter, and Adaptee, or partial matching of only Adapter and Adaptee. Object-oriented programming languages provide special language constructs that greatly simplify the implementation of a design pattern. E.g., Java provides LinkedList, ArrayList, HashMap, and Hashtable, which make the implementation of aggregate elements in patterns easy, but the detection harder.
- 19. Benchmark Correct number of pattern instances and their locations are generally not available. Lacking benchmarks is the main impediment Hard to evaluate and compare design pattern discovery techniques. Hard to judge whether an approach discovers all instances and whether the discovered ones are correct. To calculate the precision and recall of a pattern matching result, it’s essential to know The number of correct pattern instances The locations of correct pattern instances JHotDraw pattern benchmark
- 20. Conclusion We did a series of experiments on design pattern discovery from open-source systems using DP-Miner Patterns concerned: Adapter, Bridge, Strategy, Composite Experiment data: Java.AWT, JUnit, JEdit, JHotDraw We compared our experiment results with others and found several discrepancies. We analyzed the issues and discussed possible reasons. We argue for benchmarks for design pattern discovery.
- 21. Future Work Improve our tool. Experiment on other systems Manually check results of other systems and perfect the benchmark Research on other object-oriented languages, such as C++.
- 22. Thank You!
- 23. Questions?
- 24. Overall Architecture of Our Approach System Design Pattern match Source Code Intermediate Representation
- 25. Structural Analysis – Matrix
- 26. 1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button
- 27. 1 1 1 1 7 1 1 1 1 TextComponent 1 1 1 1 7 1 1 1 1 Scrollbar 1 1 1 1 7 1 1 1 1 Label 1 1 1 1 35 1 1 1 1 Container 1 1 1 5 1 1 1 1 1 Component 1 1 1 1 7 1 1 1 1 Choice 1 1 1 1 7 1 1 1 1 Checkbox 1 1 1 1 7 1 1 1 1 Canvas 1 1 1 1 7 1 1 1 1 Button TextComponent Scrollbar Label Container Component Choice Checkbox Canvas Button 1 35 1 Composite 1 1 1 Component 1 7 1 Leaf Composite Component Leaf
Editor's Notes
- #2: Good afternoon, my name is Yajing Zhao. I will present our paper, Experiments on Design Pattern Discovery.