Wcre13b.ppt
0 likes•945 views
The document describes a study that aimed to understand how developers spend their effort during maintenance activities. The researchers analyzed change history and interaction data from open source projects to match developers' effort to the complexity of code changes. They found that developers do not necessarily spend more effort on tasks requiring more complex changes. Additional factors like the number of files changed, bug severity, and developers' experience also affected the effort spent.
![How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Metrics
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Developers’ effort
Time Spend: Total duration spent on all files and their
contents
Cyclomatic complexity: Cyclomatic complexity of the
exploration graph
Complexity of the changes
Entropy: How much the changes are scattered between
files [1]
Change distance: How much difference between the
source code before the changes and source code after.
[1] A. E. Hassan, Predicting faults using the complexity of code changes, ICSE
2009
7 / 15](https://image.slidesharecdn.com/wcre13b-131017050159-phpapp02/85/Wcre13b-ppt-11-320.jpg)
![How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Bug Severity
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Bug severity
Bug severity indicates how much a bug can affect the
performance and stability of the system [2]
The resolution time of severe bugs is greater than the
resolution time of less severe bugs [3]
Developers may spent more effort when fixing severe
bugs wrt. less severe bugs
[2] Lamkanfi et al., Predicting the severity of a reported bug, MSR 2010
[3] Panjer. Predicting eclipse bug lifetimes, MSR 2007
12 / 15](https://image.slidesharecdn.com/wcre13b-131017050159-phpapp02/85/Wcre13b-ppt-24-320.jpg)
![How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Bug Severity
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
12 / 15
Bug severity
Bug severity indicates how much a bug can affect the
performance and stability of the system [2]
The resolution time of severe bugs is greater than the
resolution time of less severe bugs [3]
Developers may spent more effort when fixing severe
bugs wrt. less severe bugs](https://image.slidesharecdn.com/wcre13b-131017050159-phpapp02/85/Wcre13b-ppt-25-320.jpg)
![How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Bug Severity
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Bug severity
Bug severity indicates how much a bug can affect the
performance and stability of the system [2]
The resolution time of severe bugs is greater than the
resolution time of less severe bugs [3]
Developers may spent more effort when fixing severe
bugs wrt. less severe bugs
15
663
132
218
12 / 15](https://image.slidesharecdn.com/wcre13b-131017050159-phpapp02/85/Wcre13b-ppt-26-320.jpg)
![How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Developers’ Experience (2/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Developers’ Experience
Developers experience does not reduce their effort
When a program evolves, developers may increasingly
perform tasks on parts of the program on which they
have no previous experience
Consistent result with [4] (#commits) for Mylyn and
PDE project
⇒ NB and NF can assess developers’ experience
[4] Robbes et al., Using developer interaction data to compare expertise
metrics, MSR 2013
14 / 15](https://image.slidesharecdn.com/wcre13b-131017050159-phpapp02/85/Wcre13b-ppt-31-320.jpg)
Wcre13b.ppt
- 1. How Developers Spend their Effort Z´phyrin Soh et al. e Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Towards Understanding How Developers Spend their Effort during Maintenance Activities e e e Z´phyrin Soh, Foutse Khomh, Yann-Ga¨l Gu´h´neuc, e Giuliano Antoniol Department of Computer and Software Engineering ´ Ecole Polytechnique de Montr´al, Qu´bec, Canada e e October 16, 2013 Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
- 2. How Developers Spend their Effort Outline Z´phyrin Soh et al. e Introduction Context and Example Data Effort vs. Complexity Introduction Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Effort vs. Complexity Factors Affecting Effort Conclusion Conclusion 2 / 15
- 3. How Developers Spend their Effort Introduction Z´phyrin Soh et al. e Context and Example (1/2) Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 3 / 15 83 Eclipse bug #1880 Patch #74156 File: 2 LOC : 26 + 18 LOC - 8 LOC
- 4. How Developers Spend their Effort Introduction Z´phyrin Soh et al. e Context and Example (1/2) Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 3 / 15 84 Eclipse bug #1348 #94002 Patch File: 2 LOC : 20 + 19 LOC - 1 LOC
- 5. How Developers Spend their Effort Introduction Z´phyrin Soh et al. e Context and Example (1/2) Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Complexity of the Changes Which change is more complex? 83 Eclipse bug #1880 Patch #74156 File: 2 LOC : 26 + 18 LOC - 8 LOC 3 / 15 vs. 84 Eclipse bug #1348 Patch #94002 File: 2 LOC : 20 + 19 LOC - 1 LOC
- 6. How Developers Spend their Effort Introduction Z´phyrin Soh et al. e Context and Example (2/2) Introduction Context and Example Data Effort Needed to Provide a Patch Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Spend a certain effort to: Explore the program Find relevant program entities Understand entities and make changes 4 / 15
- 7. How Developers Spend their Effort Introduction Z´phyrin Soh et al. e Context and Example (2/2) Introduction Context and Example Data Effort Needed to Provide a Patch Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Spend a certain effort to: Explore the program Find relevant program entities Understand entities and make changes 1. How to estimate the effort spend to provide a patch? 2. Does a complex patch need more effort? 4 / 15
- 8. How Developers Spend their Effort Introduction Z´phyrin Soh et al. e Data Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 5 / 15 Need detailed information Developers’ programming activities Interactions histories ⇒ developers’ effort Changes made to address the tasks Patches ⇒ source code before and after changes
- 9. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Research Question Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 6 / 15 Does the complexity of the implementation of a task reflect developers effort?
- 10. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Metrics Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 7 / 15 Developers’ effort Time Spend: Total duration spent on all files and their contents Cyclomatic complexity: Cyclomatic complexity of the exploration graph
- 11. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Metrics Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Developers’ effort Time Spend: Total duration spent on all files and their contents Cyclomatic complexity: Cyclomatic complexity of the exploration graph Complexity of the changes Entropy: How much the changes are scattered between files [1] Change distance: How much difference between the source code before the changes and source code after. [1] A. E. Hassan, Predicting faults using the complexity of code changes, ICSE 2009 7 / 15
- 12. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (1/2) Introduction Context and Example Data How do we match interactions and patches? Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 8 / 15 2,408 Interactions histories ? 3,395 Patches
- 13. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (1/2) Introduction Context and Example Data How do we match interactions and patches? Effort vs. Complexity Research Question Metrics Matching Results 2,408 Interactions histories ? 3,395 Patches Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Assumption: An interaction is matched to a patch (i.e., the patch is the result of the corresponding interaction) if and only if both are attached to the same bug report, by the same developer at the same date (date/hour/minutes). 8 / 15
- 14. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (2/2) Introduction Context and Example Data Effort vs. Complexity Unbalanced matchings Developers modify files without interacting with them: Research Question Metrics Matching Results Changes not requiring much effort, e.g., propagation of refactoring Factors Affecting Effort Interactions are not collected when performing the task Additional Files Bug Severity Developers’ Experience Conclusion 9 / 15
- 15. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (2/2) Introduction Context and Example Data Effort vs. Complexity Unbalanced matchings Developers modify files without interacting with them: Research Question Metrics Matching Results Changes not requiring much effort, e.g., propagation of refactoring Factors Affecting Effort Interactions are not collected when performing the task Additional Files Bug Severity Developers’ Experience Conclusion 9 / 15
- 16. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (2/2) Introduction Unbalanced matchings Context and Example Data Effort vs. Complexity Developers modify files without interacting with them: Research Question Metrics Matching Results Changes not requiring much effort, e.g., propagation of refactoring Factors Affecting Effort Interactions are not collected when performing the task Additional Files Bug Severity Developers’ Experience Conclusion F1 F3 F5 F2 F4 F6 F7 9 / 15 F1 F2 F3 F8
- 17. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (2/2) Introduction Unbalanced matchings Context and Example Data Effort vs. Complexity Developers modify files without interacting with them: Research Question Metrics Matching Results Changes not requiring much effort, e.g., propagation of refactoring Factors Affecting Effort Interactions are not collected when performing the task Additional Files Bug Severity Developers’ Experience Conclusion F3 F5 F1 F2 F4 F6 F7 F1 F3 9 / 15 F2 F4 F1 F3 F2 F5 F8 F7 F6 F8 F9
- 18. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Matching (2/2) Introduction Unbalanced matchings Context and Example Data Effort vs. Complexity Developers modify files without interacting with them: Research Question Metrics Matching Results Changes not requiring much effort, e.g., propagation of refactoring Factors Affecting Effort Interactions are not collected when performing the task Additional Files Bug Severity Developers’ Experience Conclusion F3 F5 F1 F2 F4 F6 F7 F1 F3 9 / 15 F2 F4 F1 F3 F2 F5 F8 F7 F6 F8 F9
- 19. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Results Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 10 / 15 Effort vs. complexity of the changes 1028 matchings and 217 unbalanced matchings
- 20. How Developers Spend their Effort Effort vs. Complexity Z´phyrin Soh et al. e Results Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Effort vs. complexity of the changes 1028 matchings and 217 unbalanced matchings Developers do not necessary spend more effort on tasks requiring more complex changes Additional Files Bug Severity Developers’ Experience Conclusion Time (sec.) Time (sec.) Cyclomatic Complexity 10 / 15 Cyclomatic Complexity 0.16 0.27 0.31 0.33 Entropy Change distance Entropy Change distance
- 21. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Additional Files Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 11 / 15 Additional Files Exploring files that should not be modified
- 22. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Additional Files Introduction Additional Files Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Exploring files that should not be modified Significantly relevant files vs. additional (useful and accidental) files Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion F1 F2 F4 F7 F3 F5 F6 F9 62% 11 / 15 F8 38%
- 23. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Additional Files Introduction Additional Files Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Exploring files that should not be modified Significantly relevant files vs. additional (useful and accidental) files Effort vs. number of additional files: 0.63 (time) and 0.82 (cyclomatic complexity) Conclusion F1 F2 F4 F7 F3 F5 F6 F9 62% 11 / 15 F8 38%
- 24. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Bug Severity Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Bug severity Bug severity indicates how much a bug can affect the performance and stability of the system [2] The resolution time of severe bugs is greater than the resolution time of less severe bugs [3] Developers may spent more effort when fixing severe bugs wrt. less severe bugs [2] Lamkanfi et al., Predicting the severity of a reported bug, MSR 2010 [3] Panjer. Predicting eclipse bug lifetimes, MSR 2007 12 / 15
- 25. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Bug Severity Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 12 / 15 Bug severity Bug severity indicates how much a bug can affect the performance and stability of the system [2] The resolution time of severe bugs is greater than the resolution time of less severe bugs [3] Developers may spent more effort when fixing severe bugs wrt. less severe bugs
- 26. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Bug Severity Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Bug severity Bug severity indicates how much a bug can affect the performance and stability of the system [2] The resolution time of severe bugs is greater than the resolution time of less severe bugs [3] Developers may spent more effort when fixing severe bugs wrt. less severe bugs 15 663 132 218 12 / 15
- 27. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Developers’ Experience (1/2) Introduction Developers’ Experience Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 13 / 15 NB NF and NLOC (Overall and relevant)
- 28. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Developers’ Experience (1/2) Introduction Developers’ Experience Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 13 / 15 NB NF and NLOC (Overall and relevant) Task T1 NB F1 F3 F5 F2 F4 F6 F7 NF NLOC 0 0 (0) 0 (0) F1 (2 LOC) F2 (5 LOC)
- 29. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Developers’ Experience (1/2) Introduction Developers’ Experience Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience NB NF and NLOC (Overall and relevant) Task T1 Conclusion T2 (Case 1) 13 / 15 NB F1 F3 F5 F2 F4 F6 F7 F3 F4 F5 F5 F7 NLOC 0 0 (0) 0 (0) 1 2 (0) 7 (0) F1 (2 LOC) F2 (5 LOC) F5 F6 NF F3 F4
- 30. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Developers’ Experience (1/2) Introduction Developers’ Experience Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience NB NF and NLOC (Overall and relevant) Task T1 Conclusion T2 (Case 1) T2 (Case 2) 13 / 15 NB F1 F3 F5 F2 F4 F6 F7 F3 F4 F3 F4 F5 F5 F7 F5 F6 F7 0 0 (0) 0 (0) F3 1 2 (0) 7 (0) 1 2 (1) 7 (5) F4 F2 F6 NLOC F1 (2 LOC) F2 (5 LOC) F5 F2 NF F3 F4
- 31. How Developers Spend their Effort Factors Affecting Effort Z´phyrin Soh et al. e Developers’ Experience (2/2) Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion Developers’ Experience Developers experience does not reduce their effort When a program evolves, developers may increasingly perform tasks on parts of the program on which they have no previous experience Consistent result with [4] (#commits) for Mylyn and PDE project ⇒ NB and NF can assess developers’ experience [4] Robbes et al., Using developer interaction data to compare expertise metrics, MSR 2013 14 / 15
- 32. How Developers Spend their Effort Conclusion Z´phyrin Soh et al. e Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 15 / 15 Time (sec.) Time (sec.) Cyclomatic Complexity Cyclomatic Complexity 0.16 0.27 0.31 0.33 Entropy Change distance Entropy Change distance
- 33. How Developers Spend their Effort Conclusion Z´phyrin Soh et al. e Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience Conclusion 15 / 15 F1 Time (sec.) Time (sec.) Cyclomatic Complexity Cyclomatic Complexity 0.16 0.27 0.31 0.33 Change distance Entropy Change distance F2 F4 Entropy F7 F3 F5 F6 F8 F9 62% 38%
- 34. How Developers Spend their Effort Conclusion Z´phyrin Soh et al. e Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience F1 Time (sec.) Time (sec.) Cyclomatic Complexity Cyclomatic Complexity 0.16 0.27 0.31 0.33 Change distance Entropy F7 F3 F5 F6 15 663 132 218 F8 F9 62% Change distance Conclusion 15 / 15 F2 F4 Entropy 38%
- 35. How Developers Spend their Effort Conclusion Z´phyrin Soh et al. e Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience F1 Time (sec.) Time (sec.) Cyclomatic Complexity Cyclomatic Complexity 0.16 0.27 0.31 0.33 Change distance Entropy F2 F3 F4 Entropy F7 F5 F6 Conclusion F8 F9 62% Change distance 38% Task NB 15 663 132 218 T1 T2 (Case 1) 15 / 15 F1 F3 F5 F2 F4 F6 F7 F3 F4 F5 F5 F7 F2 (5 LOC) F5 F6 NF NLOC 0 0 (0) 0 (0) 1 2 (0) 7 (0) F1 (2 LOC) F3 F4
- 36. How Developers Spend their Effort Conclusion Z´phyrin Soh et al. e Thanks for your attention! Introduction Context and Example Data Effort vs. Complexity Research Question Metrics Matching Results Factors Affecting Effort Additional Files Bug Severity Developers’ Experience F1 Time (sec.) Time (sec.) Cyclomatic Complexity Cyclomatic Complexity 0.16 0.27 0.31 0.33 Change distance Entropy F2 F3 F4 Entropy F7 F5 F6 Conclusion F8 F9 62% Change distance 38% Task NB 15 663 132 218 T1 T2 (Case 1) 15 / 15 F1 F3 F5 F2 F4 F6 F7 F3 F4 F5 F5 F7 F2 (5 LOC) F5 F6 NF NLOC 0 0 (0) 0 (0) 1 2 (0) 7 (0) F1 (2 LOC) F3 F4