Sources of errors in distributed development projects implications for collaborative tools
Download as PPTX, PDF2 likes462 views
The document presents a study on errors in distributed development projects by analyzing data from 209 software projects at a multinational company, emphasizing the impact of geographic dispersion, technical dependencies, and time pressure on product quality. It discusses various factors influencing errors and provides a model to measure these influences, highlighting the significance of coordination tools in improving awareness and reducing defects in collaborative settings. Results indicate that uneven distribution of engineers and concurrent task execution are major contributors to errors, prompting implications for enhancing collaborative tools to support coordination in large-scale projects.
Sources of errors in distributed development projects implications for collaborative tools
- 1. Marcelo Cataldo, Robert Bosch LLC Presented By Bhagyashree Deokar Sources of Errors in Distributed Development Projects: Implications for Collaborative Tools
- 2. Outline of Paper Introduction Research Setting Sources of Errors in Distributed Project Measure Results Limitation Implication For Collaborative Tools
- 3. Introduction Success of Product Development Project: Market Performance Of The Product Project Cycle Time Efficiency of the Development Process Product Quality
- 4. Discussion What are the barriers these people would have faced in order to conduct this meeting?
- 5. Introduction Past research focuses on: Process improvement activities Experience Dimensions of geographic dispersion Technical dependencies Time pressure Does not consider relative significance of each of the factors
- 6. Research Setting Data from a multi-national company in the area of embedded systems Products consist of physical elements which are managed by a large and complex software Access to modification request repository and version control system 209 software development projects between 2003 and 2008
- 7. Discussion Do you agree with below statement considering today’s advance technology – “Distributed development organizations face significant challenges in terms of information sharing and integration because of the detrimental effects of distance. Moreover, the various dimensions of geographic dispersion have a differentiating and additive effect on the ability of the distributed development organizations to effectively communicate, coordinate and share information, and consequently, on the quality of the developed products “
- 8. Sources of Errors in Distributed Project Types of experience Dimensions of geographic dispersion Technical properties of the product Project’s time pressure
- 9. Activity 1 Divide into four groups. Identify factors that impact product quality from the category of your group
- 10. Measures Outcome Variables Dependent on the defects derived from system testing and integration testing Development process – two phase : Implementation phase : requirements engineering, design, implementation, module-level testing, fixing of module-level defects Integration phase : integration and system testing Number of defects identified in the second phase is a good indicator of product quality
- 11. Measures Experience Modification request (MR) Average MR Experience : average number of MRs that the project members worked on prior to the focal project Average Component Experience : average number of times that the project members modified the components that need to be changed in the focal project prior to the beginning of the focal project Average Shared Experience
- 12. Measures Geographic Dispersion Spatial distribution : Euclidean distance between each pair of location Temporal distribution : difference between time zone Configuration dispersion : People dispersion Number of Locations and number of Regional Units Number of Regional Units is more precise
- 13. Measures Technical Dependencies Interface among components is a major source of errors In-flow technical dependencies : Number of interfaces that components modified in the project use from components that are not modified in the project but that are part of the final system Out-flow technical dependencies : Interfaces exported by components that are modified in the project and used by components not changed in the project but are part of the final system
- 14. Measures Project Time Pressure Planned delivery dates = Customer requested delivery dates Number of overlapping activities Tasks Temporal Execution : standard deviation of the number of tasks completed in each month High values associated with uneven distributions indicate time pressure in particular duration with a high number of tasks to be completed
- 15. Measures Control Measure Size: sum of the number of lines of source code added, deleted or modified Process Maturity: level of discipline and sophistication of the development organization and the supporting processes Complexity = Additional Factors: Number of modification requests, number of developers
- 16. Measures Model Number of defects = count variable Negative Binomial Regression Model is appropriate in this research setting
- 17. Discussion Did you come across any of the measures which we discussed today in your past industry or academic level project experience? Did you use any tools available in market to reduce the errors due to those measures ? Describe the functionality, experience using the tool from your experience.
- 18. Results Variance Inflation Factor Variance Inflation Factor above 10 -> High Multi-collinearity Variance Inflation Factor above 5 -> Need to be handled carefully
- 19. Results : VIF
- 20. Results VIF based models Model 1 included all factors Model 2 - average component experience, number of modification requests, spatial distribution Model 3 - number of new features, number of developers, number of regional units
- 21. Results Incident Rate Ratio (IRR) Indicate the change in the estimated counts of the outcome variable for a unit increase in the independent variable holding the other variables constant Greater than or equal to 1 indicates High Value : positive relation between dependent and independent variables Less than 1 indicates increase in independent variable with decrease in dependent variable
- 22. Results : IRR
- 23. Results IRR Based models Model 1 Baseline model consist of control factors Model 2 Increase in Average MR experience and Average Shared Experience decreases errors Model 3 Higher number of outflow technical dependencies indicates poorer quality Model 4 Higher number of locations and uneven people make dispersion higher Uneven people dispersion has more impact than higher number of locations
- 24. Results : IRR
- 25. Results IRR is dependent on the scale of independent measure impact of each particular factor by understanding the changes in quality for the full range of variation of each independent factor Independent Measures % of Defects Task Temporal Execution 47.1 % People Dispersion 45.2 % Number of Locations 35.7 % Flow of Technical Dependencies 28.3% Temporal Distribution 19.2% Out- Shared Experience - 1.4% MR Experience - 29.2%
- 26. Discussion Is this statement valid in agile methodology & why? “Our analyses of 209 software development projects in a large multination organization showed that two factors, time pressure (measured as concurrent execution of tasks) and uneven distribution of engineers across locations, were the two most significant sources of errors”
- 27. Results Factors improving the awareness and co-ordination capabilities of collaborative tools: Project time pressure Technical dependencies that cross project boundaries Dimensions of distribution
- 28. Limitation Not able to collect interaction and co-ordination data Not able to access data repositories from the previous generations Did not include 31 projects which has developers working on multiple projects
- 29. Implication for Collaborative Tools Supporting Coordination and Awareness in Large- Scale Development Organizations: Supply the pattern information to tool that will provide co-ordination and awareness capabilities specific to context Awareness beyond Traditional Boundaries: Use social computing tools to build “social ties” among the members of the distributed teams
- 30. Relevance To Previous Paper Presentation “Let’s Go to the Whiteboard: How and Why Software Developers Use Drawings” Distributed projects cannot take advantage of whiteboards for understanding problem through visualization and creation of drawings collaboratively
- 31. Discussion From this research study and considering current technology trends, what are the things that should be included in collaborative tools in order to reduce errors and improve product quality?
- 32. Tool : World View
- 33. Similar to Ensemble Salesforce Chatter https://www.youtube.com/watch?v=tv4hqseuD QA
- 34. Thank You
Editor's Notes
- #5: Past research discusses impact on product development of factors such as experience, dimensions of geographic dispersion, technical dependencies and time pressure Extends this study by considering the relative significance of each of those factors
- #11: Customer failure is not a good measure because that is not so consistent Zero defect product needs to be shipped
- #12: Average MR Experience captures the average number of MR that the project members worked on prior to the focal project. Average Component Experience captures the average number of times that the project members modified the components that need to be changed in the focal project prior to the beginning of the focal project. Experience working with other project members, captured by the Average Shared Experience variable, was measured as the number of MRs where both persons in each dyad in a project worked together, averaged across all dyads.
- #13: Locations and number of developers per location was extracted from the modification requests reports associated with each project. Number of Regional Units captures more precisely the impact of major organizational barriers for distributed work manifested in difference across development processes, individual and team-level goals, managerial philosophies and labor-related policies and laws People dispersion is distributions of project members across locations Number of Locations and the Number of Regional Units that were involved in a project
- #14: ment organization kept architectural description files in the version control system. Those files were used to generate the interfaces among the different components of the software system. We use the information in such files to collect measures of technical dependencies.
- #15: number of overlapping activities - representation of the project’s time pressure pattern
- #16: Size: sum of the number of lines of source code added, deleted or modified as part of the development activities of the project. Complexity : ratio of the number of technical dependencies of the system that are associated with the components of the product modified by the project over the total number of technical dependencies of the system. Each modification request consisted in one or more transactions in the version control system that impacted one or more files associated with a particular software component. From those transactions in the version control system, A third important factor that leads to quality problems is the complexity of the product [29]. Two measures are traditionally used: a subjective evaluation of complexity based on the assessment of experts [29, 48] or an objective evaluation based on the number of relationships among components of the system [32, 46]. Unfortunately, all the projects in our dataset are based on the same product so the traditional measures of complexity would not vary across Project Number of Modification Requests, the number of those MRs that were related to new features to be implemented in the product (Number of New Features variable) and the Number of Developers that participated in the tasks associated with each project
- #17: The Poisson regression model is a basic model for count outcomes. However, the Poisson regression model has the assumption that the conditional mean of the outcome variable is equal to the conditional variance. Our dependent variable has a variance several orders of magnitude larger than its mean (see Table 1). Then, a Negative Binomial regression model would be more appropriate.
- #22: Incident Rate Ratio (IRR) exponentials of the regression coefficients and they indicate the change in the estimated counts of the outcome variable for a unit increase in the independent variable holding the other variables constant higher than or equal to 1 indicates High Value : positive relation between dependent and independent variables Lower than 1 indicates increase in independent variable increase decrease in dependent variable
- #24: Model 1 – Baseline model consist of control factors Model 2 – increase Average MR experience and Average Shared Experience decreases errors Model 3 - higher number of outflow technical dependencies indicates poorer quality. Because of complex relationship between the technical dimension defined by the properties of the product and the social and organizational dimensions defined by the projects and the development organization as a whole Model 4 - higher Number of locations and uneven people dispersion higher, low product quality uneven people dispersion has more impact than higher number of locations. Impact of temporal distribution was significant Model 5 – Uneven distribution of Task Temporal execution over duration of project indicates poor product quality
- #26: Then, considering just the IRRs as a way to comparing the relative impact of the sources of various sources of errors on product quality is not appropriate. Therefore, we also calculated the discrete change of the expected number of defects for a range of values of each statistically significant factor. We considered the [minimum, maximum] interval of each independent variable while maintaining the other variables fixed at the mean. In this way, we can get a better sense of impact of each particular factor by understanding the changes in quality for the full range of variation of each independent factor. Our results showed that the factor with the highest impact was Tasks Temporal Execution where a change from the minimum value to the maximum one resulted in an increase in the expected number of defects of 47.1%. The other factors had the following interval change percentages: People Dispersion = 45.2%, Number of Locations = 35.7%, flow Technical Dependencies = 28.3%, Temporal Distribution = 19.2%, Out- Shared Experience = -1.4%, and MR Experience = -29.2%. The measures of experience have a negative percentage values because higher levels of experience are associated with a reduction in the expected number of defects.
- #28: Current research work on awareness of technical dependencies focuses primarily on the particular piece of software, organizational unit or work item technical dependencies that cross the boundaries of projects highlight the importance of considering coordination and awareness for larger scale project within the development organization.
- #29: which would have allowed us to assess more accurately the ability of the development organization to share and integrate information The interviewees indicated that it was common practice to have periodic (e.g. once a week) status meeting. Email and telephone conversations were the most common information sharing mechanisms. All members of the development organization had access to the modification request tracking system, version control system and wide range of wikis with product information. The use of instant messaging type of tools was not allowed
- #30: teams, locations and even the individuals associated with those tasks along with their experience distribution measures used in our analyses technical dependencies among the constituent components of a system can be brought into the picture when tasks contain information about the different parts of a system affected by the task.