![Willy[-Peter] Schaub
Program Manager | Developer Ops Canada
willys@microsoft.com
@wpschaub](https://image.slidesharecdn.com/devconfpublic-moving65000microsoftiestodevopswithvisualstudioteamservices-180502232007/85/Devconf-Moving-65000-Microsofties-to-DevOps-with-Visual-Studio-Team-Services-2-320.jpg)
Devconf - Moving 65000 Microsofties to DevOps with Visual Studio Team Services
- 1. Moving 65,000 Microsofties to DevOps with VSTS Willy-Peter Schaub
- 2. Willy[-Peter] Schaub Program Manager | Developer Ops Canada [email protected] @wpschaub
- 3. I work here I was born here I grew up here
- 4. Truk Lagoon Semiahmoo Bay Protea Banks Hawaii Wondergat Aliwal Shoal Truk Lagoon
- 5. DevOps Visual Studio Application Insights Any language, Any Platform
- 6. One Engineering System using VSTS
- 7. What is DevOps? Increase flow of value Shorten cycle times Continuously Improve
- 8. Our Definition of Done Customer Focused
- 9. STARTOF OUR JOURNEY August 2010 Sprint 1 Extensions Nov 2015 Sprint 931ES June 2014 Sprint 67 GVFS June 2016 Sprint 102 Ranger community adopted VSTS April 2011 VSTS April 2014 Sprint 64 over night ~destination! continuous innovation!
- 10. Five habits we’ve learned so far ++ ++ ++
- 11. Five habits we’ve learned so far
- 12. Listen to our customers Customer Focused
- 13. Collect data broadly (but carefully)
- 14. But measure what’s important (KPI’s) Customer Focused • Original estimate • Completed hours • Lines of Code • Team capacity • Team burndown • Team velocity • # of bugs found Things we don’t watch • Acquisition • Engagement • Satisfaction • Churn • Feature Usage Usage • Time to Detect • Time to Communicate • Time to Mitigate • Customer Impact • Incident Prevention Items • Aging Live Site Problems • SLA per Customer • Customer Support Metrics Live Site Health • Time to Build • Time to Self Test • Time to Deploy • Time to Learn Process Velocity
- 15. Maximize learning and influence value Customer Focused Validatedlearning Deployment frequency PROVEN DISPROVEN VALIDATE business service value
- 16. Feature Flags – fine tune user experience Customer Focused if ( flag ) else
- 17. Five habits we’ve learned so far
- 19. Be Transparent Production First Mindset
- 20. Alerting is the key to fast detection Production First Mindset Before • Redundant alerts for same the issue • Needed to set right thresholds and tune often • Stateless alerts contributed to further noise After • Every alert must be actionable and represent a real issue with the system. • Alerts should create a sense of urgency – false alerts dilutes that • Use alerts to auto-dial the DRIs.
- 22. Tracking Deployments to Production
- 24. Security Mindset Production First Mindset Double blind test Full disclosure at or near end vs. Share tactics & lessons learned Continued evolution Assume Breach - Use War Games to the learn attacks and practice response
- 25. Five habits we’ve learned so far
- 26. Agile at Scale with Aligned Autonomy Team Autonomy & Enterprise Alignment Alignment within the business Team autonomy “Let’s try to give our teams three things…. Autonomy, Mastery, Purpose” Anarchy
- 28. Team Autonomy & Enterprise Alignment
- 29. Roles Team Autonomy & Enterprise Alignment Program Management Dev Test
- 30. Roles Team Autonomy & Enterprise Alignment Program Management Engineering
- 31. Yes, there are other roles… Team Autonomy & Enterprise Alignment Program Management Engineering Service DeliveryUX UE Service Delivery is integrated directly into our organization.
- 32. Teams Team Autonomy & Enterprise Alignment Program Management is responsible for: WHAT we’re building, and WHY we’re building it Engineering is responsible for HOW we’re building it, and that we’re building it with QUALITY
- 33. Teams Physical team rooms Cross discipline 10-12 people Self managing Clear charter and goals Intact for 12-18 months Own features in production Own deployment of features
- 34. Instead of Horizontal… Team Autonomy & Enterprise Alignment UI API Data
- 35. We strive for Vertical Team Autonomy & Enterprise Alignment UI API Data
- 36. Employee choice, not manager driven Typically <20% change, but 100% get to make a choice Cross-pollinate talent and micro-culture Sticky Note Exercise - Self Forming Teams
- 37. Sprint 3 weeks confident 1 Plan 3 sprints thoughtful 3 Season 6 months hopeful 6 Planning Team Autonomy & Enterprise Alignment Scenario 18 months asprirational Teams are responsible for the detail Leadership is responsible for driving the big picture
- 38. Planning Team Autonomy & Enterprise Alignment Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 39. Planning Team Autonomy & Enterprise Alignment Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 40. Planning Team Autonomy & Enterprise Alignment Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 41. Planning Team Autonomy & Enterprise Alignment Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 42. Traditional Planning Team Autonomy & Enterprise Alignment TIME VALUE
- 43. Plan, Deliver, and Learn Team Autonomy & Enterprise Alignment TIME VALUE
- 44. Let’s Compare Team Autonomy & Enterprise Alignment
- 45. What we accomplished 3 week sprints Team Autonomy & Enterprise Aligmen Progressive Deployment Week 1 Week 2 Week 3 Week 1 Week 2 Week 3Week 2 Week 3 Sprint 98Sprint 97 Sprint 99 The sprint plan 930!
- 46. Sprint mails
- 47. Team Chats Team Autonomy & Enterprise Alignment SpringFallSpring Fall 3 sprints
- 48. Direct. No “lost in translation”. Team Autonomy & Enterprise Alignment
- 49. Experience Reviews Team Autonomy & Enterprise Alignment SpringFallSpring Fall
- 50. Benefits Team Autonomy & Enterprise Alignment
- 51. Five habits we’ve learned so far
- 52. engineers on your team# Bug Cap Shift Left Quality 5 ?x =
- 53. Bug Cap Shift Left Quality Rule: If your bug count exceeds your bug cap… stop working on new features until you’re back under the cap. 5 50x =10
- 54. Code Test & Stabilize Code Test & Stabilize Code Complete Planning Before – Debt Cycle Shift Left Quality
- 56. • Tests that anyone can run anywhere (inc production) • Shifted to unit tests from automated functional tests • Core tests run before pull request • Fast and 100% reliable build and test is critical • Rolling tests run after commit Test Portfolio - Shift Left from Integration to Unit
- 57. Pull Requests
- 58. Release Flow Branching Structure Shift Left Quality LSI
- 59. Five habits we’ve learned so far Customer Focused Production First Mindset Team Autonomy + Enterprise Alignment Shift Left Quality Infrastructure as Flexible Resource
- 60. From Labs to VM’s to DevTest Labs
- 61. Multiple Data Centers with incremental roll out Shared Platform Services (SPS) North Central TFS SU1 North Central AT AT AT JA JA JA Blob TFS SU7 Australia TFS SU0 West Central Containerized Services
- 62. Architecture Modernization • • • • • • If you are starting out today and cloud native, consider PaaS, Service Fabric + Azure Functions • We need to ship same code to on-prem & cloud
- 64. Our DevOps Transformation – the story so far After • 3-week sprints • Vertical teams • Team rooms • Continual Planning & Learning • PM & Engineering • Continual customer engagement • Everyone in master • 8-12 person teams • Publicly shared roadmap • Zero debt • Mockups in PPT • Inner source • Flattened organization hierarchy • User satisfaction determines success • Features shipped every sprint Before • 4-6 month milestones • Horizontal teams • Personal offices • Long planning cycles • PM, Dev, Test • Yearly customer engagement • Feature branches • 20+ person teams • Secret roadmap • Bug debt • 100 page spec documents • Private repositories • Deep organizational hierarchy • Success is a measure of install numbers • Features shipped once a year
- 65. Lastly the DevOps Ranger Transformation DevOps even works for a part-time open source community After • 2-5 person teams • 0.25 program managers • Automated CI • Automated CD • 3-5 sprints cadence • 3-week sprints • Proactive telemetry • Minutes to days to resolve issues • Minutes to build • Minutes to release Before • 10-15 person teams • 2 program managers • Manual and error prone builds • Manual and error probe releases • 6-12 sprint cadence • 1 month sprints • Issues detected by users • Days to weeks to resolve issues • Hours to build • Days to release
- 66. Quick Reference Posters – ping me if you’re interested in a digital copy!
- 67. Real-world demo
- 68.
- 69. /* THANK YOU*/ WILLY SCHAUB [email protected] @wpschaub aka.ms/devops aka.ms/devopsassessment aka.ms/vsarpublications @VSTS http://www.devconf.co.za/