Lviv PMDay: Дов Німрац Як зробити процес Continuous Integration ефективним
Download as PPTX, PDF1 like228 views
The document discusses three primary tasks for effective continuous delivery (CD): breaking down monolithic codebases, designing an optimal test suite, and setting up a deployment pipeline. It emphasizes that CD requires decomposing large codebases into separate services and libraries to enable faster builds and testing. The test suite should follow a testing pyramid approach with more automated unit and integration tests. An effective deployment pipeline automates testing, builds and releases through distinct stages without shadow steps.
Lviv PMDay: Дов Німрац Як зробити процес Continuous Integration ефективним
- 1. CI is essential but not sufficient for an effective application delivery workflow Dov Nimratz 2016
- 2. E-Mail: [email protected] Skype: dovnmr About me • 30 years in R&D • 17 years in Israel HighTech • ECI, Telrad, RAD, Audiocodes companies • HW, SW, Mechanical design engineer • Project & Product Manager • Business developer for EMEA & CIS countries • 22 publications, US patent • Counseling & SW development teaching
- 3. E-Mail: [email protected] Skype: dovnmr What this speech is about? •Immutable concepts in R&D •Major CD fails •Solutions
- 4. E-Mail: [email protected] Skype: dovnmr Basement - SDLC The smaller radius creates smaller amplitude. It closer to the line.
- 5. E-Mail: [email protected] Skype: dovnmr SDLC pyramid view Requirement Analysis Design Implementation Testing Evaluation Refactoring V - Model Bug fixing
- 6. E-Mail: [email protected] Skype: dovnmr Monitoring and testing Infrastructure configuration CI Server CD parts Version control
- 7. E-Mail: [email protected] Skype: dovnmr Three primary tasks for effective CD •The trouble with monolithic codebases and approaches to break it down •Designing the test suite for optimal feedback •Setting up a deployment pipeline as the backbone of CD
- 8. E-Mail: [email protected] Skype: dovnmr Part I - monolithic codebases
- 9. E-Mail: [email protected] Skype: dovnmrMonolithic code immediate symptoms – «Adagissimo» •Sluggish build •App start-up time •Slow Auto test
- 10. E-Mail: [email protected] Skype: dovnmr Harmful impact - Big SDLC circle radius
- 11. E-Mail: [email protected] Skype: dovnmr Harmful impact New devs penetration barrier x 100
- 12. E-Mail: [email protected] Skype: dovnmr Harmful impact Less ownership for team members – ”broken build not mine”
- 13. E-Mail: [email protected] Skype: dovnmr Harmful impact Hacks grows exponential
- 14. E-Mail: [email protected] Skype: dovnmr Monolithic code, Harmful impact • Slow build bushes devs not to do it even locally before ci • Only night CI server build or rarely • Time cost for tasks are Up to (x10 more) x N devs • Many manual tests required • This creates a lot of friction in the pipeline.
- 15. E-Mail: [email protected] Skype: dovnmr Solution - Decomposing the codebase Service 1 Service 2 Service 2 REST DLL 1 DLL 2 DLL 3 BUILD Compile Flags The Entropy principle works in the development After build
- 16. E-Mail: [email protected] Skype: dovnmr The Scale Cube Scale by cloning Scale by split different services Resource duplication Functional decomposition Data Partitioning Scale by splitting similar services Z X Y
- 17. E-Mail: [email protected] Skype: dovnmr The Scale Cube Z axis • Scaled as X but for data subset • Commonly used: • Scale DB by primary key • Treatment customers by SLA • Scale DB or Application • Combine result in later stage • Each server only deals with a subset of the data. • Improves utilization (memory usage, I/O), reliability, performance • But only Y-axis scaling solve application complexity grow
- 18. E-Mail: [email protected] Skype: dovnmr Part II - Designing the test suite
- 19. E-Mail: [email protected] Skype: dovnmr Testing & Problem areas Requirement Analysis Design Implementation Testing Evaluation Technical Problems Architectural or Business Problems Bug fixing Refactoring
- 20. E-Mail: [email protected] Skype: dovnmr Testing pyramid Automated Unit Test Code reading Auto API test Auto Integration Auto component Auto GUI Test Manual Technical Problems Architectural or Business Problems
- 21. E-Mail: [email protected] Skype: dovnmr Ice-Cream cone Anti-pattern
- 22. E-Mail: [email protected] Skype: dovnmr AQA daily mail
- 23. E-Mail: [email protected] Skype: dovnmr Part III - Setting up a deployment pipeline
- 24. E-Mail: [email protected] Skype: dovnmr General rule for ISO 9001 Design how, write it, do it.
- 25. E-Mail: [email protected] Skype: dovnmr Deployment Pipeline Commit Stage Auto Acceptance Testing Auto Capacity Testing Manual Testing Release Compile Unit test Analysis Build Functional Testing Performanc e Load test Business logic Two button Deploy
- 27. E-Mail: [email protected] Skype: dovnmr TBD TRUNK Release 1.0 MB Release 2.0 MB Release 1.1 MB
- 28. E-Mail: [email protected] Skype: dovnmr TBD TRUNK Release 2.0 MB Release 1.0 M B *Release 1.0 M Release 1.1 MB M M
- 29. E-Mail: [email protected] Skype: dovnmrCommon anti-patterns with branch based workflows •Long-lived feature branches •Branch per environment (Dev, QA, Staging and Production) •Shadow process flow steps - limiting work in process (WIP)
- 30. E-Mail: [email protected] Skype: dovnmr SDLC pipe line Build Test & release
- 31. E-Mail: [email protected] Skype: dovnmr Conclusions – Effective CD when: • Short Dev Life Cycle • Non monolithic code with Library and Services • Correct test suite • Trunk Base Development • Dev pipeline without shadow steps
Editor's Notes
- #4: Резюме проблем R&D
- #8: Maintain a Single Source Repository. (все знают/хранят файлы) Automate the Build (любой с новой машины должен иметь возможность собрать систему + билд только изменения + ключи сборок) Make Your Build Self-Testing (Test Driven Development (TDD) + self code reading) Everyone Commits To the Mainline Every Day (ci -> resolve -> build –with test -> co with comment why/not what)
- #9: Continuous Integration is an Attitude, Not a Tool (не всегда с сервером CI ) Fix Broken Builds Immediately ( pending head - delay commit in TeamCity & Jemkins что бы не остановить весь тим) Keep the Build Fast (deployment pipeline / parallel build)
- #10: Test in a Clone of the Production Environment (different OS …, -> virtualization) Everyone can see what's happening (CI is all about communication)
- #13: Как только что то медленно – симптом монолитности кода.
- #14: ?????????
- #17: Дом для хаков
- #22: http://microservices.io/patterns/microservices.html tp://martinfowler.com/bliki/DevOpsCulture.html
- #23: http://microservices.io/articles/scalecube.html X-axis scaling clones – load balancing = работа однотипных серверов под управлением сортировщика. Не решает проблему усложнения разработки У-axis service split – splits the application into multiple, different services. Admin portal, Mail serever Z-axis – clone as X, but treat different SLA or portioned DB
- #24: drawback is increased application complexity. doesn’t solve the problems of increasing development and application complexity. To solve those problems we need to apply Y-axis scaling.
- #33: ISO 9001
- #35: 2 buttons: Any one Simple script (not from IDE) Restore previous
- #41: ACC (Accelerator code developer) - The build system is simply a collection of scripts - DEV среда – сборка прошла без ошибок. - STAGE среда – сборка была установлена на DEV среде и unit тесты прошли успешно. - PROD среда – сборка прошла тестирование на STAGE среде, есть не более 5% minor багов, major багов нет, QA Lead и Dev Lead поставили Confirm билду о готовности к PROD среде.