02 - Build and Deployment Management
Download as PPTX, PDF13 likes11,178 views
The document discusses build and deployment management. It covers why builds and deployments need to be managed, different build types, build phases, and popular build tools like Ant, Maven, Make, and MSBuild. Managing builds is important for large, complex applications to ensure the build process is complete, repeatable, informative, schedulable, and portable. The key difference between a build and deployment is that a build creates installation artifacts while deployment installs the application.
02 - Build and Deployment Management
- 1. BUILD AND DEPLOYMENT MANAGEMENT
- 2. PURPOSE OF CURRENT TRAINING Why do we need to manage builds Theory and practice of build and deployment management Why do we need to manage deployments 2
- 3. CONTENTS Build management What is build? Why do we need to build? Build types. Tools and their specifics: Ant, Maven, make, etc Tips Deployment management Build vs deployment Why do we need to deploy? How to deploy and when? Tools that might be useful 3
- 5. WHAT IS BUILD From Process To of conversion run at the end-user Source code Artifacts workstation 5
- 6. WHY DO WE NEED TO BUILD? Build RESULT OF Configuration defined set of components having specific versions components = functional units components and their versions are chosen to meet specific objectives or tasks Main objectives 6 Functionality Performance
- 7. WHY DO WE NEED TO BUILD? Conclusion. Why we need to build is: To incorporate additional functionality (feature) To reach defined level of performance (as an example of non-functional requirement) And what about data? Data has its own lifecycle Which is incorporated into the database integration process 7
- 8. WHY DO WE NEED TO MANAGE BUILDS? When application grows in size, it becomes more complex As a result, it turns out that routine daily activities require additional management Organizing build process is one of such activities Therefore, build process also requires management What means complex application? From configuration management point of view, most complex application incorporate all possible variant 8 properties
- 9. BUILD VS VARIANT Build: install.msi Variant: AppName-1.1.8.en.x32_beta.msi Real world example: HTC_Touch_Diamond2_WM6.5_RUU_Topaz_S_HT C_RUS_2.16.411.2_Radio_Sign_Topaz_61.44 tc.25.32_4.49.25.17_Ship.exe 9
- 10. CONNECTION BETWEEN CONFIGURATION AND VARIANTS Functional requirements for the variant being built #1 #2 #3 … #N version Maturity Platform variant OS Localization build Build type configuration + source code changes (changeset) 10 static runtime
- 11. WHAT IS VARIANT Variant is the result of the build characterized by the set of specific properties and their values 11
- 12. VARIANT PROPERTIES Version Revision Date Build type: debug or release Operating system, hardware platform (x32, x64, …) Localization (en, ru, ua, …) Platform (development, test, integration, production) Maturity (alpha, beta, release candidate, stable) Issue number (fix#387) License (MIT, GPL, BSD, CC, …) Project ID (EPM-SIGN) + Discipline ID (PD) 12
- 13. EXAMPLE. PLAIN DOCUMENT Build #2 Properties: • Language (en) • Date (24-12-2010) • Revision (4) Ctrl + N Ctrl + S Ctrl + S Ctrl + S Build #1 Properties: 13 • Language (en) • Date (22-12-2010) • Revision (3)
- 14. DEMO EXAMPLE 14
- 15. EMERGED QUESTIONS Why open source repositories are not structured properly? How to assign unique marker (version) for our changes? What repository model should we use? How do we need to manage experimental changes or development? What is initial codebase? 15
- 17. WHY DO WE NEED TO MANAGE BUILDS? Builds should be CRISPy! CRISP means: Complete Repeatable Informative Schedulable 17 Portable
- 18. HOW TO MANAGE BUILD PROCESS? Define what build tasks you need Define what build types you need Choose build phases you will need Choose build tool Structure your build file 18
- 19. HOW TO MANAGE BUILD PROCESS? Define what build tasks you need Define what build types you need Choose build phases you will need Choose build tool Structure your build file 19
- 20. BUILD TASKS BY PRIORITY Compilation Deployment DB integration Unit testing Code coverage Static analysis Source code metrics Dynamic analysis Documentation generation … 20
- 21. LET’S TALK ABOUT DIFFERENCES BETWEEN BUILDS AND DEPLOYMENTS 21
- 22. BUILD VS DEPLOYMENT Deployment is the process of software installation (software system available to use) While build is the process of installation artifacts creation Is this always true? Not really For interpreted languages (no compilation phase) there might be no significant difference between build and deployment But it is more convenient to think of all related activities (unit-testing, code coverage, inspections, etc) as a build, not deployment. 22
- 23. RESULTS OF THE BUILD Executables .exe, .bin, .msi, .cab, … Libraries .dll, .lib, .so, … Archives .zip, .tar.gz, .bz2, .7z, .ear, .jar, .war, .sar, … Packages .rpm, .deb, … 23
- 24. RESULTS OF DEPLOYMENT Installed application Updated database Added incremental changes Unpackaged library Loading and installing dependencies … 24
- 25. BUILD VS DEPLOYMENT Build Deployment Runs on single machine Might be executed on several machines Does not require remote Requires connection to connection the remote server Creates application Installs application installation Makes sense always Makes sense only in case of distributed or remote application 25 Primary Secondary
- 26. HOW TO MANAGE BUILD PROCESS? Define what build tasks you need Define what build types you need Choose build phases you will need Choose build tool Structure your build file 28
- 27. BUILD TYPES CLASSIFICATION BY: Use •Private (developers build) •Integration (centralized build) •Release (delivery to the end-user) 29
- 28. BUILD TYPES CLASSIFICATION BY: Structure, stage, result • Pre-build (ensuring zero build errors: checking space, clean, …) • Post-build (build verification tests, security tests, …) • Clean (rebuilding whole project from scratch) • Incremental (only changed files and artifacts) • Broken 30
- 29. BUILD TYPES CLASSIFICATION BY: Staging • Lightweight (recompilation, unit- tests, …) • Heavyweight (deployment, system tests, inspections, …) 31
- 30. HOW TO MANAGE BUILD PROCESS? Define what build tasks you need Define what build types you need Choose build phases you will need Choose build tool Structure your build file 32
- 31. BUILD PHASES Pre-build Clean Init Check disk space Build Compilation Deployment Post-build Inspections Documentation generation Checking build errors Security tests Tagging 33
- 32. HOW TO MANAGE BUILD PROCESS? Define what build tasks you need Define what build types you need Choose build phases you will need Choose build tool Structure your build file 34
- 33. BUILD TOOLS GNU Autotools Autotools alternatives • make • pymake • automake • CMake • autoconf • Cons • autoheader • SCons • libtool • qmake • gettext • makepp • gcc • JAM • pkg-config • waf 35
- 34. BUILD TOOLS Java based • Ant • Maven • Gant .NET based • MSBuild • Nant • Byldan • NMaven Other • Phing (PHP) 36 • Rake (Ruby)
- 35. GNU AUTOTOOLS Did you build applications from sources on UNIX platform? What steps did you need to accomplish this? 1. ./configure 2. make 3. make install When does actual compilation happens? Right. During make command execution make is the father of all build tools Heart of the build with make is the makefile 38
- 36. GNU AUTOTOOLS MAKEFILE EXAMPLE target1 target2 target3 : prerequisite1 prerequisite2 command1 command2 command3 myprogram: main.o part1.o part2.o gcc -o myprogram main.o part1.o part2.o part1.o: part1.c part1.h header.h gcc -O -c part1.c part2.o: part2.c header.h gcc -O -c part2.c main.o: main.c header.h gcc -O -c main.c 39 clean: rm -f myprogram main.o part1.o part2.o
- 37. GNU AUTOTOOLS LIST autoconf creates a configuration script for a package from a template file From Makefile.in to Makefile automake Creates template file from another template file From Makefile.am to Makefile.in libtool helps manage the creation of static and dynamic libraries gettext internationalization and localization library gcc (GNU compiler collection) 40 default compiler used for building from sources
- 38. APACHE ANT Implemented in Java For the purpose of building java applications And for the purpose of make replacement Advantages over make: Cross-platform, does not require buildfile generation Resolves circular dependencies Works with file hierarchies Easily determines outdated files Conforms to “java way of thinking” 41
- 39. APACHE ANT BUILDFILE EXAMPLE 42
- 40. APACHE MAVEN Principles: Convention over configuration Declarative execution Reuse of build logic Coherent organization of dependencies Features: Project Object Model Extensive plugin architecture Predefined build lifecycle 43 Dependencies management
- 42. MAVEN VS ANT Maven Ant Description of project Development of a build (convention over configuration) script per project Invocation of defined goals Invocation of project (targets) specific targets Project knowledge "Just" the build process build lifecycle, standard too complex scripts project layout reusable scripts are not reusable plugins, repositories 45
- 43. HOW TO MANAGE BUILD PROCESS? Define what build tasks you need Define what build types you need Choose build phases you will need Choose build tool Structure your build file 46
- 44. BUILD FILE STRUCTURE Tasks Targets compile Compilation deploy Deployment deploy-db DB integration run-tests Unit testing check-coverage Code coverage run-inspections Static analysis gather-metrics Source code metrics performance-tests Dynamic analysis generatedoc 47 Documentation generation … …
- 45. BUILD FILE TEMPLATE 48
- 46. BUILD TYPES AND BUILDFILE STRUCTURE Use types: private, integration, release 49
- 47. LOCAL BUILD VS PRIVATE BUILD? Local build: What is the difference? Compilation Omitting unnecessary steps: Unit-testing Compilation Deployment Private build: Database integration Compilation Loading initial data Deployment Unit-testing Database integration Simple static analysis Loading initial data Unit-testing 50 Simple static analysis
- 48. LOCAL BUILD Local build is required after minor source code changes for the purpose of having executable artefact Compilation Deployment Database integration Loading initial data Unit-testing Simple static analysis 51 ant compile run-tests
- 49. PRIVATE BUILD Private build assumes that changes should be deployed to the application container or web-server Compilation Deployment Database integration Loading initial data Unit-testing Simple static analysis ant compile deploy deploy-db load- 52 data run-tests inspections
- 50. PRIVATE BUILD 53
- 51. INTEGRATION BUILD Integration build assumes presence of continuous integration practice as a key part of software lifecycle Compilation Deployment Database integration Loading initial data Unit-testing Static analysis Dynamic analysis 54 Documentation generation Gathering metrics
- 52. INTEGRATION BUILD 55
- 53. RELEASE BUILD Release build is the result of development will be visible to the end user Compilation Deployment Database integration Loading initial data Security tests integrity checks 56 performance tests
- 54. RELEASE BUILD 57
- 56. DEPLOYMENT TYPES Static when the container starts Hot Ability to deploy/undeploy deployables into a running container Incremental Deploying only changed files/artifacts Remote Requires connection via specified protocol Local 59 Does not require connection
- 57. DEPLOYMENT PROTOCOLS FTP/SFTP SSH/SCP No protocol Rsync (copy/move) Mail/e-mail (!) 60
- 58. DEPLOYMENT PHASES AND TASKS Deployment Deploy Restart application server Database integration Generate ORM models, mapping Dump data Instantiate DB (DDL usage) Data load into DB (DML usage, CRUD operations) Migrate data, structure or both 61
- 59. INCREMENTAL DEPLOYMENT Rsync synchronizes files and directories from one location to another SVN export only changed files svn diff –summarize /path1 /path2 Deploy only files modified/added since last revision svn status In most cases it’s just a… HEADACHE! 62
- 60. SIMPLE DEPLOYMENT WORKFLOW WC (working copy, trunk) local deployment instance db 63
- 61. LOCAL DEPLOYMENT WORKFLOW WC (working copy, trunk) local deployment dev instance test instance prod instance 64 dev db test db prod db
- 62. REMOTE DEPLOYMENT WORKFLOW + + + dev test prod local deployment WC (working copy, trunk) remote deployment + + + 65 dev test prod
- 63. MANAGE YOUR DEPLOYMENTS 1. Define what deployment type you will use (inherits from corresponding build type) 2. Define what deployment tasks you need 3. For each deployment type prioritize chosen tasks 4. Define destinations (production, test, development) 5. Map destinations and deployment types 6. Configure deployment destinations environment 66
- 64. DEVELOPMENT WORKFLOW EXAMPLE 67
- 65. DEVELOPMENT WORKFLOW EXAMPLE Deployed application WC (working copy) Repository svn up r1049 ant deploy-private r1050 r1051 r1052 development changeset (#146) Second deployed … manual merge application svn up r1126 ant deploy-private –Dinstance=2 resolve conflicts svn commit OK? yes 68 no make corrections manual merge svn commit
- 66. DEVELOPMENT WORKFLOW EXAMPLE NOTES Does this example remind you something? Yes! Distributed version control What is the point? Svn working copy is the repository While deployed application is a working copy Difference is that all operations between „repository‟ and „working copy‟ are manual And operations history is not tracked 69
- 67. DEVELOPMENT WORKFLOW EXAMPLE NOTES Why do we need two builds? To ensure there are no integration errors Because of “Don‟t break the build“ rule Advantages: You see your source code many times during merge You can find errors/inconsistencies easily Keeping unfinished development both safe and actual Flaws: A lot of time spent before commit Works only for web-projects written in interpreted languages 70
- 68. CONCLUSION 71
- 69. BEST PRACTICES Separate build from IDE Run faster tests first Introduce staging into build process Fail build fast Organize build targets by priority Build components separately Recursive build Centralization of project assets Put sources to sources (VCS) Put binaries to binaries (dedicated repositories) Consistent directory structure 72 Develop project structure conventions
- 70. BUILDS AND DEPLOYMENT MANAGEMENT 1. Define what build/deployment tasks you need 2. Define what build types you need 3. Choose build/deployment phases you will need 4. Prioritize chosen tasks with regard to corresponding build types 5. Choose build tool 6. Structure your build file 7. Define deployment destinations 8. Map build types and deployment destinations 9. Configure deployment destinations environment 73
- 71. RECOMMENDED READING 1. The Build Master: Microsoft's Software Configuration Management Best Practices By Vincent Maraia 74
- 72. RECOMMENDED READING 2. Ant: The Definitive Guide, 2nd Edition by Steve Holzne 75
- 73. RECOMMENDED READING 3. Maven: The Definitive Guide by Timothy M. O'Brien 76
- 74. USEFUL LINKS 1. http://freshmeat.net/articles/build-and-release-management - Build and release management 2. http://onjava.com/pub/a/onjava/2003/12/17/ant_bestpractices.ht ml - Ant best practices 3. http://www.sonatype.com/books/mvnref-book/reference/public- book.html - Maven complete reference 4. http://repo.exist.com/dist/maestro/1.7.0/BetterBuildsWithMaven. pdf - Great book about Maven 5. http://kent.spillner.org/blog/work/2009/11/14/java-build- tools.html - Ant vs Maven 6. http://martinfowler.com/articles/rake.html - article about rake by Martin Fowler 7. http://www.finalbuilder.com/finalbuilder.aspx - FinalBuilder home page 77