Upping the game
Improving your software development process


  John Ferguson Smart
  Principle Consultant
  Wakaleo Consulting
  Email: john.smart@wakaleo.com
  Web: http://www.wakaleo.com
  Twitter: wakaleo
Presentation Goals
Learn how to improve, harmonize and automate your
development process using tools like Maven, Hudson, and
many others.
Speaker’s qualifications
John Ferguson Smart
Consultant, Trainer, Mentor, Author,...
Works with Enterprise Java, Web Development, and
Open Source technologies
Author of ‘Java Power Tools’ (O’Reilly)
Writes articles for sites like JavaWorld, DevX and
Java.net, and blogs on Java.net
Speaks at conferences, Java User Groups etc.
Likes to write about himself in the third person
Agenda
What we will cover today:
Industrializing your build process
Automate your builds
Better testing practices
Reducing technical debt
Why improve?
       HERE is Edward Bear, coming downstairs now, bump,
       bump, bump, on the back of his head, behind
       Christopher Robin. It is, as far as he knows, the only
       way of coming downstairs, but sometimes he feels
       that there really is another way, if only he could stop
       bumping for a moment and think of it.
                                                  -- A. A. Milne
Why should we improve?
Why should we improve our development process?
 Lower development costs
 Lower maintenance costs
 Less bugs
 Higher code quality
 Be flexible - adapt to change more easily
 Happier more productive users
How can we improve?
How can we improve our development process?
 Standardized build practices
 Better testing practices
 Better visibility
 Faster feedback
 Quality metrics
 Automate!
Tools for the job
    There are plenty of tools available - and most are free!

                         maven
                                         Build scripting



Automated code quality

                                 JUnit
                                                           Automated testing
Towards a better build process
Standardize your build process with Maven
What is Maven, anyway?
 A high-level open source build scripting framework
 Extensively used in the Java world
Towards a better build process
How does Maven help?
 Standards
 Conventions
 Lower Maintenance Costs
 Knowledge sharing
 Dependency Management
 Promoting good architecture
So how can Maven help me?
Standards and Conventions
A standard directory structure
A standard, but extensible build lifecycle

                               So new developers understand the
                               project structure and build process
                                          straight away!


      ...and the build scripts are much
     easier to understand and maintain
So how can Maven help me?
Technical documentation
Generate technical project documentation
Easy to integrate code quality metrics




       I can even generate UML
       diagrams in my Javadocs


                      And setting up code quality
                         metrics is a breeze!
So how can Maven help me?
Project architecture
Encourages developers to use modular design
More flexible architecture
Reduced complexity and maintenance costs
                                   Breaking our application down
                                    into clean modules is much
                                               easier

      ...and the smaller modules are
         easier to test and maintain
So how can Maven help me?
Dependency Management
Understand precisely what libraries your application needs
Safer and more reproducible builds
A standard way to share internal libraries
                                   All our libraries are shared and
                                  safely stored on a central server


    ...we just have to name the ones
        we need in our build script
So how can Maven help me?
Dependency Management before Maven
 Each project has its own set of JAR files
 Unnecessary duplication
 Hard to keep track of versions             lib
 Errors due to incompatible JAR files
 Overloads the source code repository




           ...and you never know what
              versions you are using
So how can Maven help me?
Dependency Management using Maven
 Library versions are stored on a central server
 Each project “declares” what libraries and versions it needs
 All the required dependencies are automatically downloaded
 The server is called a ‘Maven Enterprise Repository Manager’

                Projects declare the versions   Libraries are stored
                   of libraries they need       on a central server




                                                                  Libraries are
                                                                downloaded as
                                                                    required
               This way I know exactly what
               libraries my application uses
So how can Maven help me?
Release Management
A standard way to track and release versions
Official versions stored on a central server
Can be used to automate the deployment process




              The official versions are on that server
Towards better testing practices
Why is good testing so important?
Development costs
Maintenance costs
Visibility
Flexibility
Documentation
Towards better testing practices
Coding the traditional way

                   Requirements


                                  Design


                                       Implementation
   You write your code                                     Maybe do some unit testing


                                                    Test
          A tester tests the finished application
Towards better testing practices
Coding the traditional way

                                                       Lots of bugs
                          Fragile untested code
                                                  Changes introduce new
  Coding                  Low code coverage               bugs

Some unit testing?         No regression tests     Changes are costly

                                                     Bugs hard to fix
  Manual debugging?        Bug detected late
                                                   Bugs expensive to fix
      Manual functional    Hard to trace code
         testing?         back to requirements      Code fails to meet
                                                      requirements
Towards better testing practices
So what’s wrong with the old way?
Lots of defects. Really, lots.
High maintenance costs
Hard to introduce new features
Doesn’t meet the actual requirements
Delayed deliveries
Unhappy end-users
Towards better testing practices
                    How much does a bug cost to fix, anyway?
00
                                                                                  $$$
75



50                                                                                  Changes harder and more costly
                                 Changes easier and
                                  cheaper to make
25


                                                                                                     When was the
0                                                                                                    change requested?
     Requirements




                        Design




                                            Unit Testing




                                                           System Testing




                                                                            UAT




                                                                                        Production
Towards better testing practices
How can good testing practices help?
 Reduce bugs
 Write better-designed code
 Have more confidence in our code
 Make changes more easily
 Meet user requirements more accurately
 Lower maintenance costs
Towards better testing practices
How can good testing practices help?
                              Easier to make changes
      Well-designed code
                              Better response to user feedback
        Find bugs faster
                                Cheaper bug fixes
         Full automated
        regression tests       Lower maintenance costs

                               Make changes with confidence
          Code to the
         requirements
                                Don’t code unnecessary features

     Focus and productivity        Develop more productively

          Technical               “Executable requirements”
        documentation
                                  Lower maintenance costs
Towards better testing practices
More flexibility
 Testable code is easier to change
 Full regression tests avoid introducing errors




     I’m not afraid to change the code
        - the tests are my safety net
Towards better testing practices
Better visibility
 Tests are “executable requirements”
 Automated acceptance tests measure progress




      A feature can’t be “90% finished”
         - it either works or it doesn’t
Towards better testing practices
Documentation
 Tests are “living documentation” of your code
 Always accurate and up-to-date




     I can understand how the code
        works by reading the tests
Towards better testing practices
Lower maintenance costs
 Less bugs, found faster
 Changes are easier to make




         Maintaining this sort of
      application is a real pleasure!
Automating the build process
Continuous Integration - what’s the issue?
Traditional development cycles are bad for your health:
 Integration is long and difficult
 Poor visibility on development progress
 Functional tests are done too late
 Raised issues are harder to fix
 The client gets a sub-optimal product
Automating the build process
Continuous Integration - what’s involved?
Automating the build process
Continuous Integration - why bother?
Smoother integration process
Automatic regression testing
Regular working releases
Earlier functional testing
Faster and easier bug fixes
Better visibility

                     No more “it works on my machine”
Automating the build process
Continuous Integration - what you need
            Automated build process (e.g. Maven)




                Automated tests (JUnit, Selenium, easyb...)




                          Source code repository



                                     Continuous Build Server
Automating the build process
Continuous Integration - what can it do?
Raise integration issues - fast!
Monitor your build process
Monitor and report on code quality and code coverage
Build promotion and release management
Automated deployments
Automating the build process
Looking for a good O/S Continuous Integration tool?
Try Hudson!
 Easy to set up and configure
 Good build and code quality metrics
 Lots of plugins
Automated Code Quality
Why use code quality metrics
Better quality code
Enforce corporate coding standards
Detect potential bugs
Code is easier to maintain
Train new staff
Keep technical debt down
Automated Code Quality
What is technical debt?
 The cost of poor quality code:
   Harder to make changes
   Too much time spent fixing bugs
   Takes too long to add competitive new features


                            This legacy code takes way
                                 to long to change




               We spend all our time fixing bugs, not
                      adding new features
Automated Code Quality
How do we pay off technical debt?
 Enforce coding standards
 Teach developers good coding practices
 Spend time keeping the code clean (refactoring)




                              If I spend a little time tidying up
                                      my code today...




                 ...it will be quicker and easier for
                everyone to make changes later on
Automated Code Quality
Team code reviews
 Review code as a group
 Long and slow if done manually
 Benefits greatly from the use of tools
Automated Code Quality
Enforcing coding standards with Hudson


                             Number of violations over time
Automated Code Quality
Enforcing coding standards with Hudson




                      Drilling down
Automated Code Quality
Enforcing coding standards with Hudson




                      Details for a particular issue
Automated Code Quality
Code Coverage
See what code is being executed by your unit tests.
Isolate untested code
Can help to estimate if testing practices are being applied
Automated Code Quality
Monitoring Code Coverage with Hudson

                                Executed code




                       Unexecuted code
Automated Code Quality
Code Quality Governance with Sonar
Centralized code quality management
Works on any Maven project
Store code quality metrics in a database
Code quality metrics can be consulted on a web site
Automated Code Quality
     Code Quality Governance with Sonar
       Sonar centralizes many code quality metrics
Source code metrics                                  Code complexity metrics




                                                   Test results and code coverage


Code quality metrics


                                                            Build history


          Modules


                                             Click anywhere to drill down
Automated Code Quality
    Code Quality Governance with Sonar
      You can drill down to view the details for each type of issue
                                           Overview

                                                      Different types of violations




Violations in this class




                       Violation details
Summary
How can you improve the development process?
Standardize your build process
Improve testing practices
Use Continuous Integration
Reduce your technical debt
Automate, automate, automate!
John Ferguson Smart
                               Email: john.smart@wakaleo.com
                                Web: http://www.wakaleo.com
                                               Twitter: wakaleo




Thanks for your attention!


                                     http://weblogs.java.net/blog/
      http://www.wakaleo.com

Improving your SDLC

  • 1.
    Upping the game Improvingyour software development process John Ferguson Smart Principle Consultant Wakaleo Consulting Email: [email protected] Web: http://www.wakaleo.com Twitter: wakaleo
  • 2.
    Presentation Goals Learn howto improve, harmonize and automate your development process using tools like Maven, Hudson, and many others.
  • 3.
    Speaker’s qualifications John FergusonSmart Consultant, Trainer, Mentor, Author,... Works with Enterprise Java, Web Development, and Open Source technologies Author of ‘Java Power Tools’ (O’Reilly) Writes articles for sites like JavaWorld, DevX and Java.net, and blogs on Java.net Speaks at conferences, Java User Groups etc. Likes to write about himself in the third person
  • 4.
    Agenda What we willcover today: Industrializing your build process Automate your builds Better testing practices Reducing technical debt
  • 5.
    Why improve? HERE is Edward Bear, coming downstairs now, bump, bump, bump, on the back of his head, behind Christopher Robin. It is, as far as he knows, the only way of coming downstairs, but sometimes he feels that there really is another way, if only he could stop bumping for a moment and think of it. -- A. A. Milne
  • 6.
    Why should weimprove? Why should we improve our development process? Lower development costs Lower maintenance costs Less bugs Higher code quality Be flexible - adapt to change more easily Happier more productive users
  • 7.
    How can weimprove? How can we improve our development process? Standardized build practices Better testing practices Better visibility Faster feedback Quality metrics Automate!
  • 8.
    Tools for thejob There are plenty of tools available - and most are free! maven Build scripting Automated code quality JUnit Automated testing
  • 9.
    Towards a betterbuild process Standardize your build process with Maven What is Maven, anyway? A high-level open source build scripting framework Extensively used in the Java world
  • 10.
    Towards a betterbuild process How does Maven help? Standards Conventions Lower Maintenance Costs Knowledge sharing Dependency Management Promoting good architecture
  • 11.
    So how canMaven help me? Standards and Conventions A standard directory structure A standard, but extensible build lifecycle So new developers understand the project structure and build process straight away! ...and the build scripts are much easier to understand and maintain
  • 12.
    So how canMaven help me? Technical documentation Generate technical project documentation Easy to integrate code quality metrics I can even generate UML diagrams in my Javadocs And setting up code quality metrics is a breeze!
  • 13.
    So how canMaven help me? Project architecture Encourages developers to use modular design More flexible architecture Reduced complexity and maintenance costs Breaking our application down into clean modules is much easier ...and the smaller modules are easier to test and maintain
  • 14.
    So how canMaven help me? Dependency Management Understand precisely what libraries your application needs Safer and more reproducible builds A standard way to share internal libraries All our libraries are shared and safely stored on a central server ...we just have to name the ones we need in our build script
  • 15.
    So how canMaven help me? Dependency Management before Maven Each project has its own set of JAR files Unnecessary duplication Hard to keep track of versions lib Errors due to incompatible JAR files Overloads the source code repository ...and you never know what versions you are using
  • 16.
    So how canMaven help me? Dependency Management using Maven Library versions are stored on a central server Each project “declares” what libraries and versions it needs All the required dependencies are automatically downloaded The server is called a ‘Maven Enterprise Repository Manager’ Projects declare the versions Libraries are stored of libraries they need on a central server Libraries are downloaded as required This way I know exactly what libraries my application uses
  • 17.
    So how canMaven help me? Release Management A standard way to track and release versions Official versions stored on a central server Can be used to automate the deployment process The official versions are on that server
  • 18.
    Towards better testingpractices Why is good testing so important? Development costs Maintenance costs Visibility Flexibility Documentation
  • 19.
    Towards better testingpractices Coding the traditional way Requirements Design Implementation You write your code Maybe do some unit testing Test A tester tests the finished application
  • 20.
    Towards better testingpractices Coding the traditional way Lots of bugs Fragile untested code Changes introduce new Coding Low code coverage bugs Some unit testing? No regression tests Changes are costly Bugs hard to fix Manual debugging? Bug detected late Bugs expensive to fix Manual functional Hard to trace code testing? back to requirements Code fails to meet requirements
  • 21.
    Towards better testingpractices So what’s wrong with the old way? Lots of defects. Really, lots. High maintenance costs Hard to introduce new features Doesn’t meet the actual requirements Delayed deliveries Unhappy end-users
  • 22.
    Towards better testingpractices How much does a bug cost to fix, anyway? 00 $$$ 75 50 Changes harder and more costly Changes easier and cheaper to make 25 When was the 0 change requested? Requirements Design Unit Testing System Testing UAT Production
  • 23.
    Towards better testingpractices How can good testing practices help? Reduce bugs Write better-designed code Have more confidence in our code Make changes more easily Meet user requirements more accurately Lower maintenance costs
  • 24.
    Towards better testingpractices How can good testing practices help? Easier to make changes Well-designed code Better response to user feedback Find bugs faster Cheaper bug fixes Full automated regression tests Lower maintenance costs Make changes with confidence Code to the requirements Don’t code unnecessary features Focus and productivity Develop more productively Technical “Executable requirements” documentation Lower maintenance costs
  • 25.
    Towards better testingpractices More flexibility Testable code is easier to change Full regression tests avoid introducing errors I’m not afraid to change the code - the tests are my safety net
  • 26.
    Towards better testingpractices Better visibility Tests are “executable requirements” Automated acceptance tests measure progress A feature can’t be “90% finished” - it either works or it doesn’t
  • 27.
    Towards better testingpractices Documentation Tests are “living documentation” of your code Always accurate and up-to-date I can understand how the code works by reading the tests
  • 28.
    Towards better testingpractices Lower maintenance costs Less bugs, found faster Changes are easier to make Maintaining this sort of application is a real pleasure!
  • 29.
    Automating the buildprocess Continuous Integration - what’s the issue? Traditional development cycles are bad for your health: Integration is long and difficult Poor visibility on development progress Functional tests are done too late Raised issues are harder to fix The client gets a sub-optimal product
  • 30.
    Automating the buildprocess Continuous Integration - what’s involved?
  • 31.
    Automating the buildprocess Continuous Integration - why bother? Smoother integration process Automatic regression testing Regular working releases Earlier functional testing Faster and easier bug fixes Better visibility No more “it works on my machine”
  • 32.
    Automating the buildprocess Continuous Integration - what you need Automated build process (e.g. Maven) Automated tests (JUnit, Selenium, easyb...) Source code repository Continuous Build Server
  • 33.
    Automating the buildprocess Continuous Integration - what can it do? Raise integration issues - fast! Monitor your build process Monitor and report on code quality and code coverage Build promotion and release management Automated deployments
  • 34.
    Automating the buildprocess Looking for a good O/S Continuous Integration tool? Try Hudson! Easy to set up and configure Good build and code quality metrics Lots of plugins
  • 35.
    Automated Code Quality Whyuse code quality metrics Better quality code Enforce corporate coding standards Detect potential bugs Code is easier to maintain Train new staff Keep technical debt down
  • 36.
    Automated Code Quality Whatis technical debt? The cost of poor quality code: Harder to make changes Too much time spent fixing bugs Takes too long to add competitive new features This legacy code takes way to long to change We spend all our time fixing bugs, not adding new features
  • 37.
    Automated Code Quality Howdo we pay off technical debt? Enforce coding standards Teach developers good coding practices Spend time keeping the code clean (refactoring) If I spend a little time tidying up my code today... ...it will be quicker and easier for everyone to make changes later on
  • 38.
    Automated Code Quality Teamcode reviews Review code as a group Long and slow if done manually Benefits greatly from the use of tools
  • 39.
    Automated Code Quality Enforcingcoding standards with Hudson Number of violations over time
  • 40.
    Automated Code Quality Enforcingcoding standards with Hudson Drilling down
  • 41.
    Automated Code Quality Enforcingcoding standards with Hudson Details for a particular issue
  • 42.
    Automated Code Quality CodeCoverage See what code is being executed by your unit tests. Isolate untested code Can help to estimate if testing practices are being applied
  • 43.
    Automated Code Quality MonitoringCode Coverage with Hudson Executed code Unexecuted code
  • 44.
    Automated Code Quality CodeQuality Governance with Sonar Centralized code quality management Works on any Maven project Store code quality metrics in a database Code quality metrics can be consulted on a web site
  • 45.
    Automated Code Quality Code Quality Governance with Sonar Sonar centralizes many code quality metrics Source code metrics Code complexity metrics Test results and code coverage Code quality metrics Build history Modules Click anywhere to drill down
  • 46.
    Automated Code Quality Code Quality Governance with Sonar You can drill down to view the details for each type of issue Overview Different types of violations Violations in this class Violation details
  • 47.
    Summary How can youimprove the development process? Standardize your build process Improve testing practices Use Continuous Integration Reduce your technical debt Automate, automate, automate!
  • 48.
    John Ferguson Smart Email: [email protected] Web: http://www.wakaleo.com Twitter: wakaleo Thanks for your attention! http://weblogs.java.net/blog/ http://www.wakaleo.com