Sailing through devlopment with legacy code
- 1. Legacy code yet Agile way… A journey we started!!! Speakers:- Prasad Kunte @prasad_kunte Naresh Jain @nashjain
- 2. Hello
- 3. About IDeaS 25+ YEARS 7000+ CLIENTS 1M+ ROOMS 94 COUNTRIES 95% + ago we started helping clients manage their revenue later we’re still finding the revenue opportunity in every situation around the world are priced every single day using IDeaS on 6 continents have hotels that now run on IDeaS Revenue Management Systems More than 95 percent of all our clients are still working with us today.
- 4. • Developed & rolled out in 2007 • Mainly into maintenance untill last year • Conducive market to pick this business since last year CPRMS Product
- 5. • New Specifications for Inbound Data – Transactional data instead of summry data • New Forecasting Methodology for better forecasting • Other New features Business Requirements
- 6. & came the first surprise… • Crashed CVS Repo Server • An opportunity to migrate to SVN…. – We chose SVN http://cvs2svn.tigris.org/
- 7. • 0 Running Test Cases • No CI/CD • Production Schema/Seed not base lined in code base • Old Stack – Junit 1.5 – Spring 2.0.5 – Ant The Legacy
- 8. Agile engineering practices Safety Netting to Legacy Code Dev. Setup CI/CD Test Pyramid Clean Code
- 9. Need to have all of it… Safety Netting to Legacy Code Dev. Setup CI/CD Env Test Pyramid Clean Code We need all of it Don’t Forget Mate, Business is waiting for new features!!!
- 10. Ooops…… Which one to take first Safety Netting to Legacy Code Dev. Setup CI/CD Env Test Pyramid Clean Code 1 2 3 4 5 Just like Product Backlog, we have Agility Backlog
- 11. Sprint 0 – Backlog Prioritization Safety Netting to Legacy Code Dev. Setup CI/CD Env Test Pyramid Clean Code 1 2 3 4 5 • Spiked out Workflow tests • Removed un-necessary out dated items from SVN – – Design Docs (HLD, LLD) – Prototype HTMLs – EAP project files – Multiple JBOSS file check-ins – MySQL cooked up data files Reduced the code base size (from 2.1 GB to 500 MB) • Spiked out Jenkins spin up and local build
- 12. The most important!!! • Should we copy pase code for new specifications? --- Don’t think so • Workflow Tests for Old Specs – Ensured no existing functionality breaks – Still not database agnostic • Tests to run locally on a populated database Ensure support to existing client
- 13. The 2nd most important… A feedback loop! • Jenkins I am here to help you guys!
- 14. Build Jobs for Local and Jenkins Local Build CI Build CT Deploy Workflow Tests
- 15. Jenkins is Ready to give quick feedback
- 16. Next one – Safety Netting • Unit Tests – Independent isolated tests – Tests individual class/function/method • Business Logic Tests – – Validates a core business rule. – Might span across multiple domain objects/functions • Workflow Tests – 1 for each Scenario in New Spec Safety Netting as we move along • Refactor the code as we modify it • Write unit & business logic tests if they don’t exist • Don’t add 1 more if else to 10 existing if else(s)
- 17. Next is Test Pyramid • Right Tests for getting right feedback • Avoiding test duplication • Avoid 100s of UI Tests – they are not needed as other layers can catch most of it
- 18. Wait a second! We’ve started building 3 layers already
- 19. 3 layers already there… • https://wiki.jenkins-ci.org/display/JENKINS/LabeledTestGroupsPublisher+Plugin • Test Groups: "unit", "smoke", "regression", "integration", "special", "misc“ Image source: http://www.ascendfinancialplanning.com/your-finances-are-like-the-game-of-football-do-you-have-a-game-plan Well!! Then, we need to monitor it! We do have a plugin • Unit, Business Logic, Integration, Workflow, End to End, UI • Code and hpi available @ https://github.com/prasadkunte/labeled-test-groups- publisher • Now, we can configure & see tests on Jenkins like this Lets change labels as per our pyramid
- 20. Labelled Test Group – in Action
- 21. You can keep business happy while we pay tech. debt. Safety Netting to Legacy Code Dev. Setup CI/CD Env Test Pyramid Clean Code & this is what we started already!!!!! Dev setup is the next You guys are meeting the business needs, that’s cool!!
- 22. Dev. Setup • Dev. Setup pushed on to Artifact Repo. – Java, Jboss, MySQL • Baseline database – Structure copied from Prod – Seed data identified – Ant Script using dbdeploy to deploy base lined database
- 23. Dev. Setup • Custom Tool to get trimmed DB – For Workflow Tests
- 24. Workflow test cases – Not database agnostic
- 25. Hackathon @ IDeaS • Restified the app exposing few apis • Workflow Test Cases Refactored – Database Agnostic Before After
- 26. Dev setup in good shape too Safety Netting to Legacy Code Dev. Setup CI/CD Env Test Pyramid Clean Code We are going to improve further!!!
- 27. JaCoCo wrapper for test results Clean Code - Sonar Integration
- 28. Write Selenium UI tests only for important screens – The 4th Layer… • Integration with Jenkins pending • Taking around 1 min to cover 8 screens Another Layer in Test Pyramid - Selenium UI Test
- 29. So far, what did we achieve? • From no safety net To some safety net for legacy code • From no CI/CD To Jenkins CI/CD • From no test pyramid To 4 layers of pyramid (3 integrated in CI) • From no standard dev set up To standard dev setup While committing to Business Requirements
- 30. http://www.strongautomotive.com/everyone-good-plan-get-punched-nose/ Road ahead… Keep Improving:- 1. Increase Test Coverage % 2. Reduce Violations 3. Reduce build timings 4. Fully Automate Production Deployment
- 31. Moving Forward | Test Coverage & Viloations – Sonar Violations - Spend 20 minutes a day, – 5 tests a week – Refactor as we move – 1 UI Test a week Page 31
- 32. Earlier Latest Moving Forward | Reduce Build Timings – Faster Feedback Loop
- 33. Moving Forward | Auto Deployments
- 34. & the journey continues….
- 35. © Copyright Integrated Decisions and Systems, Inc. (IDeaS – A SAS COMPANY) Visit IDeaS online at www.ideas.com Thank you so much! Speakers: Naresh Jain @nashjain Prasad Kunte @prasad_kunte
- 36. © Copyright Integrated Decisions and Systems, Inc. (IDeaS – A SAS COMPANY) Visit IDeaS online at www.ideas.com Few References • http://blogs.agilefaqs.com/2011/02/01/inverting-the-testing-pyramid/ • https://prasadkunte.wordpress.com/2014/12/14/restifying-a-legacy-application- an-experience/ • http://blogs.agilefaqs.com/2011/03/09/big-upfront-test-creation-in-legacy- code-is-a-bad-idea/ • https://www.atlassian.com/company/about/shipit • http://www.mkyong.com/webservices/jax-rs/jersey-spring-integration-example/ • http://examples.javacodegeeks.com/enterprise-java/rest/jersey/json-example- with-jersey-jackson/
- 37. © Copyright Integrated Decisions and Systems, Inc. (IDeaS – A SAS COMPANY) Visit IDeaS online at www.ideas.com
Editor's Notes
- #2: 3/22/15 <number>
- #4: 1. IDeaS background 3/22/15 <number>
- #5: 1. CPRMS Product Background 3/22/15 <number>
- #6: Recovered the code base from cvs server area Why SVN 0 learning curve if you are used to CVS Co located team Other code bases already into SVN so Maintenance and Management becomes smooth 3/22/15 <number>
- #7: Restifying with this Spring version didn’t work Can’t simple migrate from 1.5 to 4.X withough changes 3/22/15 <number>
- #8: Talk about change in specs Data points changed Would be getting Transaction data for future booking and cars that have checked-in Few enhancements around that Talk about new features coming up Slated this year are changes for Continuous Pricing Notification Module – Managed by Exception Etc. 3/22/15 <number>
- #9: Let’s copy Production database (200+ GB) on each machine, as it is baseline scripts are not there Old Specs. - Copy, Paste code and move forward Pretty simple, let’s “try” and write test cases ONLY for newly written code & create Tech Debt stories for future iterations (2016 may be…) Nope let’s start with running local test cases first That’s crazy, it would take us 4/6 months just to build some pyramid. Anyways we are copy pasting for new spec, so let’s write some test cases only for new code 3/22/15 <number>
- #10: Dev. Tools uploaded in repo so that everyone gets same copy of Eclipse, Java, Jboss, MySQL etc. From the production copy of db, we created empty schema and identified the seed scripts Ant script written for deploying the database with seed scripts Few data points identified for unit & business logic tests cases which would be required frequently A custom tool written that would pull the data of only 1 carpark into flat files. Another command would push the data in given database 3/22/15 <number>
- #11: Will talk about Workflow test cases. Should be database agnostic and all. We identified the most critical workflows which are absolutely necessary for our system to work and wrote the workflow test cases around that… Just like feature first, we went ahead with having wftc to make sure that while we touch the code the core workflows doesn’t break. 3/22/15 <number>
- #12: As a part of Ship-It day evaluated how to restify this legacy application – finally could achieve with Jersey implementation http://www.mkyong.com/webservices/jax-rs/jersey-spring-integration-example/ http://examples.javacodegeeks.com/enterprise-java/rest/jersey/json-example-with-jersey-jackson/ http://mvnrepository.com/artifact/com.sun.jersey.contribs/jersey-spring https://prasadkunte.wordpress.com/2014/12/14/restifying-a-legacy-application-an-experience/ Workflow tests refactored to consume rest there by making it db agnostic 3/22/15 <number>
- #13: For code getting modified, unit test cases is a MUST UNIT TESTS to be written against empty seeded db Business Logic Tests to be written 3/22/15 <number>
- #14: We chose Jenkins. Started with simple project which would take the build & run the Unit and BL tests Deploy in an environment Run the Spec1 workflow tests 3/22/15 <number>
- #15: If code changes break Spec1 CI would alarm the same 3/22/15 <number>
- #16: Will talk about different sections of test pyramid Will talk about fact that it would have been difficult to build the pyramid for existing code and business wouldn’t have accepted So we chose starting with 3 layers, Workflow Test, Unit, business Logic 3/22/15 <number>
- #17: Will talk about different sections of test pyramid Will talk about fact that it would have been difficult to build the pyramid for existing code and business wouldn’t have accepted So we chose starting with 3 layers, Workflow Test, Unit, business Logic 3/22/15 <number>
- #18: In the zest to see that our test cases/ safety net is getting built as a pyramid and not as a diamond, ice-cream cone, pentagon etc., we started looking out for plugin that would give us some view about how our tests are shaping Found one.. 3/22/15 <number>
- #19: As we integrated, started looking @ a plugin that would segregate the test cases as per the pyramid. Could find one plugin Downloaded the plugin and tweaked it to have headings like "unit", "businesslogic", "integration", "workflow", "endtoend", "ui“ .hpi already uploaded @ XXX 3/22/15 <number>
- #20: Current Jenkins state, showing the Tests by Groups 3/22/15 <number>