Tech Talk on Cloud Computing
- 2. Reaching for the cloud 3 ways that the cloud will change the way you develop software Atlassian NirajBhawnani
- 3. What is the cloud?
- 4. What is the cloud? • Remote servers run applications and store your data • Accessible over the internet • Users need not care where the application is running or where their files are hosted • Examples of cloud companies:
- 5. Atlassian & the cloud
- 6. Who is Atlassian? • We build software tools for issue tracking, collaboration, continuous integration, and source control
- 7. Who is Atlassian? • Founded in 2002 by 2 Sydney-siders • ~600 employees in:
- 8. Who is Atlassian? Atlassian’s Vietnamese Team, November 2013
- 9. • Atlassian started selling software for customers to install on their own servers • Customers would keep asking if we had a hosted solution • In 2011, Atlassian OnDemand was born
- 10. • 41,150 VMs • 5,640 CPU cores • 58 TB RAM • 368 TB Disk
- 11. Why is everything moving to the cloud?
- 12. Why move to the cloud? • Great for business. One time payments become regular payments Confluence BTF vs. OnDemand 100 user license = $4000 100 user license = $300/month Assuming an upgrade every 2 years: BTF = $4000 OnDemand = $300 × 2 × 12 = $7200
- 13. Why move to the cloud? • Cheaper for customers – no staff or hardware required Confluence BTF vs. OnDemand 100 user license = $4000 Staff/hardware = $70000/yr 100 user license = $300/month No staff/hardware = $0/yr Cost every 2 years: BTF = $70000 × 2 + $4000 = $144000 OnDemand = $300 × 2 × 12 = $7200
- 14. Why move to the cloud? • Always accessible via the internet • Full control of the application server environment
- 15. Delivering software to the cloud
- 16. Traditional delivery pipeline Production Merge code Build Release Manual Task Automatic Task QA Verification Staging deployment & verification
- 17. Continuous Delivery Pipeline Production Merge code Manual Task Automatic Task Continuous Integration (Build & Test)
- 18. How will the cloud change the way you build software?
- 20. Incremental Delivery • Waterfall model does not work for cloud software • Deliver features in small increments instead of “big bang releases” • Good for customers: they get features faster • Good for you: less investment before validation
- 21. Incremental Delivery • Start with a Minimum Viable Product (MVP) • Simplest version of the feature that still adds value for users • Example: Facebook Photos • MVP allows users to just upload and view photos
- 22. Incremental Delivery • Then deliver incrementally in order of importance • V1/MVP: Upload and view standard definition photos • V2: Face tagging • V3: Video support • V4: High definition photos & full screen mode • V5: Automatic face detection
- 23. What to do when the MVP is too big?
- 24. Feature Switches • Atlassian calls them “dark features” • Wrap the feature in some condition and provide a way to switch the condition on and off • Also provide a way for tests to turn them on/off • Eliminates the need for long lived code branches
- 25. Feature Switches • Also allows us to deploy features to a subset of users • Can be used to test more risky features without upsetting too many users All users Experiment Group Control Group
- 26. 2. Automation
- 27. Automated Testing • Since there might not be a manual verification step, we must write a lot of automated tests to verify the system: • Unit tests • Integration tests • Smoke tests
- 28. Automated Testing • Automated tests need to run for every commit on every branch • Branches can be merged automatically or manually • Automatic deployment when the main branch passes • Tools: Atlassian Bamboo, Hudson, Cruise Control
- 29. Automated Testing • Check in code • Build runs: • Unit tests • Integration tests • Merge code
- 30. Unit Tests • Used to test individual units of source code in isolation • Can run very very fast • Might miss some bugs • E.g. JUnit, Mockito, QUnit, etc.
- 31. Integration Tests • Tests the whole product by functionality • Runs slowly • Selenium tests to drive the UI • Hamcrest/JUnit to do assertions
- 32. Smoke Tests • Similar to integration tests • Runs on one or more staging servers with real data that has been upgraded • Verifies that the data is intact and that basic functionality still works • Also uses Selenium
- 33. Selenium • Test code sends commands to Selenium Server • Selenium Server starts a browser • Selenium Server sends commands to the browser • Type in this form • Click the submit button • Is this element visible? Test code Selenium Server Web Browser
- 34. Selenium • Since you need to write a lot of tests, you need to reuse as much code as possible • Write “page objects” that represent each page and “page components” that represent shared components (e.g. header and navigation)
- 35. Selenium • Example code (Selenium with page objects): @Test public void testQuickSearchWithResults() { viewPage = product.login(User.TEST, ViewPage.class, testData.page); QuickSearch quickSearch = viewPage.getHeader().getQuickSearch(); SearchResultPage resultsPage = quickSearch.doSiteSearch(searchWithResults); Poller.waitUntilTrue(resultsPage.hasMatchingResults()); }
- 36. Automated Deployment • Deployments happen very frequently: • • • : 25/day : 1/day : 1/week • Updates need to be fast and automatic • Usually a collection of custom software and scripts
- 37. 3. Analytics
- 38. Analytics • Collect usage data from all instances and users in a centralised place • This can be a very large amount of data, you need to use tools that can handle it: • Graphite • Amazon Redshift
- 39. Analytics • Query the data to answer important questions: • What features are people using? • What features are people not using? • How are the features being used? • What kind of users are using what kind of features? • Use the data to make important business decisions
- 40. Analytics • You should come up with the list of questions before you ship the feature. Why?
- 41. Analytics • You should come up with the list of questions before you ship the feature. Why? • To answer some kinds of questions you may need to write additional code.
- 42. Atlassian Analytics • Leverages the existing events system • Support for client-side events • Reporting via Graphite or Apache Hive • Google Analytics • Greenzone
- 44. Atlassian Analytics Application Server HAMS Application code Sales information Java Event Web Browser Batched AJAX Analytics Plugin Database Server Greenzone Database UDP Analytics Server Google Analytics Daily SQL Queries Hive, etc.
- 45. Other Considerations • Hardware costs (particularly RAM) is the largest expense • RAM usage can be reduced by: • Multitenancy • Microservices architecture • Optimising code
- 46. Other Considerations • Deployment should not cause downtime • Upgrade a standby server • Switch proxy to new server • Store sessions in a central location e.g. database • Users experience no downtime
- 47. Other Considerations • Things don’t always go as planned • Always try to roll forward, never backward • Have a procedure to deploy hotfixes in case of emergency • Only revert to older versions as a last resort
- 48. What should I do next?
- 49. What should I do next? • Consider moving your application to the cloud • Deliver features incrementally • Write a lot of automated tests • Make deployment easy and automatic • Use collected data to make business decisions
- 50. • Learn more about Atlassian software • Hear talks like this one about industry best practices • Network with other professionals • Free beer! • https://aug.atlassian.com/
- 51. Thank You!
- 52. Questions?