Download as PDF, PPTX
![require 'rester'
require 'right_api_15’
describe "Verifies CRUD operations on the new ELB browser:", :crud => true do
before(:all) do
Session.login(:account_name => :test_account)
url = config(:LOCATION)
email = config(:EMAIL)
password = config(:PASSWORD)
RightApi15.default_login(login_info)
end
context "Setup" do
it "creates Network required for this test" do
@clouds.each do |cloud|
@api_created['network'] = NetworkManager::Network.create(:cloud => cloud, :cidr_block => '13.0.0.0/16’)
end
…
it "creates a load balancer using network/subnet with no name" do
@clouds.each do |cloud|
cloud_href = RightApi15::MetaData.get_cloud_href(:cloud_name => cloud)
network = @api_created['network'].select {|n| n.cloud_href == cloud_href}.first
subnet = @api_created['subnet'].select {|sn| sn.href.include?(cloud_href)}.first
security_group = @api_created['security_group'].select {|sg| sg.cloud_href == cloud_href}.first
LoadBalancer.create(:cloud => cloud,
:load_balancer_name => "#{@name}-api",
:VPC => { :vpc_option => network.resource_uid,
:subnets => subnet.resource_uid,](https://image.slidesharecdn.com/continuousdelivery-howrightscalereleasesweekly-151118220047-lva1-app6892/75/Continuous-Delivery-How-RightScale-Releases-Weekly-25-2048.jpg)
Uploaded byRightScale
Continuous Delivery: How RightScale Releases Weekly
The document discusses RightScale's continuous delivery process, outlining their motivations, challenges, strategies, and keys to success for weekly feature releases. It emphasizes the importance of automation, adherence to defined contracts for 'done' and 'ready to ship,' and the benefits of shorter release cycles, including increased stability and reduced bugs. The document also highlights the significance of proper testing and documentation in maintaining a seamless deployment process.
More Related Content
Similar to Continuous Delivery: How RightScale Releases Weekly
Continuous Delivery: How RightScale Releases Weekly
- 1. CONTINUOUS DELIVERY: HOW RIGHTSCALERELEASES WEEKLY
- 2. Speakers • Tim Miller •VP of Engineering • Daniel Onorato • Director of QA • Robert Sulway • Director of Development
- 3. Agenda • Defining ContinuousDelivery • Motivations and Strategies • Defining the Weekly Release • Keys to Success • Automating Is a “Must” • The Dashboard • (Surprising?) Benefits of Shorter Release Cycles
- 4. Your Roadmap toApplication Agility Code Build Integrate Test Release Deploy Operate Agile Development Continuous Integration Continuous Delivery Continuous Deployment DevOps
- 5.
- 6. Broker Cloud Serviceswith RightScale Self-Service Cloud Analytics RightScale Cloud Management Design Virtualized Environments Public Clouds IaaS+/PaaS Services Private Clouds Bare Metal Automate Multi-Cloud Orchestration & Governance OperateDeploy Report Optimize 5
- 7. • Complex codebase, lives in over 200 repositories • Some code going back 7 years • Running on ~700 servers in Production • 7-8 week release cycles • Developers, QA & Ops all set in their ways, process gets layered on top of process Landscape 6
- 8. • Get plannedfeatures to market more quickly • Respond to urgent requests • Reduce WIP • Master Branch Instability – not getting tested during regression • Increase quality, decrease escalations • Details of features “fresh in minds” on release • Mindset: My feature is going into production soon • Think about deployment! Motivations
- 9. • Spoiler Alert:These became benefits • The biggest challenge was getting past “We can’t..” • …because it would lower quality bar, QA always chasing the next release • ...roll out deployments in parallel, need to roll out serially • …because it will take too much time/people for release process The Challenges
- 10. • Go Big •Teams would need to innovate • Required to shed the old and do things differently • Add in some constraints to make things more interesting • Can’t lower the quality bar in any way • Can’t add human resources to achieve the goal • Can’t reduce velocity • If you are going to do it weekly, can’t have downtime during releases Strategies 9
- 11. • Process • Relentlessadherence to our “contracts” • Contract for the definition of “done” for features • Contract for the “ready to ship” weekly features collection • Dashboards (Jira/Travis) to see daily status of “ready to ship” • Environments • Branching strategy • Automation to stand up functional test environments for given features • Staging environment to test the weekly features collection as a whole Keys to Success
- 12. Keys to Success...continued 1111 •Automation • Write once, run anywhere system level test automation (feature test, stage integration, production) • Matrix driven automation test execution for release deployment validation in production with dashboard • Automation to facilitate rolling production deployment • Organization • Embedded QA Engineering on scrum teams • Tightly integrated Dev/Ops team
- 13.
- 14. What it is: •A vehicle to release a collection of features on a weekly basis when they are “done” What it is not: • A mandate to plan, design, implement, test and deploy a given feature in a week’s time Defining the Weekly Release
- 15. • For Libraries: •Unit test coverage in (ci) for new feature, either measured or manually verified to exist by QA • For Apps: • Unit test coverage running in (ci) for new feature(either measured or manually verified to exist by QA) • Functional test coverage in (ci) for new feature (either measured or manually verified to exist by QA) • Automated system test coverage run on a mini-rs and ready for merge into master test collateral repo • Note: we will require either the unit, functional or system test. The QA Eng/Devs will determine required coverage at the time a feature is worked. • Adequate end user documentation has been created. • Product owner sign-off for feature Definition of “done”
- 16. • For bothLibraries and Apps: • Downtime window is not required • Compatible with a rolling system restart • Documented migration instructions exist (if necessary) • Documented deployment instructions exist (if necessary) • Any required ServerTemplate updates must be coded and integrated on a mini-rs Definition of “done”
- 17. A weekly featurescollection is deemed “ready to ship” when: • Verification the new feature(s) unit test(ci), functional test(ci) and/or system level tests (staging) are passing on staging branch • Verification of a full nightly regression run on staging on staging branch with no impact on existing features and pass/fail trends • All regression issues are resolved Definition of “ready to ship”
- 18.
- 19.
- 20. Automation is aMust
- 21.
- 22. The RightScale DevOpsProcess Mini-RightScale Dev Environment Staging Environment Production Environment Deployment Automation Deployment Automation Pull code & test Deployment Automation Operations Automation Pull code & test Pull code & release Monitor alarm react Develop Templates
- 23. • Write once,run anywhere test collateral • Automation frameworks and collateral support running a given test in each of our possible test targets without updating test collateral once written. (mini-rs, staging, production). • $ bundle exec rake SPEC=ui/functional/server_array_functional.rb SETTINGS=mini-rs_93,nightly_settings SHARD=93 • $ bundle exec rake SPEC=ui/functional/server_array_functional.rb SETTINGS=us_3,nightly_settings SHARD=3 • Automated feature tests add to collection of regression tests once authored • Tying test frameworks into matrix based invocation system. Streamlining QA Automation
- 24. • Gemify (Ruby)test collateral repos to allow sharing among various automation frameworks. • Can speed up test execution in many cases • Speeds up test suite implementation • Example UI test using API test resources for setup Sharing Test Collateral Repos
- 25. require 'rester' require 'right_api_15’ describe"Verifies CRUD operations on the new ELB browser:", :crud => true do before(:all) do Session.login(:account_name => :test_account) url = config(:LOCATION) email = config(:EMAIL) password = config(:PASSWORD) RightApi15.default_login(login_info) end context "Setup" do it "creates Network required for this test" do @clouds.each do |cloud| @api_created['network'] = NetworkManager::Network.create(:cloud => cloud, :cidr_block => '13.0.0.0/16’) end … it "creates a load balancer using network/subnet with no name" do @clouds.each do |cloud| cloud_href = RightApi15::MetaData.get_cloud_href(:cloud_name => cloud) network = @api_created['network'].select {|n| n.cloud_href == cloud_href}.first subnet = @api_created['subnet'].select {|sn| sn.href.include?(cloud_href)}.first security_group = @api_created['security_group'].select {|sg| sg.cloud_href == cloud_href}.first LoadBalancer.create(:cloud => cloud, :load_balancer_name => "#{@name}-api", :VPC => { :vpc_option => network.resource_uid, :subnets => subnet.resource_uid,
- 26. Matrix Based AutomationTest Invocation
- 27.
- 28. Our “Ready toShip” Dashboard
- 29. Regression and FeatureBug Status
- 30.
- 31.
- 32.
- 33.
- 34. Benefits of shorterrelease cycles 3333 • Master code branch very stable at all times • Fewer regression bugs found in integration cycle • Automation test coverage guaranteed to grow and exist (definition of done) • Forced us to keep pushing on automation and tying systems together to lessen the human resource tax associated with each release (work in progress) • Forced us to engineer feature updates/migrations such that they can be performed in a rolling fashion with no system maintain window requiring down time. • Releases happen in the middle of the our normal work day as opposed to Friday nights / Weekends. Everyone much happier with this. • Features are deployed very close in time to there development. Details still fresh in our minds
- 35. 34 Next Steps andQ&A E-book: Continuous Integration and Delivery in the Cloud: How We Do It at RightScale: How RightScale Does It www.rightscale.com/ci-cd-ebook