SlideShare a Scribd company logo
Continuous testing webinar 041017 slideshare
2
Shashi Kiran
CMO
Quali
Marc Hornbeek
“DevOps-the-Gray”
Principal DevOps Consultant,
Trace3
Author, DevOps Test
Engineering Course
The DevOps Institute
HOST EXPERT PANEL
Pascal Joly
Director, Technology
Partnerships
Quali
Agenda
3
1 Introduction
2 Principles of Continuous Testing
3 Implementing CT
4 Demo: Continuous Testing
5 Q&A Continuous Testing
The “Missing Link”
01
02
03
Venture-funded start-up focused on Cloud-centric
DevOps and BizOps automation
Delivers “Sandboxes” - Replicas of production
environments
Strong Innovation DNA – Trusted by customers
worldwide
About Quali
4
300+ customers – Cloud providers, Telcos,
Enterprises – across FSI, Retail, Healthcare and
Government
~115 employees in USA, Israel and Europe
focused on serving customers worldwide300+
Continuous testing webinar 041017 slideshare
Infographic
Download here: http://bit.ly/2nY4FTB
The Rising Importance of (Continuous) Testing
SPEED QUALITY
TEST
Traditional Testing
Source: www.saucelabs.com
• Siloed development and testing
• Waterfall approach
• Long lead times between
identifying and solving bugs
• High risk when bugs are missed
• Rework and delays
What is Continuous Testing (CT)?
• Quality assessment strategy
• Incremental changes tested
“continuously”
• Automated tests (large %)
• DevOps Toolchain integration
• Production equivalent test
environment stages
• End-to-end testing across the
pipeline
CT – A sign of DevOps Maturity
1
2
3
4
5
Processes characterized for specific
projects. Organization is often reactive
Automated Test Execution
Projects tailor processes from
organizations development
methodology
Test Framework pipeline
Processes are measured and controlled
Test-driven Release Tools
Focus on process improvement
Dynamic Test and Environment
Blueprints
Process is unpredictable,
poorly controlled and
reactive
Manual Adhoc Tests
Hurdles to Implement Continuous Testing
Low % test automation
Budget
Missing/fragmented
toolchain
Production equivalent test
environment stages
Product
Leadership
Culture
Training and Expertise
Collaboration – Dev/Sec/Ops
Budget
Rewards and Incentives
People
Continuous Test planning
End-to-end Strategy
End-to-end testing across
pipeline
Process
CI
CT
CD
CollaborativeCultureisthedoortoDevOps,but
ContinuousTestingisthekeytoContinuous
Delivery
“
”
DevOps-the-Gray
CT provides Assessment of CD Stage Artifacts
Continuous Testing – Implementation Strategy
Adopt
5 test
Tenets
Org. and
Culture
Streamline
Process
and
Workflow
Integrating
CT Infra.
Environment
Orchestration
and Test
Automation
1 2 3 4 5
1. Five Tenets of Continuous Testing
2
Arrange tests so most
likely problems are
found in earlier stage of
pipeline.
Fail Early
1 Shift Left
Conduct each test as
early in the pipeline as
possible.
4
Focus on most
important tests and
results
Relevance
3 Test Fast
Arrange tests to run in
quick cycles
5 Fail Often
Run tests frequently
and with many different
conditions
2. Build Organizations and Culture
• DevOps does not mandate roles
• Teams decide role assignments
• Checklist help ensure tasks are
covered.
• Multiple checklists may be needed to
cover different stages of the pipeline
such as Pre-Flight, Integration,
Delivery, Deployment, etc.
3. Process and Workflow Practices
• Streamline Procurement, install and support:
o Infrastructure
o Tools
o Integration of test tools into tool frameworks
o Operating the DevOps pipeline
• Align Rewards and Incentives
• Train on Automation Skills and Tools
4. Integrated DevOps Test Infrastructure
PLAN
Collaboratively
plan integration
into each stage of
DevOps
Infrastructure
TEST
Automatically
trigger tests from
master DevOps
framework.
ASSESS
Run DevOps test
assessment for
each stage to
provide feedback
and support
workflow decisions.
Stage 2
4A. Testing Triggers – Infrastructure “Glue”
• Continuous Test framework is integrated with
the DevOps master framework.
• DevOps test automation can be performed by
the DevOps master framework directly or may
be implemented using special purpose DevOps
test framework tools.
• Example: DevOps testing for the build stage is
triggered when the DevOps master framework
triggers the DevOps test framework.
TEST
4B.Asssessment Actions Processing
• CT test executions produce verdicts.
• Verdicts provide decision support data
for analytics that determine if a change
is good enough to proceed to the next
stage in the DevOps pipeline or needs to
be remediated before proceeding.
• Analytics may be basic verdict reports or
more sophisticated verdict trend analysis.
Verdicts
Verdict Trends
Analytics
ASSESS
Automatically setup the test
environment and resources
(physical and virtual) to match
the requirements of a test.
Environment Orchestration
5. Test Environment Orchestration and
Automation
Execute test tasks without
manual work required.
Test Automation
5A. Environment Orchestration
• Test resources vary depending on
the complexity and risk of change
being tested.
• A flexible lab environment
employs orchestration tools to
elastically scale physical and
virtual resources for a specific
test.
5B. Test Automation
• Create new automated tests at the
same time changes are created.
• Run automated tests matched to
specific content being changed.
• Produce test results automatically
to support in-process DevOps test
assessments.
Continuous testing uses automation to execute tests quickly and consistently and
to keep up with the frequent changes that are processed through DevOps pipelines.
DevOps Orchestration Stack
Testing
Orchestration
simplifies
management of
test resource
topologies through
test environment
blueprints.
Test Orchestration
(Dev)
Build Automation Packaging Config Mgmt.
Dev & Integration DeliveryPackaging
Test Orchestration
(Staging and
Production)
Test Orchestration
(Acceptance)
Test Resources Cloud
CI/CD Pipeline: Application Release Automation & ALM Systems
CI/CD with CT – Leveraging Cloud Sandboxes
Pipeline Automic
Quali
Sandbox
Software
Load Security Integration
Staging/
Production
Dev
CI/CD Pipeline: Application Release Automation & ALM Systems
Quali CloudShell -Sandboxes
Blueprint Modeling Automation & Orchestration
Reservation & Scheduling Self-Service Catalog
Built-In Networking BI & Analytics
REST API
EMBEDDED GUI
DevOps Plug-Ins
BUILD ANY ENVIRONMENT. DEPLOY AS-A-SERVICE
Bare Metal
TO ANY CLOUD*
ENVIRONMENTFEATURESINTERACE
Applications Data Test Tools
Physical Infra. Virtual Infra. Services
Network Storage
Compute Security
CLOUD SANDBOX
Demo
Continuous Integration
and Testing
with Cloud Sandboxes
Test Strategies for Continuous Deployment
Reference: https://vwo.com/ab-testing/
A/B Testing Canary Testing Blue/Green Testing
Continuous Testing - Benefits
Reduced test
Time and test
wait time
Improved
Test Results
Analysis
Cost
Efficiency of
Shared
Resources
Improved
SLAs
CONTINUOUS
TESTING
Rapid
Test
Environment
Recovery
Summary
1
2
3
Don’t compromise between
speed and quality.
Support Continuous Delivery
Initiative with Continuous Testing
Cloud Sandboxes can be
orchestrate and automate
environments for continuous test
CONTINUOUS
TESTING
From “Missing Link” to “Glue”
2017 DevOps and Cloud Survey
http://info.quali.com/2017-devops-and-cloud-survey
For More Information:
www.quali.com www.trace3.com www.devopsinstitute.com
Quali Trace3 DevOps Institute
33
www.quali.com
THANK YOU

More Related Content

What's hot (20)

PDF
QA in DevOps: Transformation thru Automation via Jenkins
Tatyana Kravtsov
 
PPTX
Webinar: DevOps challenges facing QA
Qualitest
 
PDF
Continuous Testing: A Key to DevOps Success
TechWell
 
PPTX
Where Testers & QA Fit in the Story of DevOps
QASymphony
 
PPTX
Continuous Testing - The Final Frontier
Jaco Greyling
 
PDF
Continuous testing & devops with @petemar5hall
Peter Marshall
 
PPTX
#ATATalk - Episode 1 : Session on Selenium Exceptions by Pallavi Sharma
Agile Testing Alliance
 
PDF
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
PPTX
Continuous Testing and New Tools for Automation - Presentation from StarWest ...
Sauce Labs
 
PPTX
Introducing DevOps
Nishanth K Hydru
 
PPTX
Testing in a DevOps team
Laurent PY
 
PPT
Test Automation In The Hands of "The Business"
Greg Tutunjian
 
PDF
How to implement DevOps in your Organization
Dalibor Blazevic
 
PPTX
Automate Salesforce Releases with DevOps: Crawl, Walk, Run!
Susmitha Vakkalanka
 
PPTX
DevOps as-a-Service (DaaS) value
Marc Hornbeek
 
PPTX
Agile-plus-DevOps Testing for Packaged Applications
Worksoft
 
PDF
Continuous Delivery Distilled
Matt Callanan
 
PPTX
Relieveing the Testing Bottle Neck - Webinar
Cprime
 
PDF
DevOps Evolution - The Next Generation ?
Marc Hornbeek
 
PPT
Stc 2016 regional-round-ppt-automation testing with devops in agile methodolgy
Archana Krushnan
 
QA in DevOps: Transformation thru Automation via Jenkins
Tatyana Kravtsov
 
Webinar: DevOps challenges facing QA
Qualitest
 
Continuous Testing: A Key to DevOps Success
TechWell
 
Where Testers & QA Fit in the Story of DevOps
QASymphony
 
Continuous Testing - The Final Frontier
Jaco Greyling
 
Continuous testing & devops with @petemar5hall
Peter Marshall
 
#ATATalk - Episode 1 : Session on Selenium Exceptions by Pallavi Sharma
Agile Testing Alliance
 
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
Continuous Testing and New Tools for Automation - Presentation from StarWest ...
Sauce Labs
 
Introducing DevOps
Nishanth K Hydru
 
Testing in a DevOps team
Laurent PY
 
Test Automation In The Hands of "The Business"
Greg Tutunjian
 
How to implement DevOps in your Organization
Dalibor Blazevic
 
Automate Salesforce Releases with DevOps: Crawl, Walk, Run!
Susmitha Vakkalanka
 
DevOps as-a-Service (DaaS) value
Marc Hornbeek
 
Agile-plus-DevOps Testing for Packaged Applications
Worksoft
 
Continuous Delivery Distilled
Matt Callanan
 
Relieveing the Testing Bottle Neck - Webinar
Cprime
 
DevOps Evolution - The Next Generation ?
Marc Hornbeek
 
Stc 2016 regional-round-ppt-automation testing with devops in agile methodolgy
Archana Krushnan
 

Similar to Continuous testing webinar 041017 slideshare (20)

PDF
Patterns and Practices of a Successful DevOps Transformation
Chef
 
PPTX
How to Fit Performance Testing into a DevOps Environment
Neotys
 
PDF
Automated Testing in DevOps - Enhancing Software Quality and Speed.pdf
Urolime Technologies
 
PPTX
DevOps - Motivadores e Benefícios
Flávio Secchieri Mariotti
 
PDF
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
PPTX
Testing in the new age of DevOps
Moataz Mahmoud
 
PDF
No Devops Without Continuous Testing
Parasoft
 
PPTX
Dev ops != Dev+Ops
Shalu Ahuja
 
PPTX
The Evolution of Test Automation for DevOps
TEST Huddle
 
PPTX
DevOps_service.pptx
phamvinhcntt
 
PDF
Pivotal korea transformation_strategy_seminar_enterprise_dev_ops_20160630_v1.0
minseok kim
 
PPTX
Engineering DevOps Right the First Time
Marc Hornbeek
 
PPTX
week2.pptx Internet communication Technology
WaseemHanif8
 
PDF
DOO-009_Powering High Velocity Development for your Infrastructure
decode2016
 
PDF
The Journey to DevOps #MFSummit2017
Micro Focus
 
PDF
How TestOps is Redefining QA Testing in DevOps.pdf
Kanika Vatsyayan
 
PPTX
Waseem Hanif SQE Project.pptx software Quality
WaseemHanif8
 
PDF
QualiSystems-Brief TestShell
Amy Ho, PMP, PMI-ACP
 
PDF
Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
Skytap Cloud
 
PDF
Engineering DevOps to meet Business Goals
Marc Hornbeek
 
Patterns and Practices of a Successful DevOps Transformation
Chef
 
How to Fit Performance Testing into a DevOps Environment
Neotys
 
Automated Testing in DevOps - Enhancing Software Quality and Speed.pdf
Urolime Technologies
 
DevOps - Motivadores e Benefícios
Flávio Secchieri Mariotti
 
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
Testing in the new age of DevOps
Moataz Mahmoud
 
No Devops Without Continuous Testing
Parasoft
 
Dev ops != Dev+Ops
Shalu Ahuja
 
The Evolution of Test Automation for DevOps
TEST Huddle
 
DevOps_service.pptx
phamvinhcntt
 
Pivotal korea transformation_strategy_seminar_enterprise_dev_ops_20160630_v1.0
minseok kim
 
Engineering DevOps Right the First Time
Marc Hornbeek
 
week2.pptx Internet communication Technology
WaseemHanif8
 
DOO-009_Powering High Velocity Development for your Infrastructure
decode2016
 
The Journey to DevOps #MFSummit2017
Micro Focus
 
How TestOps is Redefining QA Testing in DevOps.pdf
Kanika Vatsyayan
 
Waseem Hanif SQE Project.pptx software Quality
WaseemHanif8
 
QualiSystems-Brief TestShell
Amy Ho, PMP, PMI-ACP
 
Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
Skytap Cloud
 
Engineering DevOps to meet Business Goals
Marc Hornbeek
 
Ad

More from QualiQuali (12)

PPTX
Accelerating Network Transformation with Self-service Environments and Orches...
QualiQuali
 
PDF
GDPR Compliance Countdown - Is your Application environment ready?
QualiQuali
 
PDF
implanting DevOps at scale using dynamic test environments
QualiQuali
 
PDF
The holy grail of automated labs self-service with l1 switching
QualiQuali
 
PPTX
Cloud Sandboxing Solution for Cyber Ranges
QualiQuali
 
PPTX
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
QualiQuali
 
PPTX
2016 quali continuous testing quest for quality conference
QualiQuali
 
PPTX
2016 quali dev ops_delivery of things_conference
QualiQuali
 
PDF
Quali webinar de-mystifyind dev_ops-a practitioner’s perspective
QualiQuali
 
PPTX
Mind the Gap: Crossing the DevOps Chasm
QualiQuali
 
PPTX
Mind the Gap: Crossing the DevOps Chasm
QualiQuali
 
PPTX
Heterogeneous Self-Service Automation for SDN Dev/Test
QualiQuali
 
Accelerating Network Transformation with Self-service Environments and Orches...
QualiQuali
 
GDPR Compliance Countdown - Is your Application environment ready?
QualiQuali
 
implanting DevOps at scale using dynamic test environments
QualiQuali
 
The holy grail of automated labs self-service with l1 switching
QualiQuali
 
Cloud Sandboxing Solution for Cyber Ranges
QualiQuali
 
Webinar leveraging-cloud-sandboxes-with-ansible-jenkins-j frog
QualiQuali
 
2016 quali continuous testing quest for quality conference
QualiQuali
 
2016 quali dev ops_delivery of things_conference
QualiQuali
 
Quali webinar de-mystifyind dev_ops-a practitioner’s perspective
QualiQuali
 
Mind the Gap: Crossing the DevOps Chasm
QualiQuali
 
Mind the Gap: Crossing the DevOps Chasm
QualiQuali
 
Heterogeneous Self-Service Automation for SDN Dev/Test
QualiQuali
 
Ad

Recently uploaded (20)

PDF
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
PDF
IDM Crack with Internet Download Manager 6.42 Build 43 with Patch Latest 2025
bashirkhan333g
 
PDF
Revenue streams of the Wazirx clone script.pdf
aaronjeffray
 
PDF
[Solution] Why Choose the VeryPDF DRM Protector Custom-Built Solution for You...
Lingwen1998
 
PDF
Automate Cybersecurity Tasks with Python
VICTOR MAESTRE RAMIREZ
 
PDF
SAP Firmaya İade ABAB Kodları - ABAB ile yazılmıl hazır kod örneği
Salih Küçük
 
PPTX
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
PDF
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
PPTX
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
PPTX
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
Download Canva Pro 2025 PC Crack Full Latest Version
bashirkhan333g
 
PDF
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
PPTX
Human Resources Information System (HRIS)
Amity University, Patna
 
PDF
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
PPTX
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
PDF
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
PDF
Thread In Android-Mastering Concurrency for Responsive Apps.pdf
Nabin Dhakal
 
PPTX
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
PDF
Driver Easy Pro 6.1.1 Crack Licensce key 2025 FREE
utfefguu
 
PDF
Alexander Marshalov - How to use AI Assistants with your Monitoring system Q2...
VictoriaMetrics
 
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
IDM Crack with Internet Download Manager 6.42 Build 43 with Patch Latest 2025
bashirkhan333g
 
Revenue streams of the Wazirx clone script.pdf
aaronjeffray
 
[Solution] Why Choose the VeryPDF DRM Protector Custom-Built Solution for You...
Lingwen1998
 
Automate Cybersecurity Tasks with Python
VICTOR MAESTRE RAMIREZ
 
SAP Firmaya İade ABAB Kodları - ABAB ile yazılmıl hazır kod örneği
Salih Küçük
 
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
Download Canva Pro 2025 PC Crack Full Latest Version
bashirkhan333g
 
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
Human Resources Information System (HRIS)
Amity University, Patna
 
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
Thread In Android-Mastering Concurrency for Responsive Apps.pdf
Nabin Dhakal
 
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
Driver Easy Pro 6.1.1 Crack Licensce key 2025 FREE
utfefguu
 
Alexander Marshalov - How to use AI Assistants with your Monitoring system Q2...
VictoriaMetrics
 

Continuous testing webinar 041017 slideshare

  • 2. 2 Shashi Kiran CMO Quali Marc Hornbeek “DevOps-the-Gray” Principal DevOps Consultant, Trace3 Author, DevOps Test Engineering Course The DevOps Institute HOST EXPERT PANEL Pascal Joly Director, Technology Partnerships Quali
  • 3. Agenda 3 1 Introduction 2 Principles of Continuous Testing 3 Implementing CT 4 Demo: Continuous Testing 5 Q&A Continuous Testing The “Missing Link”
  • 4. 01 02 03 Venture-funded start-up focused on Cloud-centric DevOps and BizOps automation Delivers “Sandboxes” - Replicas of production environments Strong Innovation DNA – Trusted by customers worldwide About Quali 4 300+ customers – Cloud providers, Telcos, Enterprises – across FSI, Retail, Healthcare and Government ~115 employees in USA, Israel and Europe focused on serving customers worldwide300+
  • 7. The Rising Importance of (Continuous) Testing SPEED QUALITY TEST
  • 8. Traditional Testing Source: www.saucelabs.com • Siloed development and testing • Waterfall approach • Long lead times between identifying and solving bugs • High risk when bugs are missed • Rework and delays
  • 9. What is Continuous Testing (CT)? • Quality assessment strategy • Incremental changes tested “continuously” • Automated tests (large %) • DevOps Toolchain integration • Production equivalent test environment stages • End-to-end testing across the pipeline
  • 10. CT – A sign of DevOps Maturity 1 2 3 4 5 Processes characterized for specific projects. Organization is often reactive Automated Test Execution Projects tailor processes from organizations development methodology Test Framework pipeline Processes are measured and controlled Test-driven Release Tools Focus on process improvement Dynamic Test and Environment Blueprints Process is unpredictable, poorly controlled and reactive Manual Adhoc Tests
  • 11. Hurdles to Implement Continuous Testing Low % test automation Budget Missing/fragmented toolchain Production equivalent test environment stages Product Leadership Culture Training and Expertise Collaboration – Dev/Sec/Ops Budget Rewards and Incentives People Continuous Test planning End-to-end Strategy End-to-end testing across pipeline Process CI CT CD
  • 13. CT provides Assessment of CD Stage Artifacts
  • 14. Continuous Testing – Implementation Strategy Adopt 5 test Tenets Org. and Culture Streamline Process and Workflow Integrating CT Infra. Environment Orchestration and Test Automation 1 2 3 4 5
  • 15. 1. Five Tenets of Continuous Testing 2 Arrange tests so most likely problems are found in earlier stage of pipeline. Fail Early 1 Shift Left Conduct each test as early in the pipeline as possible. 4 Focus on most important tests and results Relevance 3 Test Fast Arrange tests to run in quick cycles 5 Fail Often Run tests frequently and with many different conditions
  • 16. 2. Build Organizations and Culture • DevOps does not mandate roles • Teams decide role assignments • Checklist help ensure tasks are covered. • Multiple checklists may be needed to cover different stages of the pipeline such as Pre-Flight, Integration, Delivery, Deployment, etc.
  • 17. 3. Process and Workflow Practices • Streamline Procurement, install and support: o Infrastructure o Tools o Integration of test tools into tool frameworks o Operating the DevOps pipeline • Align Rewards and Incentives • Train on Automation Skills and Tools
  • 18. 4. Integrated DevOps Test Infrastructure PLAN Collaboratively plan integration into each stage of DevOps Infrastructure TEST Automatically trigger tests from master DevOps framework. ASSESS Run DevOps test assessment for each stage to provide feedback and support workflow decisions. Stage 2
  • 19. 4A. Testing Triggers – Infrastructure “Glue” • Continuous Test framework is integrated with the DevOps master framework. • DevOps test automation can be performed by the DevOps master framework directly or may be implemented using special purpose DevOps test framework tools. • Example: DevOps testing for the build stage is triggered when the DevOps master framework triggers the DevOps test framework. TEST
  • 20. 4B.Asssessment Actions Processing • CT test executions produce verdicts. • Verdicts provide decision support data for analytics that determine if a change is good enough to proceed to the next stage in the DevOps pipeline or needs to be remediated before proceeding. • Analytics may be basic verdict reports or more sophisticated verdict trend analysis. Verdicts Verdict Trends Analytics ASSESS
  • 21. Automatically setup the test environment and resources (physical and virtual) to match the requirements of a test. Environment Orchestration 5. Test Environment Orchestration and Automation Execute test tasks without manual work required. Test Automation
  • 22. 5A. Environment Orchestration • Test resources vary depending on the complexity and risk of change being tested. • A flexible lab environment employs orchestration tools to elastically scale physical and virtual resources for a specific test.
  • 23. 5B. Test Automation • Create new automated tests at the same time changes are created. • Run automated tests matched to specific content being changed. • Produce test results automatically to support in-process DevOps test assessments. Continuous testing uses automation to execute tests quickly and consistently and to keep up with the frequent changes that are processed through DevOps pipelines.
  • 24. DevOps Orchestration Stack Testing Orchestration simplifies management of test resource topologies through test environment blueprints. Test Orchestration (Dev) Build Automation Packaging Config Mgmt. Dev & Integration DeliveryPackaging Test Orchestration (Staging and Production) Test Orchestration (Acceptance) Test Resources Cloud CI/CD Pipeline: Application Release Automation & ALM Systems
  • 25. CI/CD with CT – Leveraging Cloud Sandboxes Pipeline Automic Quali Sandbox Software Load Security Integration Staging/ Production Dev CI/CD Pipeline: Application Release Automation & ALM Systems
  • 26. Quali CloudShell -Sandboxes Blueprint Modeling Automation & Orchestration Reservation & Scheduling Self-Service Catalog Built-In Networking BI & Analytics REST API EMBEDDED GUI DevOps Plug-Ins BUILD ANY ENVIRONMENT. DEPLOY AS-A-SERVICE Bare Metal TO ANY CLOUD* ENVIRONMENTFEATURESINTERACE Applications Data Test Tools Physical Infra. Virtual Infra. Services Network Storage Compute Security CLOUD SANDBOX
  • 28. Test Strategies for Continuous Deployment Reference: https://vwo.com/ab-testing/ A/B Testing Canary Testing Blue/Green Testing
  • 29. Continuous Testing - Benefits Reduced test Time and test wait time Improved Test Results Analysis Cost Efficiency of Shared Resources Improved SLAs CONTINUOUS TESTING Rapid Test Environment Recovery
  • 30. Summary 1 2 3 Don’t compromise between speed and quality. Support Continuous Delivery Initiative with Continuous Testing Cloud Sandboxes can be orchestrate and automate environments for continuous test CONTINUOUS TESTING From “Missing Link” to “Glue”
  • 31. 2017 DevOps and Cloud Survey http://info.quali.com/2017-devops-and-cloud-survey
  • 32. For More Information: www.quali.com www.trace3.com www.devopsinstitute.com Quali Trace3 DevOps Institute

Editor's Notes

  • #3: Principal Consultant – DevOps at Trace3 39 years engineering and managing solutions for IT infrastructures Speaker on topics including DevOps, Lab-as-a-Service and continuous test automation. Blogger on DevOps.com Author for the DevOps Institute: Continuous Delivery Architect course DevOps Test Engineering course 41-year life member of the IEEE Awarded 2016 most outstanding engineer by IEEE Western United States region 6 for “contributions to the field of automation applied to development and testing of networks, systems, protocols, labs and DevOps”