Large Scale Test Automation

Download as PPTX, PDF

0 likes62 views

This document summarizes a presentation on secrets of successful test automation. It discusses scheduling automated tests to run every night for 10-20 minutes and limiting resources to 1GB RAM, no persistent storage, and 10MB network bandwidth. Standard reports on endpoints, response times, requests, and errors are shown. Measuring UI performance includes page load time with different networks, response traffic size, request counts, and centralized monitoring. Tools mentioned include Selenium Grid, Selenoid, and Grafana for scaling acceptance tests across browsers and versions.

Technology

Secrets of
Successful Test Automation
Narek Arushanov - Ashot Karapetyan
Barcamp 2018
Synergy International Systems

Agenda
Backend Performance
UI Performance
Scaling Acceptance Test

Requirements
1. Periodicity
2. Resource Allocation.
3. Summary Reports
4. Centralized Monitoring.

Scheduled Execution
Every night
100 rps
10-20 min

Limits
1 GB RAM
No Pers.
Storage
10Mb
network
1 core

Standard Reports
Service endpoint Scale rps
95p in
millis
Total requests
count
# of Errors
ml LoadAllLanguages 1 100 19 59547 0
LoadLanguagesById 1 100 29 59548 0
LoadMessagesByKeys 1 100 92 59547 0
LoadMessagesByLanguage
Id 1 100 13 59548 0

Measuring UI Performance
1. Page load time with different network speed
2. Response traffic size
3. Requests count
4. Automated
5. Centralized monitoring

UI performance tools
1. sitespeed.io
2. Navigation Timing API
3. BrowserMob Proxy + Selenium

{HAR} {HAR}{HAR}
Graphite Database
Grafana

Scaling Acceptance Tests
1. Parallel Test Execution
2. Multi-browser
3. Browser Versions

Ways to test in browsers
1. Cloud web testing platform
BrowserStack Sauce Labs Testing Bot

Ways to test in browsers
1. Cloud web testing platform
2. Selenium Grid

Problems
1. Hub is crushing.
2. Different browser versions.
3. Temp folders
4. Window focus
5. etc …

Ways to test in browsers
1. Cloud web testing platform
2. Selenium Grid
3. Selenoid

Advantages
1. Stability
2. Tests run in isolated container
3. Run tests in different browser versions
4. Tmpfs support
5. Hot configuration reload

Resources control
1. Limit total number of sessions per host
2. Queues
3. Limit CPUs per container
4. Limit RAM per container

More Related Content

What's hot (19)

PPTX

Visibility-from web application interface to the databaseManageEngine, Zoho Corporation

PPTX

SignalR OverviewMichael Sukachev

PDF

Continuous Self-Updating Query Results over Dynamic Linked DataRuben Taelman

PDF

Flink Forward San Francisco 2019: Real-time Processing with Flink for Machine...Flink Forward

PDF

Consul scaleAriel Moskovich

PPTX

Server side scalable web performancePer Ökvist

PPTX

Building the right website monitoring strategyManageEngine, Zoho Corporation

PPTX

Apinizer - Full API Lifecycle and Integration Platform Mustafa Yildiz

PPTX

Real-time Communications with SignalRShravan Kumar Kasagoni

PDF

Continuously Updating Query Results over Real-Time Linked DataRuben Taelman

PPT

Intro to signalRMindfire Solutions

PPTX

Server Monitoring 101Shovon Paulinus Rozario

PPTX

Cloud applications monitoring in digital transformation eraManageEngine, Zoho Corporation

PDF

Introduction to SignalRUniversity of Hawai‘i at Mānoa

PPTX

signalrOwen Chen

PPTX

Introduction to web socketHarunRRayhan

PPTX

SignalR for ASP.NET DevelopersShivanand Arur

PPTX

Building Realtime Web Applications With ASP.NET SignalRShravan Kumar Kasagoni

PPTX

Analysing high throughput data in real timeHotstar

Visibility-from web application interface to the databaseManageEngine, Zoho Corporation

SignalR OverviewMichael Sukachev

Continuous Self-Updating Query Results over Dynamic Linked DataRuben Taelman

Flink Forward San Francisco 2019: Real-time Processing with Flink for Machine...Flink Forward

Consul scaleAriel Moskovich

Server side scalable web performancePer Ökvist

Building the right website monitoring strategyManageEngine, Zoho Corporation

Apinizer - Full API Lifecycle and Integration Platform Mustafa Yildiz

Real-time Communications with SignalRShravan Kumar Kasagoni

Continuously Updating Query Results over Real-Time Linked DataRuben Taelman

Intro to signalRMindfire Solutions

Server Monitoring 101Shovon Paulinus Rozario

Cloud applications monitoring in digital transformation eraManageEngine, Zoho Corporation

Introduction to SignalRUniversity of Hawai‘i at Mānoa

signalrOwen Chen

Introduction to web socketHarunRRayhan

SignalR for ASP.NET DevelopersShivanand Arur

Building Realtime Web Applications With ASP.NET SignalRShravan Kumar Kasagoni

Analysing high throughput data in real timeHotstar

Similar to Large Scale Test Automation (11)

PDF

What is Selenium Automation Testing.pdfAnanthReddy38

PDF

10 Reasons Why Selenium is used for Test Automation -Infographic.pdfAnanthReddy38

PDF

Proven Approaches to AI-Powered E2E Testing.pdfApplitools

PDF

Automation Best PracticesSauce Labs

PDF

A Test Automation Platform Designed for the FutureApplitools

PDF

Applitools Autonomous 2.0 Sneak Peek.pdfApplitools

PPTX

How to Optimise Continuous TestingSauce Labs

PDF

What is Selenium Testing.pdfAnanthReddy38

PDF

How to test performance and not die tryingDiego Cardozo

PDF

Selenium Testing: A Comprehensive Guide to Automated Web TestingpCloudy

PDF

Understanding the Basics of Automation Testing.pdfkhushnuma khan

What is Selenium Automation Testing.pdfAnanthReddy38

10 Reasons Why Selenium is used for Test Automation -Infographic.pdfAnanthReddy38

Proven Approaches to AI-Powered E2E Testing.pdfApplitools

Automation Best PracticesSauce Labs

A Test Automation Platform Designed for the FutureApplitools

Applitools Autonomous 2.0 Sneak Peek.pdfApplitools

How to Optimise Continuous TestingSauce Labs

What is Selenium Testing.pdfAnanthReddy38

How to test performance and not die tryingDiego Cardozo

Selenium Testing: A Comprehensive Guide to Automated Web TestingpCloudy

Understanding the Basics of Automation Testing.pdfkhushnuma khan

Recently uploaded (20)

PDF

DevBcn - Building 10x Organizations Using Modern Productivity MetricsJustin Reock

PDF

Building Real-Time Digital Twins with IBM Maximo & ArcGIS IndoorsSafe Software

PPTX

WooCommerce Workshop: Bring Your LaptopLaura Hartwig

PDF

Jak MŚP w Europie Środkowo-Wschodniej odnajdują się w świecie AIdominikamizerska1

PDF

Achieving Consistent and Reliable AI Code Generation - Medusa AImedusaaico

PDF

Why Orbit Edge Tech is a Top Next JS Development Company in 2025mahendraalaska08

PDF

HubSpot Main Hub: A Unified Growth PlatformJaswinder Singh

PDF

How Startups Are Growing Faster with App Developers in Australia.pdfIndia App Developer

PPTX

Building a Production-Ready Barts Health Secure Data Environment Tooling, Acc...Barts Health

PDF

CIFDAQ Token Spotlight for 9th July 2025CIFDAQ

PDF

Using FME to Develop Self-Service CAD Applications for a Major UK Police ForceSafe Software

PDF

HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...mcastillo49

PDF

Windsurf Meetup Ottawa 2025-07-12 - Planning Mode at Reliza.pdfPavel Shukhman

PDF

Presentation - Vibe Coding The Future of Techyanuarsinggih1

PDF

NewMind AI Journal - Weekly Chronicles - July'25 Week IINewMind AI

PPTX

✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨SanjeetMishra29

PDF

Building Resilience with Digital Twins : Lessons from KoreaSANGHEE SHIN

PDF

Smart Trailers 2025 Update with History and OverviewPaul Menig

PPTX

Building Search Using OpenSearch: Limitations and WorkaroundsSease

PDF

New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025BookNet Canada

DevBcn - Building 10x Organizations Using Modern Productivity MetricsJustin Reock

Building Real-Time Digital Twins with IBM Maximo & ArcGIS IndoorsSafe Software

WooCommerce Workshop: Bring Your LaptopLaura Hartwig

Jak MŚP w Europie Środkowo-Wschodniej odnajdują się w świecie AIdominikamizerska1

Achieving Consistent and Reliable AI Code Generation - Medusa AImedusaaico

Why Orbit Edge Tech is a Top Next JS Development Company in 2025mahendraalaska08

HubSpot Main Hub: A Unified Growth PlatformJaswinder Singh

How Startups Are Growing Faster with App Developers in Australia.pdfIndia App Developer

Building a Production-Ready Barts Health Secure Data Environment Tooling, Acc...Barts Health

CIFDAQ Token Spotlight for 9th July 2025CIFDAQ

Using FME to Develop Self-Service CAD Applications for a Major UK Police ForceSafe Software

HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...mcastillo49

Windsurf Meetup Ottawa 2025-07-12 - Planning Mode at Reliza.pdfPavel Shukhman

Presentation - Vibe Coding The Future of Techyanuarsinggih1

NewMind AI Journal - Weekly Chronicles - July'25 Week IINewMind AI

✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨SanjeetMishra29

Building Resilience with Digital Twins : Lessons from KoreaSANGHEE SHIN

Smart Trailers 2025 Update with History and OverviewPaul Menig

Building Search Using OpenSearch: Limitations and WorkaroundsSease

New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025BookNet Canada

Large Scale Test Automation

1. Secrets of Successful Test Automation Narek Arushanov - Ashot Karapetyan Barcamp 2018 Synergy International Systems

2. Agenda Backend Performance UI Performance Scaling Acceptance Test

3. Requirements 1. Periodicity 2. Resource Allocation. 3. Summary Reports 4. Centralized Monitoring.

4. Scheduled Execution Every night 100 rps 10-20 min

5. Limits 1 GB RAM No Pers. Storage 10Mb network 1 core

6. Tools

7. Percentiles

8. Standard Reports Service endpoint Scale rps 95p in millis Total requests count # of Errors ml LoadAllLanguages 1 100 19 59547 0 LoadLanguagesById 1 100 29 59548 0 LoadMessagesByKeys 1 100 92 59547 0 LoadMessagesByLanguage Id 1 100 13 59548 0

9. Fail

10. OK

11. Centralized Monitoring

12. Measuring UI Performance 1. Page load time with different network speed 2. Response traffic size 3. Requests count 4. Automated 5. Centralized monitoring

13. UI performance tools 1. sitespeed.io 2. Navigation Timing API 3. BrowserMob Proxy + Selenium

14. HAR file structure

15. {HAR} {HAR}{HAR} Graphite Database Grafana

16. Page Load Time

17. Request Count

18. Scaling Acceptance Test

19. Scaling Acceptance Tests 1. Parallel Test Execution 2. Multi-browser 3. Browser Versions

20. Ways to test in browsers 1. Cloud web testing platform BrowserStack Sauce Labs Testing Bot

21. Ways to test in browsers 1. Cloud web testing platform 2. Selenium Grid

22. Problems 1. Hub is crushing. 2. Different browser versions. 3. Temp folders 4. Window focus 5. etc …

23. Ways to test in browsers 1. Cloud web testing platform 2. Selenium Grid 3. Selenoid

24. How to work solenoid

25. How to work solenoid

26. Selenium Grid UI

27. Selenoid UI

28. Advantages 1. Stability 2. Tests run in isolated container 3. Run tests in different browser versions 4. Tmpfs support 5. Hot configuration reload

29. Resources control 1. Limit total number of sessions per host 2. Queues 3. Limit CPUs per container 4. Limit RAM per container

30. Cluster Solution

31. Cheers! www.synisys.com

Editor's Notes

#2: Software engi. in test
#4: Response time 1s
#5: Response time 1s
#7: Locust Scalability, because you can run Locust distributed with many agents. Based on gevent simulate thousand of users in single process
#8: orinak 10ic 5 1s 2 3s. 3 4s nkarel estex
#12: poxel chart@
#15: Verjum asel en inch karox enq chapel
#21: Sakayn mer tester@ run enq talis mer development envirementum, ev chenq uzum kaxvacutyun unenanq ayl servisneric.
#24: Selenoid @ nuyn selenium serveri implementacian e , go lezvov grac. vor@ aveli arag e ev aveli tetev. ira hamar ka shat harmar dockerov realizacia