Boston meetup blaze_meter_feb2017
2 likes•10,222 views
The document discusses open source testing tools for mobile applications. It begins by explaining that mobile testing requires automating both the application and the full user environment. It then reviews five popular open source test frameworks - Selenium, Appium, Calabash, Espresso, and XCTest UI - comparing their suitability for different uses like web, native, and hybrid apps. Key criteria that organizations should consider when choosing a framework include supporting multiple frameworks, flexibility, autonomy, full end-to-end coverage, and unattended reliable testing. The document concludes with a demonstration of a test automation tool called Quantum that integrates various open source frameworks.
Boston meetup blaze_meter_feb2017
- 1. Boston Best Practices using open source for testing Eran Kinsbruner, @ek121268 Amir Rozenberg, @Amir_Rozenberg
- 2. SHIFTING TO OPEN SOURCE, BENEFITS MOBILE OSS MARKET LANDSCAPE, TOOLS, COMPARISON DEMO!!! OSS: BEYOND TESTING Q&A
- 3. What is automated Mobile Web Automation target Cannot be automated 60%-90%20% 30% 100% The gap we want to address Degree of automation Increased complexity and no timeMany teams with varying set of skills 2/8/2017 TestersDevTestersCoders
- 5. A Day in A Life –Real User Environments Is Essential Can Any OSS Framework Handle Such Requirements? Automation ought to cover the full user environment not just the Application Under Test!
- 6. Which Tool is the best fit for you? SELENIUM APPIUM CALABASH ESPRESSO XCTEST UI When looking at today’s open-source mobile test automation landscape, there are five highly-adopted test frameworks. Each tool has advantages for your mobile and web testing depending on your needs.
- 7. Making the Decision A Comparison Detailed analysis of each test automation tool: Sometimes More Than A Single Framework is The Right Answer
- 8. Selenium • The best choice for web test automation teams testing for responsive web design or stand-alone web sites • Less suitable for developing unit testing, making this framework less appealing for developers • On its own, Selenium core test reports are not highly informative and lack unique insights
- 9. Appium • Best suited for QA teams testing the functionality of native, mobile web and mobile hybrid apps across iOS and Android • Less suitable for developers who wish to develop and perform unit testing • Appium reports are a bit limited from a debugging and fast feedback loop perspective, and do not include videos, network logs and key vitals information
- 10. Calaba.sh • Designed for organizations that work in behavior-driven development workflows • Does not cover the entire mobile system level events • Offers an easy path to both develop and test features in parallel, all in an easy user-flow based language • Appealing for both dev and QA practitioners • Provides solid insights and reports to both dev and QA teams
- 11. Espresso & XCTest UI • Both requires a separate Test App executable • Espresso is for Android and XCTest is for iOS – not a single script • Both tools are fully integrated into development IDEs such as Android Studio/Xcode, and offer very easy-to-develop techniques, including test recorders • Fully maintained by Google and Apple, which assures they always support the latest OS features so developers can stay ahead of the market and test accordingly • Support both unit testing types and functional UI testing • Both are app-context only, which limits their abilities to test for user condition scenarios XCTEST UIESPRESSO
- 12. Things to Consider • Lab: Support for variety of frameworks is important for enterprises • Flexibility • Autonomy • Complete E2E coverage • Unattended reliable testing • Automation: support for multiple OSS frameworks becomes important • Different “persona’s” need different tools under 1 lab • Cross platform scripting • Robust framework to support high test coverage • Analysis: Digital reporting is necessary for teams to take action and fix issues faster & earlier When choosing a test automation tool
- 15. Area Components Quantum Management Reporting Execution Manager Test Development Test Authoring (IDE+Language) Object Repository Object Locator/Spy Appium/Selenium ✓OSS = ✓OSS ✓OSS http://projectQuantum.io
- 16. Digital – Responsive Web Application
- 17. DEMO – BDD OSS Framework
- 18. Few other OSS-made mobile testing toys Production Insight/ ”Testing in Production” Network features: HAR file & NETEM Some others: • Security scans: https://github.com/MobSF/Mobile- Security-Framework-MobSF • Deep visual analysis • Etc.