Bug deBug Chennai 2012 Talk - Future of testing impact of mobile devices by Somenath Nag

Bug deBug Chennai 2012 Talk - Future of testing impact of mobile devices by Somenath Nag

• 1.
  Future of Testing– Impact of Mobility Devices Somenath Nag 24th March, 2012 Copyright © 2011 Calsoft Labs
• 2.
  • How isMobility Changing the World • Market Opportunity • Challenges in Testing Mobility Applications • Addressing the Challenges
• 3.
  Can we thinkof a day in our life without cellphone? COMMUNICATION UTILITIES/ TOOLS PRODUCTIVITY ENTERTAINMENT
• 4.
  Same is truefor enterprises also Execs Line of Business Managers Task and Business Users Consumers and Ecosystem Mobile E-Commerce Mobile Marketing Self-Service
• 5.
  Enterprises are adoptingmobility faster than ever Numbers and types Mobility is driving the Increasing demand of devices are “consumerization” of for enterprise changing dramatically IT applications • 1 billion • 46% of large • 20% of mobile smartphones and enterprises workers are getting 1.2 billion mobile supporting business apps from workers by 2013 personally-owned app stores today • Large enterprises devices2 • 50% of expect to triple • Billions of organizations plan their smartphone downloads from to deploy mobile user base by 2015 Apple App Store apps by 2012 Source: Gartner, IDC
• 6.
  When every industryis worried about recession, mobility is growing at a supersonic speed 585,000+ Apps 25,000,000,000+ Downloads 400,000+ Apps 10,000,000,000+ Downloads
• 7.
  The unique challengesin testing mobile applications • Diversity in Device Management • Application Usage • UX Design • Network consideration • Application & Device security • Rapid Application Testing
• 8.
  Challenge 1 -Diversity in Device Management • Different application platforms – Each platform has its own standards, application interface, Programming languages – Needs to test applications on each platform for each features – Different certification process for different platforms – Increases time, cost, and effort • Device factors – No standardization of device & form factors – New innovations creating more complexities – Following is a list of the factors that can affect your application behavior: • Screen size • Android OS versions • CPU
• 9.
  Challenge 2 -Paradigm Shift in Application Usage PC Application Mobile Application • Indoor use • Anytime, Anywhere • Application dependent • Context Sensitive • Data entry through key-board • Complex data entry process (key- pad, touch, voice, etc.) • Low or zero interruptions • High interruptions • Low environmental input • High environmental input (Camera, voice, etc.) • Information Processing through • Information processing through normal funnel an inverted funnel
• 10.
  Challenge 3 –Evolving Usage Patterns and innovations are creating new testing challenges • No established mobile software usability culture – different for different platforms • Direct usage of usability principles used in PC based applications • Impact of smaller form factors on usability • Handling interruptions • Interaction with other native and third party applications (camera, voice, music players) • Complexity created by gestures, multi-touch and screen rotation
• 11.
  Challenge 4 -Network Considerations • Unlike the network landscape of the PC environment, the network landscape of a mobile device may have gateways • Applications should be tested for their compatibility with WAP-enabled as well as HTTP-enabled devices • Unpredictable network latency leading to inconsistent data transfer speeds. • Gateways in a wireless network may act as data optimizers that may result in decreased performance for heavy traffic
• 12.
  Challenge 5 -Application and Device Security Threat Model Courtesy: OWASP Project
• 13.
  Challenge 5 -Application and Device Security Risks that Need to be Tested Top 10 Security Risks • Insecure Data Storage • Improper Session Handling • Weak Server Side Controls • Security Decisions Via Untrusted Inputs • Insufficient Transport Layer Protection • Side Channel Data Leakage • Client Side Injection • Broken Cryptography • Poor Authorization and Authentication • Sensitive Information Disclosure 13 Courtesy: OWASP Project
• 14.
  Challenge 6 –Rapid Application Testing • In order to deliver the benefits of faster time to market, RAD environments are used for mobile application development. • Since the time taken for development is reduced by the introduction of RAD tools, builds will be available for testing much earlier • RAD methodology imposes an indirect pressure on testing teams to reduce the testing cycle time without compromising quality and coverage • This pressure becomes more when applications are targeted for multiple platforms simultaneously
• 15.
  A Mobility TestingStrategy Framework for addressing these challenges Network Device Usage Performance Application Landscape Landscape Scenarios Benchmark Category Automation Consideration Security Consideration (WOASP) Test Strategy P Data Centric Apps Device Capability O based Apps Test Execution • Use a Weighted Device Platform Matrix method to identify the most critical hardware/ platform combination to test • Check the end-to-end functional flow in all possible platforms at least once. • Conduct performance testing, GUI testing, and compatibility testing using actual devices. • Measure performance only in realistic conditions of wireless traffic and user load. • Select the right automation test tool
• 16.
  Suggested Guideline forTool Selection • Support for ALL object identification methods • Secure solution • Ability to test devices taken out of a retail store - No Jailbreak/Rooting • Support for all device functionalities such as gestures, security alerts, wake and reboot • Plugs-in to existing test environment • Same test should run on different devices
• 17.
  Q&A
• 18.
  About Calsoft Labs& Alten Calsoft Labs ALTEN SA ■ Wholly owned subsidiary of ALTEN ■ Established in 1988 ■ Outsourced Product Design arm of the ALTEN ■ European Leader in Engineering and Group Technology Consulting services ■ Pioneer in Product Engineering Services ■ 14800 employees worldwide, >90% engineers and consultants ■ Over 300 satisfied customers ■ Located in 16 countries ■ Over 1000 products developed and tested ■ Revenue of € 1 Billion+ for FY 2011 Service Offerings Industry Focus People Locations • Embedded Product • Telecom • 15000+ people • Located in 16 countries Engineering Service • Healthcare • 65% of engineering team • 10 Delivery centers • Software Product • Education with 5 years+ experience • Design & Development Engineering Service • Enterprise Software • 27% Master Degree Centers in • Independent Verification • Automotive holders Bangalore, Mysore & & Validation Service • Media & Entertainment • ALTEN is one of Europe's Chennai • Design Engineering • Semiconductor most dynamic • US Delivery Centre in Service • Aeronautics / space employers, hiring almost Boston, MA • Financial 2,000 new people each year. Our Partners
• 19.
  Calsoft Labs’ TestingPractice Partner ■ 200+ dedicated test professionals ■ Track record ■ QA solutions throughout the SDLC based on TMMi framework ■ Reusable repository of test cases, scenarios, and validation libraries Service Offerings Industry Focus Thought Leadership Mobility Testing • Test Strategy Consulting • Telecom • OWASP Compliance • Field testing and • Embedded Product • Healthcare • Accelerator for OWASP Interoperability Testing • Education Compliance • Carrier validation test • Software Product Testing • Enterprise Software • Home-made Open Source • Protocol and • Test Automation • Automotive Automation Framework Conformance test • ‘On-Demand’ Testing • Media & Entertainment • Device Management • Compliance testing • Semiconductor Testing • Security Testing • Aeronautics / space • Mobile App testing • Financial • Usability Testing