Software Testing Vaidyanathan Ramalingam

Software Testing Vaidyanathan Ramalingam

• 1.
  • Software Testing • Trends & • Transformation http://www.isqtinternational.com Vaidyanathan Ramalingam (Vaidya) http://www.GiveMeDefect.com Director Engineering (Test), http://www.huawei.com Huawei Technologies India Pvt. Ltd., Bangalore Key Note Speech Silver Sponsor ISQT’s Step Auto Conference on Software Testing http://isqtinternational.com/Step-Auto/Bangalore-Details.html 15 Sep 2010 http://www.conformiq.com The Chancery Pavilion, Bangalore, Karnataka, India.
• 2.
  Few pictures fromthe Test Conference Chief Guest of Honour Mr. Gangadharaiah. CP Sr. VP and Global Head of Testing Services Wipro Technologies, Bangalore Guest of Honour Mr. Vaidyanathan R Director Engineering (Test) CRD Business Line Huawei Technologies India Pvt. Ltd, Bangalore Host Mr. Arunkumar Khannur Chairman and CEO ISQT Process & Consulting Services Pvt. Ltd Bangalore
• 3.
  Vaidya’s Profile: Mr. Vaidyanathan Ramalingam (Vaidya) has >20 Years of industry experience primarily in Software Testing domain. His strong & focused product testing experience is mostly from fortune 100+ MNC R&D Centers at Silicon Valley of India, Bengaluru includes Motorola, Nortel Networks ODC (BPL), Siemens Communication Software, Indus EDC (Ness), Intel Technology, HP R&D and Huawei R&D (present). Has diversified work experience with Global Corporate / Culture with USA, Costa Rica, Canada, Israel, Germany, Austria, China, Malaysia, Hong Kong, Philippines, Singapore and India. His domain expertise spread across Telecom/Datacom, IT Infra & Operations Management, Database Development, Semiconductor Manufacturing Automation, Enterprise Asset Management, Service Delivery Management, Competency Development, Consumer Electronics, HP Test Management & Automation Product Suite (worked as R&D Section Manager – Test CoE, BTO, HP Software. Direct reporting relationship with Mercury Interactive (Israel) Management with a focus of Post HP’s acquisition R&D integration). Vaidya is Director Engineering (Test) in Central Research & Development Business Line @ Huawei R&D Center. He is the leader of Test Department & Test Systems Group (Test CoE). Huawei is a global telecommunications solutions provider with long-term partnerships with operators around the world. Huawei actively participates in 91 international standardization organization including ITU, 3GPP, 3GPP2, ETSI, IETF, OMA and IEEE. Huawei holds the world’s #1 position in PCT Patent applications (WIPO 2008) www.huawei.com and is ranked among the top 3 in LTE essential patents. Huawei’s products and solutions have been deployed in over 100 countries and have served 45 of the world's top 50 telecom operators, as well as one third of the world's population. His article was published in “Voice & Data” SAARC edition, May 2010 (Link: TTCN3 brings the promise of better testing ) about TTCN (Testing and Test Control Notation) Ver 3.0, an internationally standardized testing language from ETSI (European Telecommunication Standard Institute). Since Y2008, he is a Honorary Member in Federation of Karnataka Chambers of Commerce & Industry (FKCCI) http://www.fkcci.org Established on 1916 by Bharat Ratna Sir M Visvesvaraya. FKCCI has representation in several Boards & Advisory Committees in Government and Institutions. Vaidya is qualified with Post Graduation (M.Tech – Information Technology), Engineering Graduation (BS - Information Systems) after Technical qualifications Diploma (Electronics & Telecommunication Engineering) & Advanced Diploma (Consumer Electronics), supplemented with management qualification (P.G. Diploma – Marketing Management). He is also interested to write a software testing book and do research in Competency Development Model with Software Application for IT Staff, Enterprise Orgs. Note: The content of this conference presentation is non commercial and based on industry/media information & self interpretations/view points. The data provided may not be full, accurate and latest. The intention is knowledge sharing & to promote the software testing. Share your feedback & suggestions to: [email protected] http://in.linkedin.com/in/rvaidya67
• 4.
  Software Testing Markets • Global Testing Service Market $13B (2008) -> $25B (2013) • Indian Testing Service Market $5B (2008) • Global Testing Service Market $17B (2010) with India $8.5B  50% of global testing service market is managed by India based test community (>95K Test Engineers) • By 2013 Indian Testing Service companies need >15K additional Test Engineers to cater ~15B Test Service Market (ie., >70% global test services market) • Testers will have their growing importance in IT Services & Product Development • QUALITY continue to be the #1 choice of the end users, service providers, network operators, solution integrators and R&D Orgs. Aug’2010 TOI News ~ 8 to 10% h/c is the software tester’s population in India based IT services companies. That comes around >95,000 testers now (Y2010) and still growing!
• 5.
  Software Testing Trends& Transformation Technology Standards/ Testers Protocol Competencies New Test Requirement Techniques Software Testing Trends & Transformation User Test Experience Tools Transformation has impact Deployment Test in all areas Integration Process The intensity varies Test Management & Innovation at all areas
• 6.
  Transformation Areas (1of 5) • Easy / simple use, flexible to meet preferences (tools -> options/settings!) • Provision to customization (ex: SAP-ERP App.), inter operability (different vendor products, Web Services-SOA, SaaS) , incremental upgrade simplicity (ex: telecom systems) Testers need to have big picture & Detailed understanding on the USER DOMAIN • Customer satisfaction, business demand, increasing people’s technology drives base for new requirements! Testers need to carefully understand, document and prepare the 100% use case/scenario coverage tests. Tester interpretation should be superior. User/Acceptance test is the validation (black box); Verification (white box) proves the construction. • Standards / protocols / interfaces / IT continuously emerge to meet the demands in the market place. Example: OTA (ex: Over The Air configuration/software upgrade in telecom) Testers to prepare the test lab environment, simulators, equipments / systems and other test readiness before starting the testing execution. (Ex: Ref standards ETSI, ITU, IEEE, SS7 in Telecom) • Information Technology (OS, HW Platform, DB, MW, Open Sources, Internet, Cloud Computing, Virtualization) Telecom (LTE, 3G..). Tests Focus: Multi Platform/Solution/Co-Existence Testing, Porting, Impact analysis & Test Suite Enhancements (in CI), Optimum Test (avoid duplicate, be effective)
• 7.
  Transformation Areas (2of 5) • First & Foremost important competency for testers is USER DOMAIN KNOWLEDGE • The Testers should be able to visualize the actual product/solution usage scenarios. • At least one or two similar competitor’s products the testers must understand • Setting up the test bed, Customer Reference Labs (helps in pre-sales, POC, bidding) , giving demos to the product owners (in Agile), involving in customer technical support (remote, on-site or during acceptance test etc) gives real time customer paint points visibility to learn! • Systematic approach in Competency Development yields better & consistent results. • Based on the business model, organisation structure, JD (Job Description), R&R (Roles & Responsibilities), complexity of the testing products/applications, test team’s competency maturity the competency development strategy to be derived. This needs to be substantiated with a knowledge retention system. • There are efforts in making Test Body Of Knowledge to streamline the test management practices by experts!  Testers need to remember that they need not compete with developers / system architects. In fact they look at testers (QA engineers) as customer (internal) representatives.  More the business domain understanding, higher the respect testers get from Program Office / Clients / Customers  The testers outcome (results) need to go beyond from just finding defects to user requirements suggestions (on features), workarounds (to avoid code / design / architecture changes), simplifying the product usage/application, product/application technical support, ensuring 5 Nine (99.999%) System Availability (for end users).
• 8.
  Transformation Areas (3of 5) • Considering the evolutions at each transformation areas, the test analysis & techniques need to be chosen & applied. • The following are broadly followed techniques in Test Industry, but specific to the product/ application/solutions/ complexities/testing nature/clients/test architects recommendation, these needs to be selected for Max ROI. White Box Test Design Black & Grey Box Test Design Test Analysis Techniques • Testability Requirement Analysis (req for Techniques • Equivalence Partitioning logs/trace, APIs, Tools, Simulator, Emulator • Coverage Techniques: • Boundary Value & Orthogonal Analysis …) Line/Statement/Decision/ • Algorithm Test •Non Functional Analysis (stress, • Decision Table Test performance, reliability) Condition/Branch … • Syntactic Test •Application / User Scenario Analysis • Semantic Test •Version Succession Impact Analysis • Elementary Comparison Test •Test Type Analysis (security, compatibility, • Data Cycle Test / Entity Life Cycle Test interoperability…) • Process Cycle Test •Feature Interaction Analysis • Program Interface Test •Weak & Strong Analysis: based on impact • Real-Life Test on code & importance to customer, • Error Guessing, Random Test, Idiot frequency of usage (module/feature …) Proofing … •Demo based Analysis (Agile) Other Test Techniques: •Bench Mark Analysis (within the company Documentation/Information Engineering, or with competition products) Help, Localization / Internationalization, •Risk based Analysis GUI & Human usability, Look & Feel, flow •Disaster Recovery Analysis tests, Installation/upgrade/un-inst. tests •5Why RCA, Prototype Analysis, FMEA …
• 9.
  Transformation Areas (4of 5) • Test Tools continue to support testers to make their testing tasks easier & efficient. Like: Test Automation (GUI/CLI/API), Standard/protocol/ Regulatory conformance Tools, Open Source Tools, Model Based Testing (state machines), Continuous Integration (staged software build & testing – for early feedback) tools etc. • Test Management tools helps the test / domain managers / scrum masters (agile) to plan & track the cost/resources/lab/timeline/versions/builds, test execution, defect trends (dashboard) and re-planning. Model Based Testing Source: Source:
• 10.
  Transformation Areas (5of 5) • Emerging Test Ref Standards: Sig Sigma, ISO 9000, CMM, CMMi, Test Maturity Model (TMM), Agile Testing, Lean Methods … • QC-> QA, Independent Validation & Verification, Integration & Validation structure). The key message is that the s/w testing keep evolving as the R&D model/approach is changing. But one trend here is that testers are becoming more closure to customer domain (user acceptance test, demos, technical support, deployment, solution testing etc) On other hand developers testing is becoming comprehensive. Example: TDD (Test Driven Development – Unit test code first – production source code later), instrumentation, code inspection … • RCA (5 Why based Root Cause Analysis) with CAPA (Corrective & Preventive Actions) with multiple/continuous years/versions data (history based analysis) gives additional dimensions for testers to think & act to improve QUALITY! • Deployment & Integration is complex and time consuming for telecom, datacom solutions, complex product migrations/system integrations (ex: BFSI applications, where not a single record to be lost). • Involvement of testers in these production system roll-out (either as observers, contributors or co-ownership with professional services-deployment team) is very useful to gain the domain knowledge. This helps to build the Customer Oriented Scenario Testing. Data Center, Production/Server Room, Host Environment
• 11.
  Key Note Conclusion • It is not Re-Formation….It is transformation • Testers Role is continue to be a key & important, but the expectation keep evolving • Testers need to wear different hats such as: arbitrator, mediator, translator, negotiator with their stakeholders. End result/objective is to ensure QUALITY! • Trend based approach is important and provides multi dimensions. The testing efforts to be effectively utilized with all the constraints to ensure QUALITY! • New version or new account/client’s (services) testing need, depth of understanding on various transformation areas discussed. The value is proven by results of testing otherwise it will be seen as an over-head • Don’t talk about how to stop the release. Instead provide resolutions in-term of ideas by which without major code/design/architecture change, how the product is still released without defects (like: configurations, documentation based approach, GUI adjustments, self protection / providing fool-proof behavior to the system [avoids mis- use] etc) • Core competencies of testers: Domain Competency (business logic), Automation (tools, scripting), process understanding supported with testing attitude & QUALITY mindset. • Keep collaborating/networking through Test Conferences, Test Blogs, Forums, Wiki, Testing Community in your company & industry. Contribute towards delivering with HIGH QUALITY & Be proud to be a software testing/QA professional!
• 12.
  Thank You! Share yourfeedback & suggestions to: [email protected] http://in.linkedin.com/in/rvaidya67