GTAC 2015
5 likes•876 views
This document summarizes key points from a GTAC 2015 conference in Boston on software testing. It discusses the speaker's background and details of the conference, including attendance numbers from various countries and the number of talks. The document then summarizes several talks on testing experience and tools. Common insights emphasized automation and speed. Testing strategies discussed included setting goals, transparency, and prioritizing automation. Specific tools mentioned include Octopus, Monkey, Diffy, Applitools Eyes, Flashback, and RpcReplay. The document concludes with additional testing thoughts and a question/answer section.
GTAC 2015
- 2. ● Software Engineer in Test since 2004 ● Biker ● Photographer ● Me on Internet ○ Twitter: @din0su ○ LinkedIn: dinosu ○ Github: dino-su # whoami Dino Su
- 4. 1562 applications from worldwide
- 5. ● 1362 attendance applications ● 200 speaker applications ● Iseral, Austrilla, South Africa, Finland, Taiwan
- 6. 25 talks in 14 hours
- 7. 32 speakers
- 10. next GTAC will be in 2017 Spring
- 11. Subject #1: Experience Sharing
- 16. ● Agree on metrics ● Progress transparency ● Allow team-based drill down ● Allow team-based metrics additions
- 19. ● product ● engineering ● operaions (Production & Release) Dimensions of Quality
- 21. ● wherever possible ● big focus on mobile/device testing ● integration testing ● no manual regression ● push process Push for automation
- 22. Step3: Set Goals
- 23. ● Start simple: cut by X in Y quarters ● Enforce via team commitments ● adjust as progress achieved Goals
- 26. ● Testing the Tube ● Chromecast Testing ● Nest Automation Infrastructure ● Enabling Streaming Experiments at Netflix ● Effective Testing of a GPS Monitoring Station Receiver ● Chrome OS Test Automation Lab More testing experience talks
- 27. Common Insights Automation & Automation & Automation Speed & Speed & Speed
- 30. The problem
- 31. Solution
- 32. ● Test Host ● Test Target ● Channel ● Signal ○ read ○ write
- 34. ● Question: But why not mock instead of E2E ? ● Answer: It’s only used for core features to increase final confidence for release.
- 37. 1. ease of use 2. Android framework compatibility 3. code coverage 4. fault detection Benchmark Criteria
- 41. Winner
- 42. Monkey test Tips ● How to setup monkey test # setup ./gradlew installDebug installDebugAndroidTest adb shell am instrument -w -e annotation com.kkbox.sqa.fuzz_test.test.Before com.kkbox.sqa.fuzz_test.test/android.support.test.runner.AndroidJUnitRunner # exercise adb shell monkey -p com.skysoft.kkbox.android -v 30000 # teardown adb shell am instrument -w -e annotation com.kkbox.sqa.fuzz_test.test.After com.kkbox.sqa.fuzz_test.test/android.support.test.runner.AndroidJUnitRunner
- 43. ● 5 minutes as maximum (30000 events)
- 48. The tool saved Twitter engineers from being layoff
- 49. ● Applitools Eyes ● The most powerful image comparison engine I've ever seen
- 55. unfortunately, it costs you $99 user / month
- 56. new network mocking tools Flashback & RpcReplay
- 59. It makes integration testing become more pleasure
- 60. Subject #3: Testing thoughts
- 61. component testing Soup Dumplings
- 70. 75% code coverage seems like a right number
- 71. The Fun Part
- 72. 4am fire alarm
- 73. Google Cambridge