BKK16-306 ART ii
0 likes664 views
The document outlines the roles and responsibilities of the Linaro Android Runtime Team (ART), detailing their focus on improving both current and upcoming Android releases. It discusses the development cycle, testing procedures, benchmarking methods, and internal and upstream code review processes. Additionally, it highlights tools and practices used for task management and code quality assurance in ART's projects.
BKK16-306 ART ii
- 1. Presented by Date Event ART II New Member Assimilation GuideSerban Constantinescu BKK16-306 March 9, 2016 Linaro Connect BKK16
- 2. Who are we? ● LMG ○ Linaro Android Runtime Team (ART) ● 7 engineers @ ARM ● 2 engineers @ Spreadtrum
- 3. What do we do? ● Linaro ART Tip (Upstream) ○ Focus on improving ART for the next Android release ● Linaro ART Stable (Linaro) ○ Focus on improving ART for the current Android release
- 4. Development Cycle Task Selection Development Internal Code Review Upstream Code Review Testing Benchmarking
- 7. Task Selection ● Jira component ○ ART-Tip, ART-Stable within the LMG project ● Jira labels ○ For classifying the ART subsystems ● Fix version ○ For identifying our current focus ● Kanban ○ With daily scrum meetings (Scrumban) ○ Works well when working on fast-paced upstream projects!
- 8. Kanban Todo In Development Review Done Development Cycle
- 9. Kanban Todo In Development Review Done Increasing Priority Development Cycle
- 10. Kanban
- 11. Development
- 12. Development ● Take a look at Scott Wakeling’s BKK16-302 ● Develop ○ Google’s coding style (take a look at the surrounding code) ○ Simple is better! ● Test ○ Gtests, Java tests, Checker tests ● Benchmark ○ ART-testing framework (write a new benchmark) ● Questions & guidance ○ #linaro-art IRC
- 14. Internal Code Review ● 2 Gerrit instances ○ Public code review (ART-Testing patches) ○ Private code review (ART patches) ● All ART patches ready for internal review ○ Should pass: mma test-art-host && mma test-art-target ● All ART patches ready for upstream review ○ +2 from Linaro ART Leads! ○ +1 from Linaro ART! ○ +1 from Linaro Automation?! ○ Have replies to all comments
- 15. Testing
- 18. Testing ● Thanks Daniel, Milosz, Vishal and [email protected]! ● Tests types ○ Run on every Gerrit patch submission ○ Run on every tree update (every 6hrs) ● Build monitors ○ Linaro ART Tip ○ Linaro ART Stable ● Every test ○ Has a short description ○ Stores useful logs and pinned-manifest.xml
- 19. Test: ARTHostTest ● Description ○ mma test-art-host ○ gtests, java tests, checker tests ○ Builds on host, runs on host ○ Generates Gtests results ● Trigger ○ Every patch submission ○ Every tree update ● When should be run? ○ On every patch submitted for review (automatically triggered)
- 20. Test: ARTTargetTest ● Description ○ mma test-art-target for ARM and ARM64 ○ gtests, java tests, checker tests ○ Builds on host, runs on target ○ Tests run in a chroot like environment ● Trigger ○ Every patch submission ○ Every tree update ● When should be run? ○ On every patch submitted for review (automatically triggered)
- 21. Test: ARTBootImageMeasurements ● Description ○ Builds boot.oat using dex2oat for all the ART targets ○ Builds on host, runs on host ○ Generates compile time and size plots ● Trigger ○ Every tree update ○ Topic trigger repo upload art/ --dest=master/IncludeARTBootImageMeasurements ● When should be run? ○ Concerned about compile time and size
- 22. Test: ARTCodeCoverage ● Description ○ Builds boot.oat using dex2oat for all the ART targets ○ Runs the host gtest ○ Builds on host, runs on host ○ Generates code coverage reports ● Trigger ○ Every tree update ○ Topic trigger repo upload art/ --dest=master/IncludeARTCodeCoverage ● When should be run? ○ Concerned about code paths exercised
- 23. Test: ARTBootToGUI-aosp_arm64 ● Description ○ Builds the aosp_arm64 target and runs it in the emulator ○ Builds on host, runs on host ○ Generates build and boot time plots ● Trigger ○ Every tree update ○ Topic trigger repo upload art/ --dest=master/IncludeBootToGUI ● When should be run? ○ Concerned about booting to GUI
- 24. How to debug a test failure? ● Look at the test trigger ○ Tree update or patch trigger? ● Take a look at ○ Test status ○ Parsed console output ○ The build logs ● Run the tests locally ○ Grab the pinned-manifest.xml ○ Run the test scripts locally (WIP)
- 25. Example: ARTTargetTest Failure ● Test status: 145-alloc-tracking-stress failed ● Trigger type: Tree Update ● Parsed console output test-art-target-run-test-default failed (ARM/ARM64) test-art-target-run-test-optimizing failed (ARM/ARM64) ● Build logs (only need to take a look at default and optimizing logs) /home/buildslave/srv/linaro-art-tip-build-ARTTargetTest/android/art/test/145-alloc-tracking-stress: FAILED! #################### info # Regression test for b/18661622 #################### diffs --- expected.txt 2016-03-02 10:00:23.446375718 +0000 +++ output.txt 2016-03-02 12:15:51.386021604 +0000 @@ -1 +1 @@ -Finishing +Aborted ####################
- 26. Example: ARTTargetTest Failure ● What next? ○ Grab the pinned-manifest.xml ○ Reproduce the failure locally! ○ Fix the failure (see more info in Scott Wakeling’s BKK16-302)
- 27. Benchmarking
- 28. Benchmarking ● Thanks Daniel, Milosz, Vishal and [email protected]! ● Jenkins & LAVA ○ Benchmarking on Nexus 9 (Nexus 5x soon) ● Runs on every patch submitted for review and every tree update ● ART Reports - sent by email ○ Every patch (posted to Gerrit), week, month ● ART Graphs ● Using ART-testing as a workload ○ Performance, compile time and size
- 29. ART Reports
- 30. ART Graphs
- 31. Benchmark: Nexus9-MicroBenchmarks ● Description ○ Jenkins builds a Nexus 9 image and sends it over to Lava ○ LAVA runs the benchmarks for ARM and ARM64 ○ LAVA stores the results in the database and sends reports ● Trigger ○ Every patch submission ○ Every tree update ● When should be run? ○ On every patch submitted for review (automatically triggered)
- 32. Example: Nexus9-MicroBenchmarks Run ● Gerrit Patch Jenkins build System Image ART Report
- 33. Future Work on Testing & Benchmarking ● Testing ○ More stable tests! ○ Ability to run Jenkins scripts locally ○ More tests - Clang sanitizers, dex fuzzing, Valgrind tests, etc. ● Benchmarking ○ More devices - Nexus 5x, 6p, future devices ○ Compilation time and sizes for various apks ○ Benchmark profiles
- 35. Upstream Code Review ● Always use two Android trees! ○ A Linaro tree - for internal review ○ An AOSP tree - for external review ● All patches submitted upstream should ○ Have a +2 in the internal review ○ Small fixes do not need to be re-reviewed internally ○ Reworks and new features need to be re-reviewed internally! ○ Have Linaro ART Leads added as reviewers upstream
- 36. Thanks!