[tools/perf] Remove memory.top_10_mobile_stress
The benchmark is now superceeded by memory.long_running_dual_browser_test
(in contrib).
Also update the docs on memory benchmarks.
Bug: 713338
Change-Id: I41b4a5b520878805dfcb0cfd30e3eaa2830dc2d8
Reviewed-on: https://chromium-review.googlesource.com/518014
Reviewed-by: Ned Nguyen <[email protected]>
Reviewed-by: Primiano Tucci <[email protected]>
Commit-Queue: Juan Antonio Navarro Pérez <[email protected]>
Cr-Commit-Position: refs/heads/master@{#475864}diff --git a/docs/memory-infra/memory_benchmarks.md b/docs/memory-infra/memory_benchmarks.md
index aa50645..a181588 100644
--- a/docs/memory-infra/memory_benchmarks.md
+++ b/docs/memory-infra/memory_benchmarks.md
@@ -187,36 +187,35 @@
### memory.top_10_mobile
-The *top-10-mobile* benchmarks are in the process of being deprecated
+The [memory.top_10_mobile][memory_py] benchmark is in the process of being deprecated
in favor of system health benchmarks. This process, however, hasn't been
finalized and currently they are still the reference benchmark used for
decision making in the Android release process. Therefore, **it is important
-to diagnose and fix regressions caught by these benchmarks**.
+to diagnose and fix regressions caught by this benchmark**.
-* [memory.top_10_mobile][memory_py] - Cycle between:
+The benchmark's work flow is:
- - load a page on Chrome, wait for it to load, [force garbage collection
- and measure memory][measure];
- - push Chrome to the background, force garbage collection and measure
- memory again.
+- Cycle between:
- Repeat for each of 10 pages *without closing the browser*.
+ - load a page on Chrome, wait for it to load, [force garbage collection
+ and measure memory][measure];
+ - push Chrome to the background, force garbage collection and measure
+ memory again.
- Close the browser, re-open and repeat the full page set a total of 5 times.
+- Repeat for each of 10 pages *without closing the browser*.
- Story groups are either `foreground` or `background` depending on the state
- of the browser at the time of measurement.
+- Close the browser, re-open and repeat the full page set a total of 5 times.
-* [memory.top_10_mobile_stress][memory_py] - same as above, but keeps a single
- instance of the browser open for the whole duration of the test and
- *does not* force any garbage collection.
+- Story groups are either `foreground` or `background` depending on the state
+ of the browser at the time of measurement.
-The main difference to watch out between these and system health benchmarks is
+The main difference to watch out between this and system health benchmarks is
that, since a single browser instance is kept open and shared by many
individual stories, they are not independent of each other. In particular, **do
not use the `--story-filter` argument when trying to reproduce regressions**
on these benchmarks, as doing so will affect the results.
+[memory_py]: https://cs.chromium.org/chromium/src/tools/perf/benchmarks/memory.py
[measure]: https://github.com/catapult-project/catapult/blob/master/telemetry/telemetry/internal/actions/action_runner.py#L133
### Dual browser benchmarks
@@ -224,16 +223,16 @@
Dual browser benchmarks are intended to assess the memory implications of
shared resources between Chrome and WebView.
-* [memory.dual_browser_test][memory_py] - cycle between doing Google searches
- on a WebView-based browser (a stand-in for the Google Search app) and
- loading pages on Chrome. Runs on Android devices only.
+* [memory.dual_browser_test][memory_extra_py] - cycle between doing Google
+ searches on a WebView-based browser (a stand-in for the Google Search app)
+ and loading pages on Chrome. Runs on Android devices only.
Story groups are either `on_chrome` or `on_webview`, indicating the browser
in foreground at the moment when the memory measurement was made.
-* [memory.long_running_dual_browser_test][memory_py] - same as above, but the
- test is run for 60 iterations keeping both browsers alive for the whole
- duration of the test and without forcing garbage collection. Intended as a
- last-resort net to catch memory leaks not apparent on shorter tests.
+* [memory.long_running_dual_browser_test][memory_extra_py] - same as above,
+ but the test is run for 60 iterations keeping both browsers alive for the
+ whole duration of the test and without forcing garbage collection. Intended
+ as a last-resort net to catch memory leaks not apparent on shorter tests.
-[memory_py]: https://chromium.googlesource.com/chromium/src/+/master/tools/perf/benchmarks/memory.py
+[memory_extra_py]: https://cs.chromium.org/chromium/src/tools/perf/contrib/memory_extras/memory_extras.py
