Update "LayoutTests" / "layout test" in docs/*.md.
This CL renames the following md files:
- layout_tests_linux -> web_tests_linux.md
- how_to_extend_layout_test_framework.md ->
how_to_extend_web_test_framework.md
- testing/layout_test_baseline_fallback.md ->
testing/web_test_baseline_fallback.md
- testing/layout_test_expectations.md -> testing/web_test_expectations.md
- testing/layout_tests.md -> testing/web_tests.md
- testing/layout_tests_in_content_shell.md ->
testing/web_tests_in_content_shell.md
- testing/layout_tests_tips.md -> testing/web_tests_tips.md
- testing/layout_tests_with_manual_fallback.md ->
testing/web_tests_with_manual_fallback.md
- testing/writing_layout_tests -> testing/writing_web_tests.md
No-Try: true
Change-Id: Ied34815c92e449888b20818540a5ed0a76107bd1
Bug: 843412
Reviewed-on: https://chromium-review.googlesource.com/c/1350559
Commit-Queue: Kent Tamura <[email protected]>
Reviewed-by: Quinten Yearsley <[email protected]>
Cr-Commit-Position: refs/heads/master@{#611024}diff --git a/docs/testing/web_tests_with_manual_fallback.md b/docs/testing/web_tests_with_manual_fallback.md
new file mode 100644
index 0000000..9acabd13d
--- /dev/null
+++ b/docs/testing/web_tests_with_manual_fallback.md
@@ -0,0 +1,119 @@
+# Web Tests with Manual Fallback
+
+Some Blink features cannot be automatically tested using the Web Platform. Prime
+examples are the APIs that require
+[user activation](https://html.spec.whatwg.org/multipage/interaction.html#triggered-by-user-activation)
+(also known as _a user gesture_), such as [Full Screen](https://developer.mozilla.org/en-US/docs/Web/API/Fullscreen_API).
+Automated tests for these Blink features must rely on special APIs, which are
+only exposed in testing environments, and are therefore not available in a
+normal browser session.
+
+A popular pattern used in these tests is to rely on the user to perform some
+manual steps in order to run the test case in a normal browser session. These
+tests are effectively
+[manual tests](https://web-platform-tests.org/writing-tests/manual.html), with
+additional JavaScript code that automatically performs the desired manual steps,
+when loaded in an environment that exposes the needed testing APIs.
+
+## Motivation
+
+Web tests that degrade to manual tests in the absence of testing APIs have
+the following benefits.
+
+* The manual test component can be debugged in a normal browser session, using
+ the rich [developer tools](https://developer.chrome.com/devtools). Tests
+ without a manual fallback can only be debugged in the test runner.
+* The manual tests can run in other browsers, making it easy to check whether
+ our behavior matches other browsers.
+* The web tests can form the basis for manual tests that are contributed to
+ [web-platform-tests](./web_platform_tests.md).
+
+Therefore, the desirability of adding a manual fallback to a test heavily
+depends on whether the feature under test is a Web Platform feature or a
+Blink-only feature, and on the developer's working style. The benefits above
+should be weighed against the added design effort needed to build a manual test,
+and the size and complexity introduced by the manual fallback.
+
+## Development Tips
+
+A natural workflow for writing a web test that gracefully degrades to a
+manual test is to first develop the manual test in a browser, and then add code
+that feature-checks for testing APIs, and uses them to automate the test's
+manual steps.
+
+Manual tests should minimize the chance of user error. This implies keeping the
+manual steps to a minimum, and having simple and clear instructions that
+describe all the configuration changes and user gestures that match the effect
+of the Blink-specific APIs used by the test.
+
+## Example
+
+Below is an example of a fairly minimal test that uses a Blink-Specific API
+(`window.eventSender`), and gracefully degrades to a manual test.
+
+```html
+<!doctype html>
+<meta charset="utf-8">
+<title>DOM: Event.isTrusted for UI events</title>
+<link rel="help" href="https://dom.spec.whatwg.org/#dom-event-istrusted">
+<link rel="help" href="https://dom.spec.whatwg.org/#constructing-events">
+<meta name="assert"
+ content="Event.isTrusted is true for events generated by user interaction">
+<script src="../../resources/testharness.js"></script>
+<script src="../../resources/testharnessreport.js"></script>
+
+<p>Please click on the button below.</p>
+<button>Click Me!</button>
+
+<script>
+'use strict';
+
+setup({ explicit_timeout: true });
+
+promise_test(() => {
+ const button = document.querySelector('button');
+ return new Promise((resolve, reject) => {
+ const button = document.querySelector('button');
+ button.addEventListener('click', (event) => {
+ resolve(event);
+ });
+
+ if (window.eventSender) {
+ eventSender.mouseMoveTo(button.offsetLeft, button.offsetTop);
+ eventSender.mouseDown();
+ eventSender.mouseUp();
+ }
+ }).then((clickEvent) => {
+ assert_true(clickEvent.isTrusted);
+ });
+
+}, 'Click generated by user interaction');
+
+</script>
+```
+
+The test exhibits the following desirable features:
+
+* It has a second specification URL (`<link rel="help">`), because the paragraph
+ that documents the tested feature (referenced by the primary URL) is not very
+ informative on its own.
+* It links to the
+ [WHATWG Living Standard](https://wiki.whatwg.org/wiki/FAQ#What_does_.22Living_Standard.22_mean.3F),
+ rather than to a frozen version of the specification.
+* It contains clear instructions for manually triggering the test conditions.
+ The test starts with a paragraph (`<p>`) that tells the tester exactly what to
+ do, and the `<button>` that needs to be clicked is clearly labeled.
+* It disables the timeout mechanism built into `testharness.js` by calling
+ `setup({ explicit_timeout: true });`
+* It checks for the presence of the Blink-specific testing APIs
+ (`window.eventSender`) before invoking them. The test does not automatically
+ fail when the APIs are not present.
+* It uses [Promises](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise)
+ to separate the test setup from the assertions. This is particularly helpful
+ for manual tests that depend on a sequence of events to occur, as Promises
+ offer a composable way to express waiting for asynchronous events that avoids
+ [callback hell](http://stackabuse.com/avoiding-callback-hell-in-node-js/).
+
+Notice that the test is pretty heavy compared to a minimal JavaScript test that
+does not rely on testing APIs. Only use testing APIs when the desired testing
+conditions cannot be set up using Web Platform APIs.
