blob: 0df2b23b8e4c07ede1bab08b9f4512781b304dbc [file] [log] [blame] [view]
Randolfb0fe9c12023-03-06 14:03:481# WebDriver BiDi for Chromium [![chromium-bidi on npm](https://img.shields.io/npm/v/chromium-bidi)](https://www.npmjs.com/package/chromium-bidi)
2
Randolf55431462023-03-29 13:00:363## CI status
Randolfb0fe9c12023-03-06 14:03:484
Randolf55431462023-03-29 13:00:365![E2E Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/e2e.yml/badge.svg)
6![Unit Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/unit.yml/badge.svg)
Randolf Jungbcb3bc82023-06-26 16:30:147![WPT Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/wpt.yml/badge.svg)
Randolf55431462023-03-29 13:00:368
Randolf55431462023-03-29 13:00:369![Pre-commit](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/pre-commit.yml/badge.svg)
Randolfb0fe9c12023-03-06 14:03:4810
Randolf Jung3e526312023-08-08 06:20:3911[![codecov](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi/branch/main/graph/badge.svg?token=LJSXSC9L09)](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi)
12
Randolfb0fe9c12023-03-06 14:03:4813This is an implementation of the
14[WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) protocol with some
15extensions (**BiDi+**)
16for Chromium, implemented as a JavaScript layer translating between BiDi and CDP,
17running inside a Chrome tab.
18
19Current status can be checked
20at [WPT WebDriver BiDi status](https://wpt.fyi/results/webdriver/tests/bidi).
21
22## BiDi+
23
24**"BiDi+"** is an extension of the WebDriver BiDi protocol. In addition to [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) it has:
25
26### Command `cdp.sendCommand`
27
28```cddl
29CdpSendCommandCommand = {
30 method: "cdp.sendCommand",
31 params: ScriptEvaluateParameters,
32}
33
34CdpSendCommandParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3235 method: text,
36 params: any,
37 session?: text,
Randolfb0fe9c12023-03-06 14:03:4838}
39
40CdpSendCommandResult = {
41 result: any,
Alex Rudenko1552f2b2023-07-11 11:18:3242 session: text,
Randolfb0fe9c12023-03-06 14:03:4843}
44```
45
46The command runs the
47described [CDP command](https://chromedevtools.github.io/devtools-protocol)
Alex Rudenko1552f2b2023-07-11 11:18:3248and returns the result.
Randolfb0fe9c12023-03-06 14:03:4849
50### Command `cdp.getSession`
51
52```cddl
53CdpGetSessionCommand = {
54 method: "cdp.sendCommand",
55 params: ScriptEvaluateParameters,
56}
57
58CdpGetSessionParameters = {
59 context: BrowsingContext,
60}
61
62CdpGetSessionResult = {
Alex Rudenko1552f2b2023-07-11 11:18:3263 session: text,
Randolfb0fe9c12023-03-06 14:03:4864}
65```
66
67The command returns the default CDP session for the selected browsing context.
68
Alex Rudenko1552f2b2023-07-11 11:18:3269### Events `cdp`
Randolfb0fe9c12023-03-06 14:03:4870
71```cddl
72CdpEventReceivedEvent = {
Alex Rudenko1552f2b2023-07-11 11:18:3273 method: "cdp.<CDP Event Name>",
74 params: CdpEventReceivedParameters,
Randolfb0fe9c12023-03-06 14:03:4875}
76
77CdpEventReceivedParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3278 event: text,
79 params: any,
80 session: text,
Randolfb0fe9c12023-03-06 14:03:4881}
82```
83
84The event contains a CDP event.
85
86### Field `channel`
87
88Each command can be extended with a `channel`:
89
90```cddl
91Command = {
92 id: js-uint,
93 channel?: text,
94 CommandData,
95 Extensible,
96}
97```
98
99If provided and non-empty string, the very same `channel` is added to the response:
100
101```cddl
102CommandResponse = {
103 id: js-uint,
104 channel?: text,
105 result: ResultData,
106 Extensible,
107}
108
109ErrorResponse = {
110 id: js-uint / null,
111 channel?: text,
112 error: ErrorCode,
113 message: text,
114 ?stacktrace: text,
115 Extensible
116}
117```
118
119When client uses
120commands [`session.subscribe`](https://w3c.github.io/webdriver-bidi/#command-session-subscribe)
121and [`session.unsubscribe`](https://w3c.github.io/webdriver-bidi/#command-session-unsubscribe)
122with `channel`, the subscriptions are handled per channel, and the corresponding
123`channel` filed is added to the event message:
124
125```cddl
126Event = {
127 channel?: text,
128 EventData,
129 Extensible,
130}
131```
132
133## Dev Setup
134
135### `npm`
136
137This is a Node.js project, so install dependencies as usual:
138
139```sh
140npm install
141```
142
Randolf Jung3e526312023-08-08 06:20:39143### `cargo`
144
145<!-- TODO(jrandolf): Remove after binaries get published -->
146
Alex Rudenko662ce9c2023-08-29 08:05:30147We use [cddlconv](https://github.com/google/cddlconv) to generate our WebDriverBidi types before building.
Randolf Jung3e526312023-08-08 06:20:39148
1491. Install [Rust](https://rustup.rs/).
Alex Rudenko662ce9c2023-08-29 08:05:301502. Run `cargo install --git https://github.com/google/cddlconv.git cddlconv`
Randolf Jung3e526312023-08-08 06:20:39151
Randolfb0fe9c12023-03-06 14:03:48152### pre-commit.com integration
153
154Refer to the documentation at [.pre-commit-config.yaml](.pre-commit-config.yaml).
155
156### Starting the Server
157
158This will run the server on port `8080`:
159
160```sh
161npm run server
162```
163
164Use the `PORT=` environment variable or `--port=` argument to run it on another port:
165
166```sh
167PORT=8081 npm run server
168npm run server -- --port=8081
169```
170
171Use the `DEBUG` environment variable to see debug info:
172
173```sh
174DEBUG=* npm run server
175```
176
177Use the CLI argument `--headless=false` to run browser in headful mode:
178
179```sh
180npm run server -- --headless=false
181```
182
183Use the `CHANNEL=...` environment variable or `--channel=...` argument with one of
Randolf55431462023-03-29 13:00:36184the following values to run the specific Chrome channel: `stable`,
185`beta`, `canary`, `dev`.
Randolfb0fe9c12023-03-06 14:03:48186
187The requested Chrome version should be installed.
188
189```sh
Randolf55431462023-03-29 13:00:36190CHANNEL=dev npm run server
191npm run server -- --channel=dev
192```
193
Alex Rudenko1552f2b2023-07-11 11:18:32194Use the CLI argument `--verbose` to have CDP events printed to the console. Note: you have to enable debugging output `bidi:mapper:debug:*` as well.
Randolf55431462023-03-29 13:00:36195
196```sh
Alex Rudenko1552f2b2023-07-11 11:18:32197DEBUG=bidi:mapper:debug:* npm run server -- --verbose
Randolf55431462023-03-29 13:00:36198```
199
200or
201
202```sh
203DEBUG=* npm run server -- --verbose
Randolfb0fe9c12023-03-06 14:03:48204```
205
206### Starting on Linux and Mac
207
Randolf Jungbcb3bc82023-06-26 16:30:14208TODO: verify it works on Windows.
Randolfb0fe9c12023-03-06 14:03:48209
Randolf Jung3e526312023-08-08 06:20:39210You can also run the server by using `npm run server`. It will write
Randolfb0fe9c12023-03-06 14:03:48211output to the file `log.txt`:
212
213```sh
Randolf Jung3e526312023-08-08 06:20:39214npm run server -- --port=8081 --headless=false
Randolfb0fe9c12023-03-06 14:03:48215```
216
217## Running
218
219### Unit tests
220
221Running:
222
223```sh
Randolf Jungbcb3bc82023-06-26 16:30:14224npm run unit
Randolfb0fe9c12023-03-06 14:03:48225```
226
227### E2E tests
228
229The E2E tests are written using Python, in order to learn how to eventually do
230this in web-platform-tests.
231
232### Installation
233
234Python 3.6+ and some dependencies are required:
235
236```sh
237python3 -m pip install --user -r tests/requirements.txt
238```
239
240### Running
241
242The E2E tests require BiDi server running on the same host. By default, tests
243try to connect to the port `8080`. The server can be run from the project root:
244
245```sh
Randolf Jungbcb3bc82023-06-26 16:30:14246npm run e2e # alias to to e2e-headless
247npm run e2e-headful
248npm run e2e-headless
Randolfb0fe9c12023-03-06 14:03:48249```
250
251Use the `PORT` environment variable to connect to another port:
252
253```sh
254PORT=8081 npm run e2e
255```
256
257### Examples
258
259Refer to [examples/README.md](examples/README.md).
260
261## WPT (Web Platform Tests)
262
263WPT is added as
264a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). To get run
265WPT tests:
266
267### Check out and setup WPT
268
269#### 1. Check out WPT
270
271```sh
272git submodule update --init
273```
274
275#### 2. Go to the WPT folder
276
277```sh
278cd wpt
279```
280
281#### 3. Set up virtualenv
282
283Follow the [_System
284Setup_](https://web-platform-tests.org/running-tests/from-local-system.html#system-setup)
285instructions.
286
287#### 4. Setup `hosts` file
288
289Follow
290the [`hosts` File Setup](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup)
291instructions.
292
293##### 4.a On Linux, macOS or other UNIX-like system
294
295```sh
296./wpt make-hosts-file | sudo tee -a /etc/hosts
297```
298
299##### 4.b On **Windows**
300
301This must be run in a PowerShell session with Administrator privileges:
302
303```sh
304python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append
305```
306
307If you are behind a proxy, you also need to make sure the domains above are excluded
308from your proxy lookups.
309
Randolf Jungbcb3bc82023-06-26 16:30:14310#### 5. Set `BROWSER_BIN`
Randolfb0fe9c12023-03-06 14:03:48311
Randolf Jungbcb3bc82023-06-26 16:30:14312Set the `BROWSER_BIN` environment variable to a Chrome, Edge or Chromium binary to launch.
Randolfb0fe9c12023-03-06 14:03:48313For example, on macOS:
314
315```sh
316# Chrome
Randolf Jungbcb3bc82023-06-26 16:30:14317export BROWSER_BIN="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
318export BROWSER_BIN="/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev"
319export BROWSER_BIN="/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"
320export BROWSER_BIN="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
321export BROWSER_BIN="/Applications/Chromium.app/Contents/MacOS/Chromium"
Randolfb0fe9c12023-03-06 14:03:48322
323# Edge
Randolf Jungbcb3bc82023-06-26 16:30:14324export BROWSER_BIN="/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary"
325export BROWSER_BIN="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
Randolfb0fe9c12023-03-06 14:03:48326```
327
328### Run WPT tests
329
330#### 1. Make sure you have Chrome Dev installed
331
332https://www.google.com/chrome/dev/
333
Randolf55431462023-03-29 13:00:36334#### 2. Build Chromedriver BiDi
Randolfb0fe9c12023-03-06 14:03:48335
336Oneshot:
337
338```sh
339npm run build
340```
341
342Continuously:
343
344```sh
Randolf Jungbcb3bc82023-06-26 16:30:14345npm run build --watch
Randolfb0fe9c12023-03-06 14:03:48346```
347
348#### 3. Run
349
350```sh
Randolf Jungbcb3bc82023-06-26 16:30:14351npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48352```
353
354### Update WPT expectations if needed
355
Randolfb0fe9c12023-03-06 14:03:48356```sh
Randolf Jungbcb3bc82023-06-26 16:30:14357UPDATE_EXPECTATIONS=true npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48358```
359
360## How does it work?
361
362The architecture is described in the
363[WebDriver BiDi in Chrome Context implementation plan](https://docs.google.com/document/d/1VfQ9tv0wPSnb5TI-MOobjoQ5CXLnJJx9F_PxOMQc8kY)
364.
365
366There are 2 main modules:
367
3681. backend WS server in `src`. It runs webSocket server, and for each ws connection
369 runs an instance of browser with BiDi Mapper.
3702. front-end BiDi Mapper in `src/bidiMapper`. Gets BiDi commands from the backend,
371 and map them to CDP commands.
372
373### Contributing
374
375The BiDi commands are processed in the `src/bidiMapper/commandProcessor.ts`. To add a
376new command, add it to `_processCommand`, write and call processor for it.
377
378### Publish new `npm` release
379
Alex Rudenko1552f2b2023-07-11 11:18:32380#### Automatic release
Randolf Jungbcb3bc82023-06-26 16:30:14381
Alex Rudenko1552f2b2023-07-11 11:18:32382We use [release-please](https://github.com/googleapis/release-please) to automate releases. When a release should be done, check for the release PR in our [pull requests](https://github.com/GoogleChromeLabs/chromium-bidi/pulls) and merge it.
Randolf Jungbcb3bc82023-06-26 16:30:14383
384#### Manual release
385
3861. Dry-run
387
388 ```sh
389 npm publish --dry-run
390 ```
391
Randolf55431462023-03-29 13:00:363921. Open a PR bumping the chromium-bidi version number in `package.json` for review:
Randolfb0fe9c12023-03-06 14:03:48393
394 ```sh
Alex Rudenko1552f2b2023-07-11 11:18:32395 npm version patch -m 'chore: Release v%s' --no-git-tag-version
Randolfb0fe9c12023-03-06 14:03:48396 ```
397
398 Instead of `patch`, use `minor` or `major` [as needed](https://semver.org/).
399
Randolf55431462023-03-29 13:00:364001. After the PR is reviewed, [create a GitHub release](https://github.com/GoogleChromeLabs/chromium-bidi/releases/new) specifying the tag name matching the bumped version.
401 Our CI then automatically publishes the new release to npm based on the tag name.
Randolf Jung3e526312023-08-08 06:20:39402
403#### Roll into Chromium
404
Alex Rudenkob013f872023-09-29 05:56:03405This section assumes you already have a Chromium set-up locally,
406and knowledge on [how to submit changes to the repo](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/contributing.md).
407Otherwise submit an issue for a project maintainer.
Randolf Jung3e526312023-08-08 06:20:39408
Alex Rudenkob013f872023-09-29 05:56:034091. Create a new branch in chromium `src/`.
4102. Update the mapper version:
Randolf Jung3e526312023-08-08 06:20:39411
Alex Rudenkob013f872023-09-29 05:56:03412```shell
413third_party/bidimapper/pull.sh
414third_party/bidimapper/build.sh
Randolf Jung3e526312023-08-08 06:20:39415```
416
Alex Rudenkob013f872023-09-29 05:56:034173. Submit a CL with bug `chromedriver:4226`.
4184. [Regenerate WPT expectations metadata](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_platform_tests_wptrunner.md#updating-expectations):
419
4204.1. Trigger a build and test run:
421
422```shell
423third_party/blink/tools/blink_tool.py update-metadata --build="linux-blink-rel"
424```
425
4264.2. Once the test completes on the builder, rerun that command to update the metadata.
427Commit the changes (if any), and upload the new patch to the CL.
428
4295. Add appropriate reviewers or comment the CL link on the PR.