blob: 49b1edee93ce70211d7f7ea6f38b3070e185ec2f [file] [log] [blame] [view]
Randolfb0fe9c12023-03-06 14:03:481# WebDriver BiDi for Chromium [![chromium-bidi on npm](https://img.shields.io/npm/v/chromium-bidi)](https://www.npmjs.com/package/chromium-bidi)
2
Randolf55431462023-03-29 13:00:363## CI status
Randolfb0fe9c12023-03-06 14:03:484
Randolf55431462023-03-29 13:00:365![E2E Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/e2e.yml/badge.svg)
6![Unit Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/unit.yml/badge.svg)
Randolf Jungbcb3bc82023-06-26 16:30:147![WPT Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/wpt.yml/badge.svg)
Randolf55431462023-03-29 13:00:368
Randolf55431462023-03-29 13:00:369![Pre-commit](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/pre-commit.yml/badge.svg)
Randolfb0fe9c12023-03-06 14:03:4810
Randolf Jung3e526312023-08-08 06:20:3911[![codecov](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi/branch/main/graph/badge.svg?token=LJSXSC9L09)](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi)
12
Randolfb0fe9c12023-03-06 14:03:4813This is an implementation of the
14[WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) protocol with some
15extensions (**BiDi+**)
16for Chromium, implemented as a JavaScript layer translating between BiDi and CDP,
17running inside a Chrome tab.
18
19Current status can be checked
20at [WPT WebDriver BiDi status](https://wpt.fyi/results/webdriver/tests/bidi).
21
22## BiDi+
23
24**"BiDi+"** is an extension of the WebDriver BiDi protocol. In addition to [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) it has:
25
26### Command `cdp.sendCommand`
27
28```cddl
29CdpSendCommandCommand = {
30 method: "cdp.sendCommand",
31 params: ScriptEvaluateParameters,
32}
33
34CdpSendCommandParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3235 method: text,
36 params: any,
37 session?: text,
Randolfb0fe9c12023-03-06 14:03:4838}
39
40CdpSendCommandResult = {
41 result: any,
Alex Rudenko1552f2b2023-07-11 11:18:3242 session: text,
Randolfb0fe9c12023-03-06 14:03:4843}
44```
45
46The command runs the
47described [CDP command](https://chromedevtools.github.io/devtools-protocol)
Alex Rudenko1552f2b2023-07-11 11:18:3248and returns the result.
Randolfb0fe9c12023-03-06 14:03:4849
50### Command `cdp.getSession`
51
52```cddl
53CdpGetSessionCommand = {
54 method: "cdp.sendCommand",
55 params: ScriptEvaluateParameters,
56}
57
58CdpGetSessionParameters = {
59 context: BrowsingContext,
60}
61
62CdpGetSessionResult = {
Alex Rudenko1552f2b2023-07-11 11:18:3263 session: text,
Randolfb0fe9c12023-03-06 14:03:4864}
65```
66
67The command returns the default CDP session for the selected browsing context.
68
Alex Rudenko1552f2b2023-07-11 11:18:3269### Events `cdp`
Randolfb0fe9c12023-03-06 14:03:4870
71```cddl
72CdpEventReceivedEvent = {
Alex Rudenko1552f2b2023-07-11 11:18:3273 method: "cdp.<CDP Event Name>",
74 params: CdpEventReceivedParameters,
Randolfb0fe9c12023-03-06 14:03:4875}
76
77CdpEventReceivedParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3278 event: text,
79 params: any,
80 session: text,
Randolfb0fe9c12023-03-06 14:03:4881}
82```
83
84The event contains a CDP event.
85
86### Field `channel`
87
88Each command can be extended with a `channel`:
89
90```cddl
91Command = {
92 id: js-uint,
93 channel?: text,
94 CommandData,
95 Extensible,
96}
97```
98
99If provided and non-empty string, the very same `channel` is added to the response:
100
101```cddl
102CommandResponse = {
103 id: js-uint,
104 channel?: text,
105 result: ResultData,
106 Extensible,
107}
108
109ErrorResponse = {
110 id: js-uint / null,
111 channel?: text,
112 error: ErrorCode,
113 message: text,
114 ?stacktrace: text,
115 Extensible
116}
117```
118
119When client uses
120commands [`session.subscribe`](https://w3c.github.io/webdriver-bidi/#command-session-subscribe)
121and [`session.unsubscribe`](https://w3c.github.io/webdriver-bidi/#command-session-unsubscribe)
122with `channel`, the subscriptions are handled per channel, and the corresponding
123`channel` filed is added to the event message:
124
125```cddl
126Event = {
127 channel?: text,
128 EventData,
129 Extensible,
130}
131```
132
133## Dev Setup
134
135### `npm`
136
137This is a Node.js project, so install dependencies as usual:
138
139```sh
140npm install
141```
142
Randolf Jung3e526312023-08-08 06:20:39143### `cargo`
144
145<!-- TODO(jrandolf): Remove after binaries get published -->
146
Alex Rudenko662ce9c2023-08-29 08:05:30147We use [cddlconv](https://github.com/google/cddlconv) to generate our WebDriverBidi types before building.
Randolf Jung3e526312023-08-08 06:20:39148
1491. Install [Rust](https://rustup.rs/).
Alex Rudenko662ce9c2023-08-29 08:05:301502. Run `cargo install --git https://github.com/google/cddlconv.git cddlconv`
Randolf Jung3e526312023-08-08 06:20:39151
Randolfb0fe9c12023-03-06 14:03:48152### pre-commit.com integration
153
154Refer to the documentation at [.pre-commit-config.yaml](.pre-commit-config.yaml).
155
Alex Rudenkof7ea7ab2023-10-24 09:40:51156```sh
157pre-commit install --hook-type pre-push
158```
159
160### Starting WebDriver BiDi Server
Randolfb0fe9c12023-03-06 14:03:48161
162This will run the server on port `8080`:
163
164```sh
165npm run server
166```
167
168Use the `PORT=` environment variable or `--port=` argument to run it on another port:
169
170```sh
171PORT=8081 npm run server
172npm run server -- --port=8081
173```
174
175Use the `DEBUG` environment variable to see debug info:
176
177```sh
178DEBUG=* npm run server
179```
180
Alex Rudenko5fe99f82023-10-06 12:01:27181Use the `DEBUG_DEPTH` (default: `10`) environment variable to see debug deeply nested objects:
182
183```sh
184DEBUG_DEPTH=100 DEBUG=* npm run server
185```
186
Randolfb0fe9c12023-03-06 14:03:48187Use the CLI argument `--headless=false` to run browser in headful mode:
188
189```sh
190npm run server -- --headless=false
191```
192
193Use the `CHANNEL=...` environment variable or `--channel=...` argument with one of
Randolf55431462023-03-29 13:00:36194the following values to run the specific Chrome channel: `stable`,
195`beta`, `canary`, `dev`.
Randolfb0fe9c12023-03-06 14:03:48196
197The requested Chrome version should be installed.
198
199```sh
Randolf55431462023-03-29 13:00:36200CHANNEL=dev npm run server
201npm run server -- --channel=dev
202```
203
Alex Rudenko1552f2b2023-07-11 11:18:32204Use the CLI argument `--verbose` to have CDP events printed to the console. Note: you have to enable debugging output `bidi:mapper:debug:*` as well.
Randolf55431462023-03-29 13:00:36205
206```sh
Alex Rudenko1552f2b2023-07-11 11:18:32207DEBUG=bidi:mapper:debug:* npm run server -- --verbose
Randolf55431462023-03-29 13:00:36208```
209
210or
211
212```sh
213DEBUG=* npm run server -- --verbose
Randolfb0fe9c12023-03-06 14:03:48214```
215
216### Starting on Linux and Mac
217
Randolf Jungbcb3bc82023-06-26 16:30:14218TODO: verify it works on Windows.
Randolfb0fe9c12023-03-06 14:03:48219
Randolf Jung3e526312023-08-08 06:20:39220You can also run the server by using `npm run server`. It will write
Randolfb0fe9c12023-03-06 14:03:48221output to the file `log.txt`:
222
223```sh
Randolf Jung3e526312023-08-08 06:20:39224npm run server -- --port=8081 --headless=false
Randolfb0fe9c12023-03-06 14:03:48225```
226
Alex Rudenko5fe99f82023-10-06 12:01:27227### Running with in other project
228
229Sometimes it good to verify that a change will not affect thing downstream for other packages.
230There is a useful `puppeteer` label you can add to any PR to run Puppeteer test with your changes.
231It will bundle `chromium-bidi` and install it in Puppeteer project then run that package test.
232
Randolfb0fe9c12023-03-06 14:03:48233## Running
234
235### Unit tests
236
237Running:
238
239```sh
Randolf Jungbcb3bc82023-06-26 16:30:14240npm run unit
Randolfb0fe9c12023-03-06 14:03:48241```
242
243### E2E tests
244
245The E2E tests are written using Python, in order to learn how to eventually do
246this in web-platform-tests.
247
Alex Rudenko5fe99f82023-10-06 12:01:27248#### Installation
Randolfb0fe9c12023-03-06 14:03:48249
Alex Rudenkoaf11b7c2024-02-06 10:44:42250Python 3.10+ and some dependencies are required:
Randolfb0fe9c12023-03-06 14:03:48251
252```sh
Alex Rudenkoaf11b7c2024-02-06 10:44:42253python -m pip install --user pipenv
254pipenv install
Randolfb0fe9c12023-03-06 14:03:48255```
256
Alex Rudenko5fe99f82023-10-06 12:01:27257#### Running
Randolfb0fe9c12023-03-06 14:03:48258
259The E2E tests require BiDi server running on the same host. By default, tests
260try to connect to the port `8080`. The server can be run from the project root:
261
262```sh
Alex Rudenkod04dd452024-02-27 08:25:35263npm run e2e # alias to to e2e:headless
264npm run e2e:headful
265npm run e2e:headless
Randolfb0fe9c12023-03-06 14:03:48266```
267
268Use the `PORT` environment variable to connect to another port:
269
270```sh
271PORT=8081 npm run e2e
272```
273
Alex Rudenkoaf11b7c2024-02-06 10:44:42274#### Updating snapshots
275
276```sh
277npm run e2e -- --snapshot-update
278```
279
280See https://github.com/tophat/syrupy for more information.
281
Alex Rudenkof7ea7ab2023-10-24 09:40:51282### Local http server
283
284E2E tests use local http
285server [`pytest-httpserver`](https://pytest-httpserver.readthedocs.io/), which is run
286automatically with the tests. However,
287sometimes it is useful to run the http server outside the test
288case, for example for manual debugging. This can be done by running:
289
290```sh
Alex Rudenkoaf11b7c2024-02-06 10:44:42291pipenv run local_http_server
Alex Rudenkof7ea7ab2023-10-24 09:40:51292```
293
Alex Rudenkoaf11b7c2024-02-06 10:44:42294...or directly:
Alex Rudenkof7ea7ab2023-10-24 09:40:51295
296```sh
Alex Rudenkoaf11b7c2024-02-06 10:44:42297python tests/tools/local_http_server.py
Alex Rudenkof7ea7ab2023-10-24 09:40:51298```
299
300### Examples
Randolfb0fe9c12023-03-06 14:03:48301
302Refer to [examples/README.md](examples/README.md).
303
304## WPT (Web Platform Tests)
305
306WPT is added as
307a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). To get run
308WPT tests:
309
310### Check out and setup WPT
311
312#### 1. Check out WPT
313
314```sh
315git submodule update --init
316```
317
318#### 2. Go to the WPT folder
319
320```sh
321cd wpt
322```
323
324#### 3. Set up virtualenv
325
326Follow the [_System
327Setup_](https://web-platform-tests.org/running-tests/from-local-system.html#system-setup)
328instructions.
329
330#### 4. Setup `hosts` file
331
332Follow
333the [`hosts` File Setup](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup)
334instructions.
335
336##### 4.a On Linux, macOS or other UNIX-like system
337
338```sh
339./wpt make-hosts-file | sudo tee -a /etc/hosts
340```
341
342##### 4.b On **Windows**
343
344This must be run in a PowerShell session with Administrator privileges:
345
346```sh
347python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append
348```
349
350If you are behind a proxy, you also need to make sure the domains above are excluded
351from your proxy lookups.
352
Randolf Jungbcb3bc82023-06-26 16:30:14353#### 5. Set `BROWSER_BIN`
Randolfb0fe9c12023-03-06 14:03:48354
Randolf Jungbcb3bc82023-06-26 16:30:14355Set the `BROWSER_BIN` environment variable to a Chrome, Edge or Chromium binary to launch.
Randolfb0fe9c12023-03-06 14:03:48356For example, on macOS:
357
358```sh
359# Chrome
Randolf Jungbcb3bc82023-06-26 16:30:14360export BROWSER_BIN="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
361export BROWSER_BIN="/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev"
362export BROWSER_BIN="/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"
363export BROWSER_BIN="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
364export BROWSER_BIN="/Applications/Chromium.app/Contents/MacOS/Chromium"
Randolfb0fe9c12023-03-06 14:03:48365
366# Edge
Randolf Jungbcb3bc82023-06-26 16:30:14367export BROWSER_BIN="/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary"
368export BROWSER_BIN="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
Randolfb0fe9c12023-03-06 14:03:48369```
370
371### Run WPT tests
372
373#### 1. Make sure you have Chrome Dev installed
374
375https://www.google.com/chrome/dev/
376
Randolf55431462023-03-29 13:00:36377#### 2. Build Chromedriver BiDi
Randolfb0fe9c12023-03-06 14:03:48378
379Oneshot:
380
381```sh
382npm run build
383```
384
385Continuously:
386
387```sh
Randolf Jungbcb3bc82023-06-26 16:30:14388npm run build --watch
Randolfb0fe9c12023-03-06 14:03:48389```
390
391#### 3. Run
392
393```sh
Randolf Jungbcb3bc82023-06-26 16:30:14394npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48395```
396
397### Update WPT expectations if needed
398
Randolfb0fe9c12023-03-06 14:03:48399```sh
Randolf Jungbcb3bc82023-06-26 16:30:14400UPDATE_EXPECTATIONS=true npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48401```
402
403## How does it work?
404
405The architecture is described in the
406[WebDriver BiDi in Chrome Context implementation plan](https://docs.google.com/document/d/1VfQ9tv0wPSnb5TI-MOobjoQ5CXLnJJx9F_PxOMQc8kY)
407.
408
409There are 2 main modules:
410
4111. backend WS server in `src`. It runs webSocket server, and for each ws connection
412 runs an instance of browser with BiDi Mapper.
4132. front-end BiDi Mapper in `src/bidiMapper`. Gets BiDi commands from the backend,
414 and map them to CDP commands.
415
Alex Rudenko5fe99f82023-10-06 12:01:27416## Contributing
Randolfb0fe9c12023-03-06 14:03:48417
418The BiDi commands are processed in the `src/bidiMapper/commandProcessor.ts`. To add a
419new command, add it to `_processCommand`, write and call processor for it.
420
421### Publish new `npm` release
422
Alex Rudenko1552f2b2023-07-11 11:18:32423#### Automatic release
Randolf Jungbcb3bc82023-06-26 16:30:14424
Alex Rudenko1552f2b2023-07-11 11:18:32425We use [release-please](https://github.com/googleapis/release-please) to automate releases. When a release should be done, check for the release PR in our [pull requests](https://github.com/GoogleChromeLabs/chromium-bidi/pulls) and merge it.
Randolf Jungbcb3bc82023-06-26 16:30:14426
427#### Manual release
428
4291. Dry-run
430
431 ```sh
432 npm publish --dry-run
433 ```
434
Randolf55431462023-03-29 13:00:364351. Open a PR bumping the chromium-bidi version number in `package.json` for review:
Randolfb0fe9c12023-03-06 14:03:48436
437 ```sh
Alex Rudenko1552f2b2023-07-11 11:18:32438 npm version patch -m 'chore: Release v%s' --no-git-tag-version
Randolfb0fe9c12023-03-06 14:03:48439 ```
440
441 Instead of `patch`, use `minor` or `major` [as needed](https://semver.org/).
442
Randolf55431462023-03-29 13:00:364431. After the PR is reviewed, [create a GitHub release](https://github.com/GoogleChromeLabs/chromium-bidi/releases/new) specifying the tag name matching the bumped version.
444 Our CI then automatically publishes the new release to npm based on the tag name.
Randolf Jung3e526312023-08-08 06:20:39445
446#### Roll into Chromium
447
Alex Rudenkob013f872023-09-29 05:56:03448This section assumes you already have a Chromium set-up locally,
449and knowledge on [how to submit changes to the repo](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/contributing.md).
450Otherwise submit an issue for a project maintainer.
Randolf Jung3e526312023-08-08 06:20:39451
Alex Rudenkob013f872023-09-29 05:56:034521. Create a new branch in chromium `src/`.
4532. Update the mapper version:
Randolf Jung3e526312023-08-08 06:20:39454
Alex Rudenkob013f872023-09-29 05:56:03455```shell
456third_party/bidimapper/pull.sh
457third_party/bidimapper/build.sh
Randolf Jung3e526312023-08-08 06:20:39458```
459
Alex Rudenkob013f872023-09-29 05:56:034603. Submit a CL with bug `chromedriver:4226`.
Alex Rudenkob013f872023-09-29 05:56:03461
Alex Rudenkoaf11b7c2024-02-06 10:44:424624. [Regenerate WPT expectations or baselines](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/run_web_platform_tests.md#test-expectations-and-baselines):
Alex Rudenkob013f872023-09-29 05:56:03463
Alex Rudenkoaf11b7c2024-02-06 10:44:42464 4.1. Trigger a build and test run:
Alex Rudenkob013f872023-09-29 05:56:03465
Alex Rudenkoaf11b7c2024-02-06 10:44:42466 ```shell
467 third_party/blink/tools/blink_tool.py rebaseline-cl --build="linux-blink-rel" --verbose
468 ```
469
470 4.2. Once the test completes on the builder, rerun that command to update the
471 baselines. Update test expectations if there are any crashes or timeouts.
472 Commit the changes (if any), and upload the new patch to the CL.
Alex Rudenkob013f872023-09-29 05:56:03473
4745. Add appropriate reviewers or comment the CL link on the PR.