blob: 865ae8c665ec07771b43f4d9140c97c49de9bca8 [file] [log] [blame] [view]
Randolfb0fe9c12023-03-06 14:03:481# WebDriver BiDi for Chromium [![chromium-bidi on npm](https://img.shields.io/npm/v/chromium-bidi)](https://www.npmjs.com/package/chromium-bidi)
2
Randolf55431462023-03-29 13:00:363## CI status
Randolfb0fe9c12023-03-06 14:03:484
Randolf55431462023-03-29 13:00:365![E2E Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/e2e.yml/badge.svg)
6![Unit Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/unit.yml/badge.svg)
Randolf Jungbcb3bc82023-06-26 16:30:147![WPT Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/wpt.yml/badge.svg)
Randolf55431462023-03-29 13:00:368
Randolf55431462023-03-29 13:00:369![Pre-commit](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/pre-commit.yml/badge.svg)
Randolfb0fe9c12023-03-06 14:03:4810
Randolf Jung3e526312023-08-08 06:20:3911[![codecov](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi/branch/main/graph/badge.svg?token=LJSXSC9L09)](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi)
12
Randolfb0fe9c12023-03-06 14:03:4813This is an implementation of the
14[WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) protocol with some
15extensions (**BiDi+**)
16for Chromium, implemented as a JavaScript layer translating between BiDi and CDP,
17running inside a Chrome tab.
18
19Current status can be checked
20at [WPT WebDriver BiDi status](https://wpt.fyi/results/webdriver/tests/bidi).
21
22## BiDi+
23
24**"BiDi+"** is an extension of the WebDriver BiDi protocol. In addition to [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) it has:
25
26### Command `cdp.sendCommand`
27
28```cddl
29CdpSendCommandCommand = {
30 method: "cdp.sendCommand",
31 params: ScriptEvaluateParameters,
32}
33
34CdpSendCommandParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3235 method: text,
36 params: any,
37 session?: text,
Randolfb0fe9c12023-03-06 14:03:4838}
39
40CdpSendCommandResult = {
41 result: any,
Alex Rudenko1552f2b2023-07-11 11:18:3242 session: text,
Randolfb0fe9c12023-03-06 14:03:4843}
44```
45
46The command runs the
47described [CDP command](https://chromedevtools.github.io/devtools-protocol)
Alex Rudenko1552f2b2023-07-11 11:18:3248and returns the result.
Randolfb0fe9c12023-03-06 14:03:4849
50### Command `cdp.getSession`
51
52```cddl
53CdpGetSessionCommand = {
54 method: "cdp.sendCommand",
55 params: ScriptEvaluateParameters,
56}
57
58CdpGetSessionParameters = {
59 context: BrowsingContext,
60}
61
62CdpGetSessionResult = {
Alex Rudenko1552f2b2023-07-11 11:18:3263 session: text,
Randolfb0fe9c12023-03-06 14:03:4864}
65```
66
67The command returns the default CDP session for the selected browsing context.
68
Alex Rudenko1552f2b2023-07-11 11:18:3269### Events `cdp`
Randolfb0fe9c12023-03-06 14:03:4870
71```cddl
72CdpEventReceivedEvent = {
Alex Rudenko1552f2b2023-07-11 11:18:3273 method: "cdp.<CDP Event Name>",
74 params: CdpEventReceivedParameters,
Randolfb0fe9c12023-03-06 14:03:4875}
76
77CdpEventReceivedParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3278 event: text,
79 params: any,
80 session: text,
Randolfb0fe9c12023-03-06 14:03:4881}
82```
83
84The event contains a CDP event.
85
86### Field `channel`
87
88Each command can be extended with a `channel`:
89
90```cddl
91Command = {
92 id: js-uint,
93 channel?: text,
94 CommandData,
95 Extensible,
96}
97```
98
99If provided and non-empty string, the very same `channel` is added to the response:
100
101```cddl
102CommandResponse = {
103 id: js-uint,
104 channel?: text,
105 result: ResultData,
106 Extensible,
107}
108
109ErrorResponse = {
110 id: js-uint / null,
111 channel?: text,
112 error: ErrorCode,
113 message: text,
114 ?stacktrace: text,
115 Extensible
116}
117```
118
119When client uses
120commands [`session.subscribe`](https://w3c.github.io/webdriver-bidi/#command-session-subscribe)
121and [`session.unsubscribe`](https://w3c.github.io/webdriver-bidi/#command-session-unsubscribe)
122with `channel`, the subscriptions are handled per channel, and the corresponding
123`channel` filed is added to the event message:
124
125```cddl
126Event = {
127 channel?: text,
128 EventData,
129 Extensible,
130}
131```
132
133## Dev Setup
134
135### `npm`
136
137This is a Node.js project, so install dependencies as usual:
138
139```sh
140npm install
141```
142
Randolf Jung3e526312023-08-08 06:20:39143### `cargo`
144
145<!-- TODO(jrandolf): Remove after binaries get published -->
146
Alex Rudenko662ce9c2023-08-29 08:05:30147We use [cddlconv](https://github.com/google/cddlconv) to generate our WebDriverBidi types before building.
Randolf Jung3e526312023-08-08 06:20:39148
1491. Install [Rust](https://rustup.rs/).
Alex Rudenko662ce9c2023-08-29 08:05:301502. Run `cargo install --git https://github.com/google/cddlconv.git cddlconv`
Randolf Jung3e526312023-08-08 06:20:39151
Randolfb0fe9c12023-03-06 14:03:48152### pre-commit.com integration
153
154Refer to the documentation at [.pre-commit-config.yaml](.pre-commit-config.yaml).
155
156### Starting the Server
157
158This will run the server on port `8080`:
159
160```sh
161npm run server
162```
163
164Use the `PORT=` environment variable or `--port=` argument to run it on another port:
165
166```sh
167PORT=8081 npm run server
168npm run server -- --port=8081
169```
170
171Use the `DEBUG` environment variable to see debug info:
172
173```sh
174DEBUG=* npm run server
175```
176
Alex Rudenko5fe99f82023-10-06 12:01:27177Use the `DEBUG_DEPTH` (default: `10`) environment variable to see debug deeply nested objects:
178
179```sh
180DEBUG_DEPTH=100 DEBUG=* npm run server
181```
182
Randolfb0fe9c12023-03-06 14:03:48183Use the CLI argument `--headless=false` to run browser in headful mode:
184
185```sh
186npm run server -- --headless=false
187```
188
189Use the `CHANNEL=...` environment variable or `--channel=...` argument with one of
Randolf55431462023-03-29 13:00:36190the following values to run the specific Chrome channel: `stable`,
191`beta`, `canary`, `dev`.
Randolfb0fe9c12023-03-06 14:03:48192
193The requested Chrome version should be installed.
194
195```sh
Randolf55431462023-03-29 13:00:36196CHANNEL=dev npm run server
197npm run server -- --channel=dev
198```
199
Alex Rudenko1552f2b2023-07-11 11:18:32200Use the CLI argument `--verbose` to have CDP events printed to the console. Note: you have to enable debugging output `bidi:mapper:debug:*` as well.
Randolf55431462023-03-29 13:00:36201
202```sh
Alex Rudenko1552f2b2023-07-11 11:18:32203DEBUG=bidi:mapper:debug:* npm run server -- --verbose
Randolf55431462023-03-29 13:00:36204```
205
206or
207
208```sh
209DEBUG=* npm run server -- --verbose
Randolfb0fe9c12023-03-06 14:03:48210```
211
212### Starting on Linux and Mac
213
Randolf Jungbcb3bc82023-06-26 16:30:14214TODO: verify it works on Windows.
Randolfb0fe9c12023-03-06 14:03:48215
Randolf Jung3e526312023-08-08 06:20:39216You can also run the server by using `npm run server`. It will write
Randolfb0fe9c12023-03-06 14:03:48217output to the file `log.txt`:
218
219```sh
Randolf Jung3e526312023-08-08 06:20:39220npm run server -- --port=8081 --headless=false
Randolfb0fe9c12023-03-06 14:03:48221```
222
Alex Rudenko5fe99f82023-10-06 12:01:27223### Running with in other project
224
225Sometimes it good to verify that a change will not affect thing downstream for other packages.
226There is a useful `puppeteer` label you can add to any PR to run Puppeteer test with your changes.
227It will bundle `chromium-bidi` and install it in Puppeteer project then run that package test.
228
Randolfb0fe9c12023-03-06 14:03:48229## Running
230
231### Unit tests
232
233Running:
234
235```sh
Randolf Jungbcb3bc82023-06-26 16:30:14236npm run unit
Randolfb0fe9c12023-03-06 14:03:48237```
238
239### E2E tests
240
241The E2E tests are written using Python, in order to learn how to eventually do
242this in web-platform-tests.
243
Alex Rudenko5fe99f82023-10-06 12:01:27244#### Installation
Randolfb0fe9c12023-03-06 14:03:48245
246Python 3.6+ and some dependencies are required:
247
248```sh
249python3 -m pip install --user -r tests/requirements.txt
250```
251
Alex Rudenko5fe99f82023-10-06 12:01:27252#### Running
Randolfb0fe9c12023-03-06 14:03:48253
254The E2E tests require BiDi server running on the same host. By default, tests
255try to connect to the port `8080`. The server can be run from the project root:
256
257```sh
Randolf Jungbcb3bc82023-06-26 16:30:14258npm run e2e # alias to to e2e-headless
259npm run e2e-headful
260npm run e2e-headless
Randolfb0fe9c12023-03-06 14:03:48261```
262
263Use the `PORT` environment variable to connect to another port:
264
265```sh
266PORT=8081 npm run e2e
267```
268
Alex Rudenko5fe99f82023-10-06 12:01:27269#### Examples
Randolfb0fe9c12023-03-06 14:03:48270
271Refer to [examples/README.md](examples/README.md).
272
273## WPT (Web Platform Tests)
274
275WPT is added as
276a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). To get run
277WPT tests:
278
279### Check out and setup WPT
280
281#### 1. Check out WPT
282
283```sh
284git submodule update --init
285```
286
287#### 2. Go to the WPT folder
288
289```sh
290cd wpt
291```
292
293#### 3. Set up virtualenv
294
295Follow the [_System
296Setup_](https://web-platform-tests.org/running-tests/from-local-system.html#system-setup)
297instructions.
298
299#### 4. Setup `hosts` file
300
301Follow
302the [`hosts` File Setup](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup)
303instructions.
304
305##### 4.a On Linux, macOS or other UNIX-like system
306
307```sh
308./wpt make-hosts-file | sudo tee -a /etc/hosts
309```
310
311##### 4.b On **Windows**
312
313This must be run in a PowerShell session with Administrator privileges:
314
315```sh
316python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append
317```
318
319If you are behind a proxy, you also need to make sure the domains above are excluded
320from your proxy lookups.
321
Randolf Jungbcb3bc82023-06-26 16:30:14322#### 5. Set `BROWSER_BIN`
Randolfb0fe9c12023-03-06 14:03:48323
Randolf Jungbcb3bc82023-06-26 16:30:14324Set the `BROWSER_BIN` environment variable to a Chrome, Edge or Chromium binary to launch.
Randolfb0fe9c12023-03-06 14:03:48325For example, on macOS:
326
327```sh
328# Chrome
Randolf Jungbcb3bc82023-06-26 16:30:14329export BROWSER_BIN="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
330export BROWSER_BIN="/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev"
331export BROWSER_BIN="/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"
332export BROWSER_BIN="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
333export BROWSER_BIN="/Applications/Chromium.app/Contents/MacOS/Chromium"
Randolfb0fe9c12023-03-06 14:03:48334
335# Edge
Randolf Jungbcb3bc82023-06-26 16:30:14336export BROWSER_BIN="/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary"
337export BROWSER_BIN="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
Randolfb0fe9c12023-03-06 14:03:48338```
339
340### Run WPT tests
341
342#### 1. Make sure you have Chrome Dev installed
343
344https://www.google.com/chrome/dev/
345
Randolf55431462023-03-29 13:00:36346#### 2. Build Chromedriver BiDi
Randolfb0fe9c12023-03-06 14:03:48347
348Oneshot:
349
350```sh
351npm run build
352```
353
354Continuously:
355
356```sh
Randolf Jungbcb3bc82023-06-26 16:30:14357npm run build --watch
Randolfb0fe9c12023-03-06 14:03:48358```
359
360#### 3. Run
361
362```sh
Randolf Jungbcb3bc82023-06-26 16:30:14363npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48364```
365
366### Update WPT expectations if needed
367
Randolfb0fe9c12023-03-06 14:03:48368```sh
Randolf Jungbcb3bc82023-06-26 16:30:14369UPDATE_EXPECTATIONS=true npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48370```
371
372## How does it work?
373
374The architecture is described in the
375[WebDriver BiDi in Chrome Context implementation plan](https://docs.google.com/document/d/1VfQ9tv0wPSnb5TI-MOobjoQ5CXLnJJx9F_PxOMQc8kY)
376.
377
378There are 2 main modules:
379
3801. backend WS server in `src`. It runs webSocket server, and for each ws connection
381 runs an instance of browser with BiDi Mapper.
3822. front-end BiDi Mapper in `src/bidiMapper`. Gets BiDi commands from the backend,
383 and map them to CDP commands.
384
Alex Rudenko5fe99f82023-10-06 12:01:27385## Contributing
Randolfb0fe9c12023-03-06 14:03:48386
387The BiDi commands are processed in the `src/bidiMapper/commandProcessor.ts`. To add a
388new command, add it to `_processCommand`, write and call processor for it.
389
390### Publish new `npm` release
391
Alex Rudenko1552f2b2023-07-11 11:18:32392#### Automatic release
Randolf Jungbcb3bc82023-06-26 16:30:14393
Alex Rudenko1552f2b2023-07-11 11:18:32394We use [release-please](https://github.com/googleapis/release-please) to automate releases. When a release should be done, check for the release PR in our [pull requests](https://github.com/GoogleChromeLabs/chromium-bidi/pulls) and merge it.
Randolf Jungbcb3bc82023-06-26 16:30:14395
396#### Manual release
397
3981. Dry-run
399
400 ```sh
401 npm publish --dry-run
402 ```
403
Randolf55431462023-03-29 13:00:364041. Open a PR bumping the chromium-bidi version number in `package.json` for review:
Randolfb0fe9c12023-03-06 14:03:48405
406 ```sh
Alex Rudenko1552f2b2023-07-11 11:18:32407 npm version patch -m 'chore: Release v%s' --no-git-tag-version
Randolfb0fe9c12023-03-06 14:03:48408 ```
409
410 Instead of `patch`, use `minor` or `major` [as needed](https://semver.org/).
411
Randolf55431462023-03-29 13:00:364121. After the PR is reviewed, [create a GitHub release](https://github.com/GoogleChromeLabs/chromium-bidi/releases/new) specifying the tag name matching the bumped version.
413 Our CI then automatically publishes the new release to npm based on the tag name.
Randolf Jung3e526312023-08-08 06:20:39414
415#### Roll into Chromium
416
Alex Rudenkob013f872023-09-29 05:56:03417This section assumes you already have a Chromium set-up locally,
418and knowledge on [how to submit changes to the repo](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/contributing.md).
419Otherwise submit an issue for a project maintainer.
Randolf Jung3e526312023-08-08 06:20:39420
Alex Rudenkob013f872023-09-29 05:56:034211. Create a new branch in chromium `src/`.
4222. Update the mapper version:
Randolf Jung3e526312023-08-08 06:20:39423
Alex Rudenkob013f872023-09-29 05:56:03424```shell
425third_party/bidimapper/pull.sh
426third_party/bidimapper/build.sh
Randolf Jung3e526312023-08-08 06:20:39427```
428
Alex Rudenkob013f872023-09-29 05:56:034293. Submit a CL with bug `chromedriver:4226`.
4304. [Regenerate WPT expectations metadata](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_platform_tests_wptrunner.md#updating-expectations):
431
4324.1. Trigger a build and test run:
433
434```shell
435third_party/blink/tools/blink_tool.py update-metadata --build="linux-blink-rel"
436```
437
4384.2. Once the test completes on the builder, rerun that command to update the metadata.
439Commit the changes (if any), and upload the new patch to the CL.
440
4415. Add appropriate reviewers or comment the CL link on the PR.