blob: 66a4163109f8c40656063283be1d9a79c5d89954 [file] [log] [blame] [view]
Randolfb0fe9c12023-03-06 14:03:481# WebDriver BiDi for Chromium [![chromium-bidi on npm](https://img.shields.io/npm/v/chromium-bidi)](https://www.npmjs.com/package/chromium-bidi)
2
Randolf55431462023-03-29 13:00:363## CI status
Randolfb0fe9c12023-03-06 14:03:484
Randolf55431462023-03-29 13:00:365![E2E Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/e2e.yml/badge.svg)
6![Unit Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/unit.yml/badge.svg)
Randolf Jungbcb3bc82023-06-26 16:30:147![WPT Tests](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/wpt.yml/badge.svg)
Randolf55431462023-03-29 13:00:368
Randolf55431462023-03-29 13:00:369![Pre-commit](https://github.com/GoogleChromeLabs/chromium-bidi/actions/workflows/pre-commit.yml/badge.svg)
Randolfb0fe9c12023-03-06 14:03:4810
Randolf Jung3e526312023-08-08 06:20:3911[![codecov](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi/branch/main/graph/badge.svg?token=LJSXSC9L09)](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi)
12
Randolfb0fe9c12023-03-06 14:03:4813This is an implementation of the
14[WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) protocol with some
15extensions (**BiDi+**)
16for Chromium, implemented as a JavaScript layer translating between BiDi and CDP,
17running inside a Chrome tab.
18
19Current status can be checked
20at [WPT WebDriver BiDi status](https://wpt.fyi/results/webdriver/tests/bidi).
21
22## BiDi+
23
24**"BiDi+"** is an extension of the WebDriver BiDi protocol. In addition to [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) it has:
25
26### Command `cdp.sendCommand`
27
28```cddl
29CdpSendCommandCommand = {
30 method: "cdp.sendCommand",
31 params: ScriptEvaluateParameters,
32}
33
34CdpSendCommandParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3235 method: text,
36 params: any,
37 session?: text,
Randolfb0fe9c12023-03-06 14:03:4838}
39
40CdpSendCommandResult = {
41 result: any,
Alex Rudenko1552f2b2023-07-11 11:18:3242 session: text,
Randolfb0fe9c12023-03-06 14:03:4843}
44```
45
46The command runs the
47described [CDP command](https://chromedevtools.github.io/devtools-protocol)
Alex Rudenko1552f2b2023-07-11 11:18:3248and returns the result.
Randolfb0fe9c12023-03-06 14:03:4849
50### Command `cdp.getSession`
51
52```cddl
53CdpGetSessionCommand = {
54 method: "cdp.sendCommand",
55 params: ScriptEvaluateParameters,
56}
57
58CdpGetSessionParameters = {
59 context: BrowsingContext,
60}
61
62CdpGetSessionResult = {
Alex Rudenko1552f2b2023-07-11 11:18:3263 session: text,
Randolfb0fe9c12023-03-06 14:03:4864}
65```
66
67The command returns the default CDP session for the selected browsing context.
68
Alex Rudenko1552f2b2023-07-11 11:18:3269### Events `cdp`
Randolfb0fe9c12023-03-06 14:03:4870
71```cddl
72CdpEventReceivedEvent = {
Alex Rudenko1552f2b2023-07-11 11:18:3273 method: "cdp.<CDP Event Name>",
74 params: CdpEventReceivedParameters,
Randolfb0fe9c12023-03-06 14:03:4875}
76
77CdpEventReceivedParameters = {
Alex Rudenko1552f2b2023-07-11 11:18:3278 event: text,
79 params: any,
80 session: text,
Randolfb0fe9c12023-03-06 14:03:4881}
82```
83
84The event contains a CDP event.
85
86### Field `channel`
87
88Each command can be extended with a `channel`:
89
90```cddl
91Command = {
92 id: js-uint,
93 channel?: text,
94 CommandData,
95 Extensible,
96}
97```
98
99If provided and non-empty string, the very same `channel` is added to the response:
100
101```cddl
102CommandResponse = {
103 id: js-uint,
104 channel?: text,
105 result: ResultData,
106 Extensible,
107}
108
109ErrorResponse = {
110 id: js-uint / null,
111 channel?: text,
112 error: ErrorCode,
113 message: text,
114 ?stacktrace: text,
115 Extensible
116}
117```
118
119When client uses
120commands [`session.subscribe`](https://w3c.github.io/webdriver-bidi/#command-session-subscribe)
121and [`session.unsubscribe`](https://w3c.github.io/webdriver-bidi/#command-session-unsubscribe)
122with `channel`, the subscriptions are handled per channel, and the corresponding
123`channel` filed is added to the event message:
124
125```cddl
126Event = {
127 channel?: text,
128 EventData,
129 Extensible,
130}
131```
132
133## Dev Setup
134
135### `npm`
136
137This is a Node.js project, so install dependencies as usual:
138
139```sh
140npm install
141```
142
Randolf Jung3e526312023-08-08 06:20:39143### `cargo`
144
145<!-- TODO(jrandolf): Remove after binaries get published -->
146
Alex Rudenko662ce9c2023-08-29 08:05:30147We use [cddlconv](https://github.com/google/cddlconv) to generate our WebDriverBidi types before building.
Randolf Jung3e526312023-08-08 06:20:39148
1491. Install [Rust](https://rustup.rs/).
Alex Rudenko662ce9c2023-08-29 08:05:301502. Run `cargo install --git https://github.com/google/cddlconv.git cddlconv`
Randolf Jung3e526312023-08-08 06:20:39151
Randolfb0fe9c12023-03-06 14:03:48152### pre-commit.com integration
153
154Refer to the documentation at [.pre-commit-config.yaml](.pre-commit-config.yaml).
155
Alex Rudenkof7ea7ab2023-10-24 09:40:51156```sh
157pre-commit install --hook-type pre-push
158```
159
160### Starting WebDriver BiDi Server
Randolfb0fe9c12023-03-06 14:03:48161
162This will run the server on port `8080`:
163
164```sh
165npm run server
166```
167
168Use the `PORT=` environment variable or `--port=` argument to run it on another port:
169
170```sh
171PORT=8081 npm run server
172npm run server -- --port=8081
173```
174
175Use the `DEBUG` environment variable to see debug info:
176
177```sh
178DEBUG=* npm run server
179```
180
Alex Rudenko5fe99f82023-10-06 12:01:27181Use the `DEBUG_DEPTH` (default: `10`) environment variable to see debug deeply nested objects:
182
183```sh
184DEBUG_DEPTH=100 DEBUG=* npm run server
185```
186
Randolfb0fe9c12023-03-06 14:03:48187Use the CLI argument `--headless=false` to run browser in headful mode:
188
189```sh
190npm run server -- --headless=false
191```
192
193Use the `CHANNEL=...` environment variable or `--channel=...` argument with one of
Randolf55431462023-03-29 13:00:36194the following values to run the specific Chrome channel: `stable`,
195`beta`, `canary`, `dev`.
Randolfb0fe9c12023-03-06 14:03:48196
197The requested Chrome version should be installed.
198
199```sh
Randolf55431462023-03-29 13:00:36200CHANNEL=dev npm run server
201npm run server -- --channel=dev
202```
203
Alex Rudenko1552f2b2023-07-11 11:18:32204Use the CLI argument `--verbose` to have CDP events printed to the console. Note: you have to enable debugging output `bidi:mapper:debug:*` as well.
Randolf55431462023-03-29 13:00:36205
206```sh
Alex Rudenko1552f2b2023-07-11 11:18:32207DEBUG=bidi:mapper:debug:* npm run server -- --verbose
Randolf55431462023-03-29 13:00:36208```
209
210or
211
212```sh
213DEBUG=* npm run server -- --verbose
Randolfb0fe9c12023-03-06 14:03:48214```
215
216### Starting on Linux and Mac
217
Randolf Jungbcb3bc82023-06-26 16:30:14218TODO: verify it works on Windows.
Randolfb0fe9c12023-03-06 14:03:48219
Randolf Jung3e526312023-08-08 06:20:39220You can also run the server by using `npm run server`. It will write
Randolfb0fe9c12023-03-06 14:03:48221output to the file `log.txt`:
222
223```sh
Randolf Jung3e526312023-08-08 06:20:39224npm run server -- --port=8081 --headless=false
Randolfb0fe9c12023-03-06 14:03:48225```
226
Alex Rudenko5fe99f82023-10-06 12:01:27227### Running with in other project
228
229Sometimes it good to verify that a change will not affect thing downstream for other packages.
230There is a useful `puppeteer` label you can add to any PR to run Puppeteer test with your changes.
231It will bundle `chromium-bidi` and install it in Puppeteer project then run that package test.
232
Randolfb0fe9c12023-03-06 14:03:48233## Running
234
235### Unit tests
236
237Running:
238
239```sh
Randolf Jungbcb3bc82023-06-26 16:30:14240npm run unit
Randolfb0fe9c12023-03-06 14:03:48241```
242
243### E2E tests
244
245The E2E tests are written using Python, in order to learn how to eventually do
246this in web-platform-tests.
247
Alex Rudenko5fe99f82023-10-06 12:01:27248#### Installation
Randolfb0fe9c12023-03-06 14:03:48249
250Python 3.6+ and some dependencies are required:
251
252```sh
253python3 -m pip install --user -r tests/requirements.txt
254```
255
Alex Rudenko5fe99f82023-10-06 12:01:27256#### Running
Randolfb0fe9c12023-03-06 14:03:48257
258The E2E tests require BiDi server running on the same host. By default, tests
259try to connect to the port `8080`. The server can be run from the project root:
260
261```sh
Randolf Jungbcb3bc82023-06-26 16:30:14262npm run e2e # alias to to e2e-headless
263npm run e2e-headful
264npm run e2e-headless
Randolfb0fe9c12023-03-06 14:03:48265```
266
267Use the `PORT` environment variable to connect to another port:
268
269```sh
270PORT=8081 npm run e2e
271```
272
Alex Rudenkof7ea7ab2023-10-24 09:40:51273### Local http server
274
275E2E tests use local http
276server [`pytest-httpserver`](https://pytest-httpserver.readthedocs.io/), which is run
277automatically with the tests. However,
278sometimes it is useful to run the http server outside the test
279case, for example for manual debugging. This can be done by running:
280
281```sh
282python3 tools/run_local_http_server.py
283```
284
285or use npm:
286
287```sh
288npm run local-http-server
289```
290
291### Examples
Randolfb0fe9c12023-03-06 14:03:48292
293Refer to [examples/README.md](examples/README.md).
294
295## WPT (Web Platform Tests)
296
297WPT is added as
298a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). To get run
299WPT tests:
300
301### Check out and setup WPT
302
303#### 1. Check out WPT
304
305```sh
306git submodule update --init
307```
308
309#### 2. Go to the WPT folder
310
311```sh
312cd wpt
313```
314
315#### 3. Set up virtualenv
316
317Follow the [_System
318Setup_](https://web-platform-tests.org/running-tests/from-local-system.html#system-setup)
319instructions.
320
321#### 4. Setup `hosts` file
322
323Follow
324the [`hosts` File Setup](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup)
325instructions.
326
327##### 4.a On Linux, macOS or other UNIX-like system
328
329```sh
330./wpt make-hosts-file | sudo tee -a /etc/hosts
331```
332
333##### 4.b On **Windows**
334
335This must be run in a PowerShell session with Administrator privileges:
336
337```sh
338python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append
339```
340
341If you are behind a proxy, you also need to make sure the domains above are excluded
342from your proxy lookups.
343
Randolf Jungbcb3bc82023-06-26 16:30:14344#### 5. Set `BROWSER_BIN`
Randolfb0fe9c12023-03-06 14:03:48345
Randolf Jungbcb3bc82023-06-26 16:30:14346Set the `BROWSER_BIN` environment variable to a Chrome, Edge or Chromium binary to launch.
Randolfb0fe9c12023-03-06 14:03:48347For example, on macOS:
348
349```sh
350# Chrome
Randolf Jungbcb3bc82023-06-26 16:30:14351export BROWSER_BIN="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
352export BROWSER_BIN="/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev"
353export BROWSER_BIN="/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"
354export BROWSER_BIN="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
355export BROWSER_BIN="/Applications/Chromium.app/Contents/MacOS/Chromium"
Randolfb0fe9c12023-03-06 14:03:48356
357# Edge
Randolf Jungbcb3bc82023-06-26 16:30:14358export BROWSER_BIN="/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary"
359export BROWSER_BIN="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
Randolfb0fe9c12023-03-06 14:03:48360```
361
362### Run WPT tests
363
364#### 1. Make sure you have Chrome Dev installed
365
366https://www.google.com/chrome/dev/
367
Randolf55431462023-03-29 13:00:36368#### 2. Build Chromedriver BiDi
Randolfb0fe9c12023-03-06 14:03:48369
370Oneshot:
371
372```sh
373npm run build
374```
375
376Continuously:
377
378```sh
Randolf Jungbcb3bc82023-06-26 16:30:14379npm run build --watch
Randolfb0fe9c12023-03-06 14:03:48380```
381
382#### 3. Run
383
384```sh
Randolf Jungbcb3bc82023-06-26 16:30:14385npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48386```
387
388### Update WPT expectations if needed
389
Randolfb0fe9c12023-03-06 14:03:48390```sh
Randolf Jungbcb3bc82023-06-26 16:30:14391UPDATE_EXPECTATIONS=true npm run wpt -- webdriver/tests/bidi/
Randolfb0fe9c12023-03-06 14:03:48392```
393
394## How does it work?
395
396The architecture is described in the
397[WebDriver BiDi in Chrome Context implementation plan](https://docs.google.com/document/d/1VfQ9tv0wPSnb5TI-MOobjoQ5CXLnJJx9F_PxOMQc8kY)
398.
399
400There are 2 main modules:
401
4021. backend WS server in `src`. It runs webSocket server, and for each ws connection
403 runs an instance of browser with BiDi Mapper.
4042. front-end BiDi Mapper in `src/bidiMapper`. Gets BiDi commands from the backend,
405 and map them to CDP commands.
406
Alex Rudenko5fe99f82023-10-06 12:01:27407## Contributing
Randolfb0fe9c12023-03-06 14:03:48408
409The BiDi commands are processed in the `src/bidiMapper/commandProcessor.ts`. To add a
410new command, add it to `_processCommand`, write and call processor for it.
411
412### Publish new `npm` release
413
Alex Rudenko1552f2b2023-07-11 11:18:32414#### Automatic release
Randolf Jungbcb3bc82023-06-26 16:30:14415
Alex Rudenko1552f2b2023-07-11 11:18:32416We use [release-please](https://github.com/googleapis/release-please) to automate releases. When a release should be done, check for the release PR in our [pull requests](https://github.com/GoogleChromeLabs/chromium-bidi/pulls) and merge it.
Randolf Jungbcb3bc82023-06-26 16:30:14417
418#### Manual release
419
4201. Dry-run
421
422 ```sh
423 npm publish --dry-run
424 ```
425
Randolf55431462023-03-29 13:00:364261. Open a PR bumping the chromium-bidi version number in `package.json` for review:
Randolfb0fe9c12023-03-06 14:03:48427
428 ```sh
Alex Rudenko1552f2b2023-07-11 11:18:32429 npm version patch -m 'chore: Release v%s' --no-git-tag-version
Randolfb0fe9c12023-03-06 14:03:48430 ```
431
432 Instead of `patch`, use `minor` or `major` [as needed](https://semver.org/).
433
Randolf55431462023-03-29 13:00:364341. After the PR is reviewed, [create a GitHub release](https://github.com/GoogleChromeLabs/chromium-bidi/releases/new) specifying the tag name matching the bumped version.
435 Our CI then automatically publishes the new release to npm based on the tag name.
Randolf Jung3e526312023-08-08 06:20:39436
437#### Roll into Chromium
438
Alex Rudenkob013f872023-09-29 05:56:03439This section assumes you already have a Chromium set-up locally,
440and knowledge on [how to submit changes to the repo](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/contributing.md).
441Otherwise submit an issue for a project maintainer.
Randolf Jung3e526312023-08-08 06:20:39442
Alex Rudenkob013f872023-09-29 05:56:034431. Create a new branch in chromium `src/`.
4442. Update the mapper version:
Randolf Jung3e526312023-08-08 06:20:39445
Alex Rudenkob013f872023-09-29 05:56:03446```shell
447third_party/bidimapper/pull.sh
448third_party/bidimapper/build.sh
Randolf Jung3e526312023-08-08 06:20:39449```
450
Alex Rudenkob013f872023-09-29 05:56:034513. Submit a CL with bug `chromedriver:4226`.
4524. [Regenerate WPT expectations metadata](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/testing/web_platform_tests_wptrunner.md#updating-expectations):
453
4544.1. Trigger a build and test run:
455
456```shell
457third_party/blink/tools/blink_tool.py update-metadata --build="linux-blink-rel"
458```
459
4604.2. Once the test completes on the builder, rerun that command to update the metadata.
461Commit the changes (if any), and upload the new patch to the CL.
462
4635. Add appropriate reviewers or comment the CL link on the PR.