Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 1 | # WebDriver BiDi for Chromium [](https://www.npmjs.com/package/chromium-bidi) |
| 2 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 3 | ## CI status |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 4 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 5 |  |
| 6 |  |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 7 |  |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 8 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 9 |  |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 10 | |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 11 | [](https://codecov.io/gh/GoogleChromeLabs/chromium-bidi) |
| 12 | |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 13 | This is an implementation of the |
| 14 | [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) protocol with some |
| 15 | extensions (**BiDi+**) |
| 16 | for Chromium, implemented as a JavaScript layer translating between BiDi and CDP, |
| 17 | running inside a Chrome tab. |
| 18 | |
| 19 | Current status can be checked |
| 20 | at [WPT WebDriver BiDi status](https://wpt.fyi/results/webdriver/tests/bidi). |
| 21 | |
| 22 | ## BiDi+ |
| 23 | |
| 24 | **"BiDi+"** is an extension of the WebDriver BiDi protocol. In addition to [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) it has: |
| 25 | |
| 26 | ### Command `cdp.sendCommand` |
| 27 | |
| 28 | ```cddl |
| 29 | CdpSendCommandCommand = { |
| 30 | method: "cdp.sendCommand", |
| 31 | params: ScriptEvaluateParameters, |
| 32 | } |
| 33 | |
| 34 | CdpSendCommandParameters = { |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 35 | method: text, |
| 36 | params: any, |
| 37 | session?: text, |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 38 | } |
| 39 | |
| 40 | CdpSendCommandResult = { |
| 41 | result: any, |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 42 | session: text, |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 43 | } |
| 44 | ``` |
| 45 | |
| 46 | The command runs the |
| 47 | described [CDP command](https://chromedevtools.github.io/devtools-protocol) |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 48 | and returns the result. |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 49 | |
| 50 | ### Command `cdp.getSession` |
| 51 | |
| 52 | ```cddl |
| 53 | CdpGetSessionCommand = { |
| 54 | method: "cdp.sendCommand", |
| 55 | params: ScriptEvaluateParameters, |
| 56 | } |
| 57 | |
| 58 | CdpGetSessionParameters = { |
| 59 | context: BrowsingContext, |
| 60 | } |
| 61 | |
| 62 | CdpGetSessionResult = { |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 63 | session: text, |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 64 | } |
| 65 | ``` |
| 66 | |
| 67 | The command returns the default CDP session for the selected browsing context. |
| 68 | |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 69 | ### Events `cdp` |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 70 | |
| 71 | ```cddl |
| 72 | CdpEventReceivedEvent = { |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 73 | method: "cdp.<CDP Event Name>", |
| 74 | params: CdpEventReceivedParameters, |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 75 | } |
| 76 | |
| 77 | CdpEventReceivedParameters = { |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 78 | event: text, |
| 79 | params: any, |
| 80 | session: text, |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 81 | } |
| 82 | ``` |
| 83 | |
| 84 | The event contains a CDP event. |
| 85 | |
| 86 | ### Field `channel` |
| 87 | |
| 88 | Each command can be extended with a `channel`: |
| 89 | |
| 90 | ```cddl |
| 91 | Command = { |
| 92 | id: js-uint, |
| 93 | channel?: text, |
| 94 | CommandData, |
| 95 | Extensible, |
| 96 | } |
| 97 | ``` |
| 98 | |
| 99 | If provided and non-empty string, the very same `channel` is added to the response: |
| 100 | |
| 101 | ```cddl |
| 102 | CommandResponse = { |
| 103 | id: js-uint, |
| 104 | channel?: text, |
| 105 | result: ResultData, |
| 106 | Extensible, |
| 107 | } |
| 108 | |
| 109 | ErrorResponse = { |
| 110 | id: js-uint / null, |
| 111 | channel?: text, |
| 112 | error: ErrorCode, |
| 113 | message: text, |
| 114 | ?stacktrace: text, |
| 115 | Extensible |
| 116 | } |
| 117 | ``` |
| 118 | |
| 119 | When client uses |
| 120 | commands [`session.subscribe`](https://w3c.github.io/webdriver-bidi/#command-session-subscribe) |
| 121 | and [`session.unsubscribe`](https://w3c.github.io/webdriver-bidi/#command-session-unsubscribe) |
| 122 | with `channel`, the subscriptions are handled per channel, and the corresponding |
| 123 | `channel` filed is added to the event message: |
| 124 | |
| 125 | ```cddl |
| 126 | Event = { |
| 127 | channel?: text, |
| 128 | EventData, |
| 129 | Extensible, |
| 130 | } |
| 131 | ``` |
| 132 | |
| 133 | ## Dev Setup |
| 134 | |
| 135 | ### `npm` |
| 136 | |
| 137 | This is a Node.js project, so install dependencies as usual: |
| 138 | |
| 139 | ```sh |
| 140 | npm install |
| 141 | ``` |
| 142 | |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 143 | ### `cargo` |
| 144 | |
| 145 | <!-- TODO(jrandolf): Remove after binaries get published --> |
| 146 | |
Alex Rudenko | 662ce9c | 2023-08-29 08:05:30 | [diff] [blame^] | 147 | We use [cddlconv](https://github.com/google/cddlconv) to generate our WebDriverBidi types before building. |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 148 | |
| 149 | 1. Install [Rust](https://rustup.rs/). |
Alex Rudenko | 662ce9c | 2023-08-29 08:05:30 | [diff] [blame^] | 150 | 2. Run `cargo install --git https://github.com/google/cddlconv.git cddlconv` |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 151 | |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 152 | ### pre-commit.com integration |
| 153 | |
| 154 | Refer to the documentation at [.pre-commit-config.yaml](.pre-commit-config.yaml). |
| 155 | |
| 156 | ### Starting the Server |
| 157 | |
| 158 | This will run the server on port `8080`: |
| 159 | |
| 160 | ```sh |
| 161 | npm run server |
| 162 | ``` |
| 163 | |
| 164 | Use the `PORT=` environment variable or `--port=` argument to run it on another port: |
| 165 | |
| 166 | ```sh |
| 167 | PORT=8081 npm run server |
| 168 | npm run server -- --port=8081 |
| 169 | ``` |
| 170 | |
| 171 | Use the `DEBUG` environment variable to see debug info: |
| 172 | |
| 173 | ```sh |
| 174 | DEBUG=* npm run server |
| 175 | ``` |
| 176 | |
| 177 | Use the CLI argument `--headless=false` to run browser in headful mode: |
| 178 | |
| 179 | ```sh |
| 180 | npm run server -- --headless=false |
| 181 | ``` |
| 182 | |
| 183 | Use the `CHANNEL=...` environment variable or `--channel=...` argument with one of |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 184 | the following values to run the specific Chrome channel: `stable`, |
| 185 | `beta`, `canary`, `dev`. |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 186 | |
| 187 | The requested Chrome version should be installed. |
| 188 | |
| 189 | ```sh |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 190 | CHANNEL=dev npm run server |
| 191 | npm run server -- --channel=dev |
| 192 | ``` |
| 193 | |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 194 | Use the CLI argument `--verbose` to have CDP events printed to the console. Note: you have to enable debugging output `bidi:mapper:debug:*` as well. |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 195 | |
| 196 | ```sh |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 197 | DEBUG=bidi:mapper:debug:* npm run server -- --verbose |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 198 | ``` |
| 199 | |
| 200 | or |
| 201 | |
| 202 | ```sh |
| 203 | DEBUG=* npm run server -- --verbose |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 204 | ``` |
| 205 | |
| 206 | ### Starting on Linux and Mac |
| 207 | |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 208 | TODO: verify it works on Windows. |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 209 | |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 210 | You can also run the server by using `npm run server`. It will write |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 211 | output to the file `log.txt`: |
| 212 | |
| 213 | ```sh |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 214 | npm run server -- --port=8081 --headless=false |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 215 | ``` |
| 216 | |
| 217 | ## Running |
| 218 | |
| 219 | ### Unit tests |
| 220 | |
| 221 | Running: |
| 222 | |
| 223 | ```sh |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 224 | npm run unit |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 225 | ``` |
| 226 | |
| 227 | ### E2E tests |
| 228 | |
| 229 | The E2E tests are written using Python, in order to learn how to eventually do |
| 230 | this in web-platform-tests. |
| 231 | |
| 232 | ### Installation |
| 233 | |
| 234 | Python 3.6+ and some dependencies are required: |
| 235 | |
| 236 | ```sh |
| 237 | python3 -m pip install --user -r tests/requirements.txt |
| 238 | ``` |
| 239 | |
| 240 | ### Running |
| 241 | |
| 242 | The E2E tests require BiDi server running on the same host. By default, tests |
| 243 | try to connect to the port `8080`. The server can be run from the project root: |
| 244 | |
| 245 | ```sh |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 246 | npm run e2e # alias to to e2e-headless |
| 247 | npm run e2e-headful |
| 248 | npm run e2e-headless |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 249 | ``` |
| 250 | |
| 251 | Use the `PORT` environment variable to connect to another port: |
| 252 | |
| 253 | ```sh |
| 254 | PORT=8081 npm run e2e |
| 255 | ``` |
| 256 | |
| 257 | ### Examples |
| 258 | |
| 259 | Refer to [examples/README.md](examples/README.md). |
| 260 | |
| 261 | ## WPT (Web Platform Tests) |
| 262 | |
| 263 | WPT is added as |
| 264 | a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). To get run |
| 265 | WPT tests: |
| 266 | |
| 267 | ### Check out and setup WPT |
| 268 | |
| 269 | #### 1. Check out WPT |
| 270 | |
| 271 | ```sh |
| 272 | git submodule update --init |
| 273 | ``` |
| 274 | |
| 275 | #### 2. Go to the WPT folder |
| 276 | |
| 277 | ```sh |
| 278 | cd wpt |
| 279 | ``` |
| 280 | |
| 281 | #### 3. Set up virtualenv |
| 282 | |
| 283 | Follow the [_System |
| 284 | Setup_](https://web-platform-tests.org/running-tests/from-local-system.html#system-setup) |
| 285 | instructions. |
| 286 | |
| 287 | #### 4. Setup `hosts` file |
| 288 | |
| 289 | Follow |
| 290 | the [`hosts` File Setup](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup) |
| 291 | instructions. |
| 292 | |
| 293 | ##### 4.a On Linux, macOS or other UNIX-like system |
| 294 | |
| 295 | ```sh |
| 296 | ./wpt make-hosts-file | sudo tee -a /etc/hosts |
| 297 | ``` |
| 298 | |
| 299 | ##### 4.b On **Windows** |
| 300 | |
| 301 | This must be run in a PowerShell session with Administrator privileges: |
| 302 | |
| 303 | ```sh |
| 304 | python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append |
| 305 | ``` |
| 306 | |
| 307 | If you are behind a proxy, you also need to make sure the domains above are excluded |
| 308 | from your proxy lookups. |
| 309 | |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 310 | #### 5. Set `BROWSER_BIN` |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 311 | |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 312 | Set the `BROWSER_BIN` environment variable to a Chrome, Edge or Chromium binary to launch. |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 313 | For example, on macOS: |
| 314 | |
| 315 | ```sh |
| 316 | # Chrome |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 317 | export BROWSER_BIN="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary" |
| 318 | export BROWSER_BIN="/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev" |
| 319 | export BROWSER_BIN="/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta" |
| 320 | export BROWSER_BIN="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" |
| 321 | export BROWSER_BIN="/Applications/Chromium.app/Contents/MacOS/Chromium" |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 322 | |
| 323 | # Edge |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 324 | export BROWSER_BIN="/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary" |
| 325 | export BROWSER_BIN="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge" |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 326 | ``` |
| 327 | |
| 328 | ### Run WPT tests |
| 329 | |
| 330 | #### 1. Make sure you have Chrome Dev installed |
| 331 | |
| 332 | https://www.google.com/chrome/dev/ |
| 333 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 334 | #### 2. Build Chromedriver BiDi |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 335 | |
| 336 | Oneshot: |
| 337 | |
| 338 | ```sh |
| 339 | npm run build |
| 340 | ``` |
| 341 | |
| 342 | Continuously: |
| 343 | |
| 344 | ```sh |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 345 | npm run build --watch |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 346 | ``` |
| 347 | |
| 348 | #### 3. Run |
| 349 | |
| 350 | ```sh |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 351 | npm run wpt -- webdriver/tests/bidi/ |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 352 | ``` |
| 353 | |
| 354 | ### Update WPT expectations if needed |
| 355 | |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 356 | ```sh |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 357 | UPDATE_EXPECTATIONS=true npm run wpt -- webdriver/tests/bidi/ |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 358 | ``` |
| 359 | |
| 360 | ## How does it work? |
| 361 | |
| 362 | The architecture is described in the |
| 363 | [WebDriver BiDi in Chrome Context implementation plan](https://docs.google.com/document/d/1VfQ9tv0wPSnb5TI-MOobjoQ5CXLnJJx9F_PxOMQc8kY) |
| 364 | . |
| 365 | |
| 366 | There are 2 main modules: |
| 367 | |
| 368 | 1. backend WS server in `src`. It runs webSocket server, and for each ws connection |
| 369 | runs an instance of browser with BiDi Mapper. |
| 370 | 2. front-end BiDi Mapper in `src/bidiMapper`. Gets BiDi commands from the backend, |
| 371 | and map them to CDP commands. |
| 372 | |
| 373 | ### Contributing |
| 374 | |
| 375 | The BiDi commands are processed in the `src/bidiMapper/commandProcessor.ts`. To add a |
| 376 | new command, add it to `_processCommand`, write and call processor for it. |
| 377 | |
| 378 | ### Publish new `npm` release |
| 379 | |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 380 | #### Automatic release |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 381 | |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 382 | We use [release-please](https://github.com/googleapis/release-please) to automate releases. When a release should be done, check for the release PR in our [pull requests](https://github.com/GoogleChromeLabs/chromium-bidi/pulls) and merge it. |
Randolf Jung | bcb3bc8 | 2023-06-26 16:30:14 | [diff] [blame] | 383 | |
| 384 | #### Manual release |
| 385 | |
| 386 | 1. Dry-run |
| 387 | |
| 388 | ```sh |
| 389 | npm publish --dry-run |
| 390 | ``` |
| 391 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 392 | 1. Open a PR bumping the chromium-bidi version number in `package.json` for review: |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 393 | |
| 394 | ```sh |
Alex Rudenko | 1552f2b | 2023-07-11 11:18:32 | [diff] [blame] | 395 | npm version patch -m 'chore: Release v%s' --no-git-tag-version |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 396 | ``` |
| 397 | |
| 398 | Instead of `patch`, use `minor` or `major` [as needed](https://semver.org/). |
| 399 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame] | 400 | 1. After the PR is reviewed, [create a GitHub release](https://github.com/GoogleChromeLabs/chromium-bidi/releases/new) specifying the tag name matching the bumped version. |
| 401 | Our CI then automatically publishes the new release to npm based on the tag name. |
Randolf Jung | 3e52631 | 2023-08-08 06:20:39 | [diff] [blame] | 402 | |
| 403 | #### Roll into Chromium |
| 404 | |
| 405 | > Assuming that you already have a Chromium set-up locally, and knowledge on how to submit changes to the repo. |
| 406 | > Else submit an issue for a project maintainer to do. |
| 407 | |
| 408 | 1. Create a new branch. |
| 409 | 2. Navigate to `third_party/bidimapper/`. |
| 410 | 3. Then run the following commands: |
| 411 | |
| 412 | ```bash |
| 413 | ./pull.sh |
| 414 | ./build.sh |
| 415 | ``` |
| 416 | |
| 417 | 4. Submit a CL with bug `chromedriver:4226` |
| 418 | 5. Add appropriate reviewers or comment the CL link on the PR |