Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 1 | # WebDriver BiDi for Chromium [](https://www.npmjs.com/package/chromium-bidi) |
| 2 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 3 | ## CI status |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 4 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 5 |  |
| 6 |  |
| 7 |  |
| 8 |  |
| 9 | |
| 10 |  |
| 11 |  |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 12 | |
| 13 | This is an implementation of the |
| 14 | [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) protocol with some |
| 15 | extensions (**BiDi+**) |
| 16 | for Chromium, implemented as a JavaScript layer translating between BiDi and CDP, |
| 17 | running inside a Chrome tab. |
| 18 | |
| 19 | Current status can be checked |
| 20 | at [WPT WebDriver BiDi status](https://wpt.fyi/results/webdriver/tests/bidi). |
| 21 | |
| 22 | ## BiDi+ |
| 23 | |
| 24 | **"BiDi+"** is an extension of the WebDriver BiDi protocol. In addition to [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/) it has: |
| 25 | |
| 26 | ### Command `cdp.sendCommand` |
| 27 | |
| 28 | ```cddl |
| 29 | CdpSendCommandCommand = { |
| 30 | method: "cdp.sendCommand", |
| 31 | params: ScriptEvaluateParameters, |
| 32 | } |
| 33 | |
| 34 | CdpSendCommandParameters = { |
| 35 | cdpMethod: text, |
| 36 | cdpParams: any, |
| 37 | cdpSession?: text, |
| 38 | } |
| 39 | |
| 40 | CdpSendCommandResult = { |
| 41 | result: any, |
| 42 | cdpSession: text, |
| 43 | } |
| 44 | ``` |
| 45 | |
| 46 | The command runs the |
| 47 | described [CDP command](https://chromedevtools.github.io/devtools-protocol) |
| 48 | and returns result. |
| 49 | |
| 50 | ### Command `cdp.getSession` |
| 51 | |
| 52 | ```cddl |
| 53 | CdpGetSessionCommand = { |
| 54 | method: "cdp.sendCommand", |
| 55 | params: ScriptEvaluateParameters, |
| 56 | } |
| 57 | |
| 58 | CdpGetSessionParameters = { |
| 59 | context: BrowsingContext, |
| 60 | } |
| 61 | |
| 62 | CdpGetSessionResult = { |
| 63 | cdpSession: text, |
| 64 | } |
| 65 | ``` |
| 66 | |
| 67 | The command returns the default CDP session for the selected browsing context. |
| 68 | |
| 69 | ### Event `cdp.eventReceived` |
| 70 | |
| 71 | ```cddl |
| 72 | CdpEventReceivedEvent = { |
| 73 | method: "cdp.eventReceived", |
| 74 | params: ScriptEvaluateParameters, |
| 75 | } |
| 76 | |
| 77 | CdpEventReceivedParameters = { |
| 78 | cdpMethod: text, |
| 79 | cdpParams: any, |
| 80 | cdpSession: string, |
| 81 | } |
| 82 | ``` |
| 83 | |
| 84 | The event contains a CDP event. |
| 85 | |
| 86 | ### Field `channel` |
| 87 | |
| 88 | Each command can be extended with a `channel`: |
| 89 | |
| 90 | ```cddl |
| 91 | Command = { |
| 92 | id: js-uint, |
| 93 | channel?: text, |
| 94 | CommandData, |
| 95 | Extensible, |
| 96 | } |
| 97 | ``` |
| 98 | |
| 99 | If provided and non-empty string, the very same `channel` is added to the response: |
| 100 | |
| 101 | ```cddl |
| 102 | CommandResponse = { |
| 103 | id: js-uint, |
| 104 | channel?: text, |
| 105 | result: ResultData, |
| 106 | Extensible, |
| 107 | } |
| 108 | |
| 109 | ErrorResponse = { |
| 110 | id: js-uint / null, |
| 111 | channel?: text, |
| 112 | error: ErrorCode, |
| 113 | message: text, |
| 114 | ?stacktrace: text, |
| 115 | Extensible |
| 116 | } |
| 117 | ``` |
| 118 | |
| 119 | When client uses |
| 120 | commands [`session.subscribe`](https://w3c.github.io/webdriver-bidi/#command-session-subscribe) |
| 121 | and [`session.unsubscribe`](https://w3c.github.io/webdriver-bidi/#command-session-unsubscribe) |
| 122 | with `channel`, the subscriptions are handled per channel, and the corresponding |
| 123 | `channel` filed is added to the event message: |
| 124 | |
| 125 | ```cddl |
| 126 | Event = { |
| 127 | channel?: text, |
| 128 | EventData, |
| 129 | Extensible, |
| 130 | } |
| 131 | ``` |
| 132 | |
| 133 | ## Dev Setup |
| 134 | |
| 135 | ### `npm` |
| 136 | |
| 137 | This is a Node.js project, so install dependencies as usual: |
| 138 | |
| 139 | ```sh |
| 140 | npm install |
| 141 | ``` |
| 142 | |
| 143 | ### pre-commit.com integration |
| 144 | |
| 145 | Refer to the documentation at [.pre-commit-config.yaml](.pre-commit-config.yaml). |
| 146 | |
| 147 | ### Starting the Server |
| 148 | |
| 149 | This will run the server on port `8080`: |
| 150 | |
| 151 | ```sh |
| 152 | npm run server |
| 153 | ``` |
| 154 | |
| 155 | Use the `PORT=` environment variable or `--port=` argument to run it on another port: |
| 156 | |
| 157 | ```sh |
| 158 | PORT=8081 npm run server |
| 159 | npm run server -- --port=8081 |
| 160 | ``` |
| 161 | |
| 162 | Use the `DEBUG` environment variable to see debug info: |
| 163 | |
| 164 | ```sh |
| 165 | DEBUG=* npm run server |
| 166 | ``` |
| 167 | |
| 168 | Use the CLI argument `--headless=false` to run browser in headful mode: |
| 169 | |
| 170 | ```sh |
| 171 | npm run server -- --headless=false |
| 172 | ``` |
| 173 | |
| 174 | Use the `CHANNEL=...` environment variable or `--channel=...` argument with one of |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 175 | the following values to run the specific Chrome channel: `stable`, |
| 176 | `beta`, `canary`, `dev`. |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 177 | |
| 178 | The requested Chrome version should be installed. |
| 179 | |
| 180 | ```sh |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 181 | CHANNEL=dev npm run server |
| 182 | npm run server -- --channel=dev |
| 183 | ``` |
| 184 | |
| 185 | Use the CLI argument `--verbose` to have CDP events printed to the console. Note: you have to enable debugging output `bidiMapper:mapperDebug:*` as well. |
| 186 | |
| 187 | ```sh |
| 188 | DEBUG=bidiMapper:mapperDebug:* npm run server -- --verbose |
| 189 | ``` |
| 190 | |
| 191 | or |
| 192 | |
| 193 | ```sh |
| 194 | DEBUG=* npm run server -- --verbose |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 195 | ``` |
| 196 | |
| 197 | ### Starting on Linux and Mac |
| 198 | |
| 199 | TODO: verify if it works on Windows. |
| 200 | |
| 201 | You can also run the server by using script `./runBiDiServer.sh`. It will write |
| 202 | output to the file `log.txt`: |
| 203 | |
| 204 | ```sh |
| 205 | ./runBiDiServer.sh --port=8081 --headless=false |
| 206 | ``` |
| 207 | |
| 208 | ## Running |
| 209 | |
| 210 | ### Unit tests |
| 211 | |
| 212 | Running: |
| 213 | |
| 214 | ```sh |
| 215 | npm test |
| 216 | ``` |
| 217 | |
| 218 | ### E2E tests |
| 219 | |
| 220 | The E2E tests are written using Python, in order to learn how to eventually do |
| 221 | this in web-platform-tests. |
| 222 | |
| 223 | ### Installation |
| 224 | |
| 225 | Python 3.6+ and some dependencies are required: |
| 226 | |
| 227 | ```sh |
| 228 | python3 -m pip install --user -r tests/requirements.txt |
| 229 | ``` |
| 230 | |
| 231 | ### Running |
| 232 | |
| 233 | The E2E tests require BiDi server running on the same host. By default, tests |
| 234 | try to connect to the port `8080`. The server can be run from the project root: |
| 235 | |
| 236 | ```sh |
| 237 | npm run e2e |
| 238 | ``` |
| 239 | |
| 240 | Use the `PORT` environment variable to connect to another port: |
| 241 | |
| 242 | ```sh |
| 243 | PORT=8081 npm run e2e |
| 244 | ``` |
| 245 | |
| 246 | ### Examples |
| 247 | |
| 248 | Refer to [examples/README.md](examples/README.md). |
| 249 | |
| 250 | ## WPT (Web Platform Tests) |
| 251 | |
| 252 | WPT is added as |
| 253 | a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). To get run |
| 254 | WPT tests: |
| 255 | |
| 256 | ### Check out and setup WPT |
| 257 | |
| 258 | #### 1. Check out WPT |
| 259 | |
| 260 | ```sh |
| 261 | git submodule update --init |
| 262 | ``` |
| 263 | |
| 264 | #### 2. Go to the WPT folder |
| 265 | |
| 266 | ```sh |
| 267 | cd wpt |
| 268 | ``` |
| 269 | |
| 270 | #### 3. Set up virtualenv |
| 271 | |
| 272 | Follow the [_System |
| 273 | Setup_](https://web-platform-tests.org/running-tests/from-local-system.html#system-setup) |
| 274 | instructions. |
| 275 | |
| 276 | #### 4. Setup `hosts` file |
| 277 | |
| 278 | Follow |
| 279 | the [`hosts` File Setup](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup) |
| 280 | instructions. |
| 281 | |
| 282 | ##### 4.a On Linux, macOS or other UNIX-like system |
| 283 | |
| 284 | ```sh |
| 285 | ./wpt make-hosts-file | sudo tee -a /etc/hosts |
| 286 | ``` |
| 287 | |
| 288 | ##### 4.b On **Windows** |
| 289 | |
| 290 | This must be run in a PowerShell session with Administrator privileges: |
| 291 | |
| 292 | ```sh |
| 293 | python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append |
| 294 | ``` |
| 295 | |
| 296 | If you are behind a proxy, you also need to make sure the domains above are excluded |
| 297 | from your proxy lookups. |
| 298 | |
| 299 | #### 5. Set `WPT_BROWSER_PATH` |
| 300 | |
| 301 | Set the `WPT_BROWSER_PATH` environment variable to a Chrome, Edge or Chromium binary to launch. |
| 302 | For example, on macOS: |
| 303 | |
| 304 | ```sh |
| 305 | # Chrome |
| 306 | export WPT_BROWSER_PATH="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary" |
| 307 | export WPT_BROWSER_PATH="/Applications/Google Chrome Dev.app/Contents/MacOS/Google Chrome Dev" |
| 308 | export WPT_BROWSER_PATH="/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta" |
| 309 | export WPT_BROWSER_PATH="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" |
| 310 | export WPT_BROWSER_PATH="/Applications/Chromium.app/Contents/MacOS/Chromium" |
| 311 | |
| 312 | # Edge |
| 313 | export WPT_BROWSER_PATH="/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary" |
| 314 | export WPT_BROWSER_PATH="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge" |
| 315 | ``` |
| 316 | |
| 317 | ### Run WPT tests |
| 318 | |
| 319 | #### 1. Make sure you have Chrome Dev installed |
| 320 | |
| 321 | https://www.google.com/chrome/dev/ |
| 322 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 323 | #### 2. Build Chromedriver BiDi |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 324 | |
| 325 | Oneshot: |
| 326 | |
| 327 | ```sh |
| 328 | npm run build |
| 329 | ``` |
| 330 | |
| 331 | Continuously: |
| 332 | |
| 333 | ```sh |
| 334 | npm run watch |
| 335 | ``` |
| 336 | |
| 337 | #### 3. Run |
| 338 | |
| 339 | ```sh |
| 340 | ./wpt/wpt run \ |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 341 | --webdriver-binary runBiDiServer.sh \ |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 342 | --binary "$WPT_BROWSER_PATH" \ |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 343 | --manifest wpt/MANIFEST.json \ |
| 344 | --metadata wpt-metadata/mapper/headless \ |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 345 | chromium \ |
| 346 | webdriver/tests/bidi/ |
| 347 | ``` |
| 348 | |
| 349 | ### Update WPT expectations if needed |
| 350 | |
| 351 | #### 1. Run WPT tests with custom `log-wptreport`: |
| 352 | |
| 353 | ```sh |
| 354 | ./wpt/wpt run \ |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 355 | --webdriver-binary runBiDiServer.sh \ |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 356 | --binary "$WPT_BROWSER_PATH" \ |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 357 | --manifest wpt/MANIFEST.json \ |
| 358 | --metadata wpt-metadata/mapper/headless \ |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 359 | --log-wptreport wptreport.json \ |
| 360 | chromium \ |
| 361 | webdriver/tests/bidi/ |
| 362 | ``` |
| 363 | |
| 364 | #### 2. Update expectations based on the previous test run: |
| 365 | |
| 366 | ```sh |
| 367 | ./wpt/wpt update-expectations \ |
| 368 | --product chromium \ |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 369 | --manifest wpt/MANIFEST.json \ |
| 370 | --metadata wpt-metadata/mapper/headless \ |
| 371 | wptreport.json |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 372 | ``` |
| 373 | |
| 374 | ## How does it work? |
| 375 | |
| 376 | The architecture is described in the |
| 377 | [WebDriver BiDi in Chrome Context implementation plan](https://docs.google.com/document/d/1VfQ9tv0wPSnb5TI-MOobjoQ5CXLnJJx9F_PxOMQc8kY) |
| 378 | . |
| 379 | |
| 380 | There are 2 main modules: |
| 381 | |
| 382 | 1. backend WS server in `src`. It runs webSocket server, and for each ws connection |
| 383 | runs an instance of browser with BiDi Mapper. |
| 384 | 2. front-end BiDi Mapper in `src/bidiMapper`. Gets BiDi commands from the backend, |
| 385 | and map them to CDP commands. |
| 386 | |
| 387 | ### Contributing |
| 388 | |
| 389 | The BiDi commands are processed in the `src/bidiMapper/commandProcessor.ts`. To add a |
| 390 | new command, add it to `_processCommand`, write and call processor for it. |
| 391 | |
| 392 | ### Publish new `npm` release |
| 393 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 394 | 1. Open a PR bumping the chromium-bidi version number in `package.json` for review: |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 395 | |
| 396 | ```sh |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 397 | npm version patch -m 'Release v%s' --no-git-tag-version |
Randolf | b0fe9c1 | 2023-03-06 14:03:48 | [diff] [blame] | 398 | ``` |
| 399 | |
| 400 | Instead of `patch`, use `minor` or `major` [as needed](https://semver.org/). |
| 401 | |
Randolf | 5543146 | 2023-03-29 13:00:36 | [diff] [blame^] | 402 | 1. After the PR is reviewed, [create a GitHub release](https://github.com/GoogleChromeLabs/chromium-bidi/releases/new) specifying the tag name matching the bumped version. |
| 403 | Our CI then automatically publishes the new release to npm based on the tag name. |