WebRTC Browsers n Stacks Implementation differences
14 likes4,866 views
This document discusses differences in how browsers implement WebRTC. It describes the architectures of Chrome, Firefox, and Webkit and how they vary in their use of the libWebRTC library. For Chrome, it notes that Chrome does not use libWebRTC directly due to its layered architecture and security model. It also provides code snippets to enable logging to trace the different browser implementations without recompiling.
WebRTC Browsers n Stacks Implementation differences
- 1. Browsers and stack implementation differences Dr. Alex Gouaillard
- 2. Plan • global architectures of browsers and stacks • -- chrome / firefox / webkit • -- webrtc stand alone, webrtc in chrome, webrtc in firefox • - how to trace the different browsers without recompiling
- 3. WebRTC Arch
- 6. Chrome Variation • Wait .. What? Chrome does not use libwebrtc? Why?
- 9. Chrome Variation • Wait .. What? Chrome does not use libwebrtc? Why? – Chrome layering – Sandbox and per tab security – Echo canceling – Special capturers (screen/window/tab) – Capturer access security – Pretty much the same applies to all browsers
- 10. Chrome Variations • OK, Chrome does not use libwebrtc? Why should I Care? • If you use the webrtc lib in your project, and expect the same results as chrome, be ready for big surprises! – No Screen/windows/tab sharing – Obsolete mac capturer – No HTTP PROXY support
- 11. LibWEBRTC src Files layout
- 12. Libwebrtc src file layout • base/ • common_audio/ • common_video/ • libjingle/ • modules/ • p2p/ • sound/ • system_wrappers/ • tools/ • video/ • video_engine/ • voice_engine/ • Too many to list • Basic signal processing (FFT, window, transforms, …) • Frame buffer, I420, libyuv, … • Session management • See next slide • ICE Transports: TURN, STUN, DTLS, …. • Platform specific sound system wrappers • Platform specific low level wrappers (threads, timer, …) • Poach and weaponize!! • Video encoder/decoder/rec and send stream • Capturer, Channel, Encoder, Sync, REMB, Send • Channel, DTMF, …
- 13. Libwebrtc src file layout • audio_coding/ • audio_conference_mixer/ • audio_device/ • audio_processing/ • bitrate_controller/ • desktop_capture/ • media_file/ • pacing/ • remote_bitrate_estimator/ • rtp_rtcp/ • utility/ • video_capture/ • video_coding/ • video_processing/ • video_render/ • ACM2, NACK, NETQ, DTMF tone gen., …. • mixer • Platform specific • Noise, typing, gain detection, NS, Echo Cancellation, … • Bandwidth estimation and bitrate controller • Desktop capture (abstract only) • Audio files reading/writing • Paced sender • Obvious • RTP, RTCP, Payloads, FEC, … • More Audio files utilities, rtp_dump, recorder, player • Generic and platform dependent capturers (OLD) • Codecs, encoders and decoders, send and rec, jitter, … • Video processing and image enhancement • Generic and platform dependent renderers (OLD)
- 14. Libwebrtc src file layout • Why do I care? – 1. you might need to go in the code – 2. you need to know the layout for debugging: chrome.exe --enable-logging -- vmodule=*/webrtc/*=2,*/libjingle/*=2,*=-2 --no- sandbox
- 16. LibWEBRTC Arch
- 17. LibWEBRTC DEPS • binutils • Boringssl • Colorama • Drmemory • Expat • Icu • instrumented_libraries • Jsoncpp • Libjpeg • libjpeg_turbo • Libsrtp • libudev • libvpx • Libyuv • llvm-build • Nss • Ocmock • openmax_dl • Opus • Protobuf • Sqlite • Syzygy • Usrsctp • Yasm • zlib
- 18. Webrtc.org
- 19. Chrome Specific Webrtc Bits • //src/content/renderer/media/ • //src/third_party/WebKit/public/platform/ • //src/third_party/WebKit/Source/modules/mediastream • //src/content/browser/media/ • //src/content/browser/media/capture/ • //src/media/capture/video/ • //src/content/browser/renderer_host/media/ • . • . • . • . • . • . • .
- 20. Chrome Specific Webrtc Bits // MediaStreamManager is used to open/enumerate media capture devices (video // supported now). Call flow: // 1. GenerateStream is called when a render process wants to use a capture // device. // 2. MediaStreamManager will ask MediaStreamUIController for permission to // use devices and for which device to use. // 3. MediaStreamManager will request the corresponding media device manager(s) // to enumerate available devices. The result will be given to // MediaStreamUIController. // 4. MediaStreamUIController will, by posting the request to UI, let the // users to select which devices to use and send callback to // MediaStreamManager with the result. // 5. MediaStreamManager will call the proper media device manager to open the // device and let the MediaStreamRequester know it has been done.
- 21. Chrome GUM example capturers / frames / Encoding / ….
- 22. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser)GUM JS API (tab/sandbox) 1. Send request
- 23. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser)GUM JS API (tab/sandbox) 2. Check if MST is already available
- 24. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser)GUM JS API (tab/sandbox) Security Manager (source, origin) 3. Check rights 2. Check if MST is already available - NO
- 25. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) GUM JS API (tab/sandbox) 4. Ask Corresponding capturer type to start capturing 3. Check rights - OK
- 26. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) 4. Ask Corresponding capturer type to create one - OK V 5. Store the MST
- 27. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V 6. Trigger callback Keep feeding frames
- 28. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V NOTE 1: second call for same device with same constraints will directly return the MST, that allows to share streams across tabs without blocking
- 29. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V NOTE 2: Recently, a second call for the same device but with different constraints (think simulcast) will indeed return a different resolution. Before it would return the first resolution asked.
- 30. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V NOTE 3: Not only this allow to share cams across processes, it allows for global echo cancellation (yes, including the key strokes). Before tabs could cross feed.
- 31. Chrome Screensharing 2 steps (1) Media Stream Manager (singleton@browser) Screen/Windows/Tab Capturer Security Manager (source, origin) Screensharing (extension) 1 2 3 4 S 5
- 32. Chrome Screensharing 2 steps (2) Media Stream Manager (singleton@browser) Screen/Windows/Tab Capturer S GUM JS API (tab/sandbox) With ID
- 34. Firefox Variation – Desktop 1process
- 35. Firefox Variation – B2G
- 36. FireFox Arch • PeerConnection – PeerConnection.js -- shim translation layer to let us do API adaptation to the C++ – PeerConnectionImpl -- C++ implementation of the PeerConnection interface. – SIPCC -- handles SDP and media negotiation. Provided by Cisco but not a downstream. • Media – Webrtc.org/GIPS -- handles media encoding and decoding. Downstream from Google. – MediaConduit -- Generic wrapper around Webrtc.org – MediaPipeline -- Wrapper to hold the MediaConduit, mtransport subsystem, and the SRTP contexts, as well as interface with MediaStreams. • Transport – mtransport -- generic transport subsystem with implementations for ICE, DTLS, etc. – NSS -- new DTLS stack. Mentioned because we need to land the new version of NSS – nICEr -- ICE stack; downstream from reSIProcate project – nrappkit --portable runtime, utility library; downstream from nrappkit.sourceforge.net • DataChannel – DataChannel implementation in the DOM – libsctp -- SCTP implementation; downstream from the BSD SCTP guys
- 37. Webkit (safari ?) Current work: - Implement datachannel for webkit (yin liu) - Implement webrtc.org as a back end for webkit (adam tiamou) - Implement windows support for OWR
- 38. Webkit Arch
- 39. Webkit Arch
- 41. Webkit Arch – Data Channel
- 42. Webkit Arch – Data Channel