Recording and media manipulation of WebRTC streams
Download as PPTX, PDF3 likes3,204 views
The document discusses the development of WebRTC applications with advanced media processing, focusing on the functionalities and evolution of media servers. It highlights the Kurento Media Server, its features, and the capabilities it provides for real-time communications, including transcoding and augmented reality. Additionally, it provides guidance on installation and resources for using Kurento in developing WebRTC applications.
Recording and media manipulation of WebRTC streams
- 2. Developing WebRTC applications with advanced media processing Luis Lopez [email protected] www.kurento.org
- 3. WebRTC infrastructures: at the heart of the long tail Peer-to-Peer WebRTC Application (without media infrastructure) WebRTC video stream WebRTC Application based on media infrastructure media infrastructure
- 4. WebRTC media servers: the simplest WebRTC infrastructure • Media server – Monolithic software – Based on shared memory – Scales well vertically – Can’t scale horizontally • Media server vendors – Software vendors – Hardware vendors
- 5. WebRTC media server vendors
- 6. Common functions of WebRTC media servers Transcoding media server VP8 H.264 MCU/SFU media server Recording media server
- 7. Evolution of media servers: functionality What common WebRTC Media Servers do: • Transcoding • MCU • Recording What future Media Servers will do: • Flexible processing • Augmented reality • Blending • Mixing • Analyzing • Etc. Media is here Media goes there Current media servers Transcoding MCU/SFU Recording Media is here Rich Media goes there Future media servers Media Events Transcoding, MCU/SFU, Recording, Enrich, Augment, Analyze, Combine, Transform, Adapt, … Context Content Commands
- 8. Why is this important? Explicit augmentation with target information Implicit augmentation with external knowledge Why do you communicate? To see other’s face? WebRTC convergence with Telco Why don’t converge with Content?
- 9. Kurento Media Server: an open source software WebRTC media server Applications define the processing of streams getting through KMS Send Receive Analyze Augment Enrich Transform Transcode Record Process Replicate Media Source Media Sink KMS Java API JavaScript API Other languages
- 10. What does Kurento mean?
- 11. Kurento History 2012 •Kurento goes open source LGPL v2.1. Professional services provided by Naevatec (startup) June’13 •1M€ of investment for providing multimedia capabilities to FI-WARE Dec’13 •Kurento Featured at WebRTC Expo (Paris) Feb’14 •3.3M€ of investment for making Kurento an elastic cloud platform (NUBOMEDIA) April’14 •Kurento featured at IMS World Conference (Barcelone) July’14 •Kurento featured at DevCon5 (New York) Sep’14 •500K€ of investment for maintenance of Kurento FI-WARE APIs
- 12. Kurento Client API Sink SRC Sink SRC SRC Sink Sink Media Element • Provides a specific media functionality › Send/receive media › Process media › Transform media • Exchange media through › Sources › Sinks Media pipeline • Chain of media elements implementing the desired media logic. • The Media API provides the capability of creating media pipelines by joining media elements of the toolbox Media Element Sink SRC
- 13. Creating applications: the Lego game H pEndpoint WebRtcEndpoint Protocols & Codecs Application 2 IPTVConnector Mixer FaceDetector PointerTracker BarCodeReader Computer Vision HeartRateBlender ChromaFilter FaceOverlay Augmented Reality Mul sensory Mul media DataChannelEndpoint PlayerEndpoint RecorderEndpoint Media Repository IPTV Integra on Group Communica ons Sink% Sink% SRC% RtpEndpoint SRC$ Sink% SRC% Sink% SRC% Sink% Sink% SRC% SRC% Sink% SRC% Sink% Sink% SRC% SRC% Sink% Sink% SRC$ CdnUpload Sink% Sink SRC Sink Sink SRC SRC Sink Sink Application 3 Sink SRC Sink Sink SRC Sink Sink SRC Application 1 Sink SRC Sink SRC Sink SRC Sink SRC Toolbox of media elements
- 14. Architecting a Kurento application
- 15. Ready?
- 16. Installing Kurento • The doc – http://www.kurento.org/docs/current/installation_guide.html • The video – https://www.youtube.com/watch?v=RGdDpyZAPus • In short sudo add-apt-repository ppa:kurento/kurento wget -O - http://ubuntu.kurento.org/kurento.gpg.key | sudo apt-key add - sudo apt-get update sudo apt-get install kurento-media-server
- 17. Get a Kurento cloud instance for free • FI-WARE (http://www.fi-ware.org) – Funded through European Commission research programmes – PaaS cloud infrastructure offering a bunch of capabilities and enablers for the “Future Internet” – You can get money developing WebRTC applications with Kurento and FI-WARE • http://www.fi-ware.org/accelerators/ • How to get a Kurento instance in FIWARE in 5 minutes – https://www.youtube.com/watch?v=lFM12pXeWu8
- 19. Source & demo https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-hello-world
- 20. Recording and playing Media Pipeline WebRTC Streaming Sink SRC Sink Media to file or URI Media Pipeline WebRTC Streaming Sink SRC Media from file or URI SRC WebRtcEndpoint WebRtcEndpoint RecorderEndpoint PlayerEndpoint
- 21. Source code kurentoClient.create("MediaPipeline", function(error, p) { if(error) return onError(error); pipeline = p; pipeline.create("WebRtcEndpoint", function(error, webRtc){ if(error) return onError(error); pipeline.create(”RecorderEndpoint”, function(error, recorder){ if(error) return onError(error); webRtc.connect(recorder, function(error){ if(error) return onError(error); webRtc.connect(webRtc, function(error){ if(error) return onError(error); webRtc.processOffer(sdpOffer, … if(error) return onError(error); webRtcPeer.processSdpAnswer(sdpAnswer); }); }); }); }); });
- 22. I don’t like callbacks From that code structure to this code structure
- 23. Second try … generators • https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-hello-world-recorder-generator • Transactions very soon … – By end Q4’14
- 25. Source & demo • https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-magic-mirror
- 26. Create your own filter: working with Kurento external modules WebRTC Streaming Sink SRC WebRtcEndpoint Sink ArMarkerDetector SRC
- 27. Source & demo • https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-alvar
- 28. Putting it altogether: calls with recording and filtering
- 29. Source & demo: let’s switch to Java for fun • https://github.com/Kurento/kurento-tutorial-java/ tree/master/kurento-one2one-call-advanced
- 30. Bonus: going beyond the media server vision WebRTC Application based on media infrastructure WebRTC Cloud
- 31. WebRTC cloud infrastructures IaaS SaaS PaaS Provider - Computing resources Provider Developer - Development API - Installation - Administration Developer - Security - Application logic - Application logic Provider - Service Developer - Nothing to do
- 32. Multimedia PaaS: scalability types “Broadcasting Non-RTC” Number of concurrent sessions Number of users per session WebRTC PaaS Phone system TV Broadcasting CDNs Media Server based Services “RTC calls” “Media server RTC” “Beyond RTC calls”
- 33. The scalability problem in call clouds Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call
- 34. The scalability problem beyond call clouds Media stream One to MANY
- 35. WebRTC services beyond the call model Users in call Monitoring users Users can dynamically change their role
- 36. Nubomedia: Kurento as an elastic media server Load%Balancer%Func. on% Applica. on%Server%Func. on% Broker%Func. on% Media%Server%Func. on% IaaS%Cloud%Manager% Cloud% Orchestrator%
- 37. Thank you Partners Luis Lopez [email protected] www.kurento.org