Ebu mpeg dash-webinar043
- 1. DASH MPEG's Dynamic Adaptive Streaming over HTTP (DASH) – Enabling Formats for Video Streaming over the Open Internet Thomas Stockhammer Qualcomm Incorporated Webinar at EBU - Nov 22, 2011
- 2. 2
- 3. Long-start- DRM not up low quality supported You need frequent plugin stalls Wrong Bitrate Protocol intense Wrong User no DVD Format Frustration experience
- 4. Video is dominating the Internet • Internet: Real-time video is 50% of the traffic at peak periods – notably 30% from Netflix and 11% from Youtube • Mobile: Video traffic is growing exponentially & is a large portion. every year mobile video traffic doubles 66% mobile video by Source: Cisco White Paper: 2015 Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2010-2015
- 5. Apple HLS MPEG 2008 MS SS 2010 2008 OIPF W3C 2009 2011? 3GPP DASH others 2009 2011 Open Standard Delivery Format 5
- 6. MPEG DASH ISO/IEC 23009-1 • MPEG DASH ISO/IEC 23009-1 technically frozen in August 2011 • Timeline and Activities – ISO/IEC 23009-1 known as MPEG-DASH has been ratified unanimously by positive votes from 26 National bodies in November 2011 pending minor editorial improvements. – Editorial updates during MPEG#98 and processing at ITTF such that expected publication of standard in March 2012. – Parallel approval process for extensions to • ISO base media FF to support DASH 14496-12/AMD 3 • Common Encryption 23001-7 – Continuous coordination with 3GPP and other SDOs (DECE, OIPF, etc.) – Conformance and Reference Software activities kicked off (see WD 23009-2) • Good news: Converging standard for adaptive streaming on the way 6
- 7. DASH in a Nutshell • What: Video streaming solution where small pieces of video streams/files are requested with HTTP and spliced together by the client. Client entirely controls delivery. • Why: reuse widely deployed standard HTTP servers/caches for scalable delivery, e.g. existing Internet CDNs; traverse NAT/Firewalls; simple rate adaptation; fixed-mobile convergence; convergence of services, etc. • Use case: Accessing OTT video streaming services over any access network to any device HTTP over any Access Network (fixed, Media mobile) Preparation Media HTTP HTTP Origin Servers Caches
- 8. DASH in a Nutshell • What: Video streaming solution where small pieces of video streams/files are requested with HTTP and spliced together by the client. Client entirely controls delivery. • Why: reuse widely deployed standard HTTP servers/caches for scalable delivery, e.g. existing Internet CDNs; traverse NAT/Firewalls; simple rate adaptation; fixed-mobile convergence; convergence of services, etc. • Use case: Accessing OTT video streaming services over any access network to any device HTTP over any Access Network (fixed, Media mobile) Preparation Media HTTP HTTP Origin Servers Caches
- 9. Example Deployment Architecture Original • Original content encoded into video streams content at different media rates Content Media player Preparation • Segments video streams into HTTP resources • MPD file generated for the video files • URL generated for MPD file Ingestion component • All intelligence is here • Accesses MPD file based on URL & makes requests for appropriate video files • Continually monitors and adjusts media rate based on network conditions, etc. HTTP delivery Serving Cloud HTTP/IP (HTTP web caching servers) network DASH client Any radio access: any cellular, WiFi, parallel delivery 9
- 10. (Some) DASH Design Principles • DASH is not: – system, protocol, presentation, codec, middleware, client specification • DASH is an enabler – provides formats to enable efficient and high-quality delivery of streaming services over the Internet considered as one component in an e2e service – System definition left to other organizations (SDOs, Fora, Companies, etc.) • It attempts to be very good in what is to be addressed by the standard – Enables reuse of existing technologies (containers, codecs, DRM etc.) – Enables deployment on top of HTTP-CDNs (Web Infrastructures, caching) – Enables very high user-experience (low start-up, no rebuffering, trick modes) – Enables selection based on network and device capability, user preferences – Enables seamless switching – Enables live and DVD-kind of experiences – addresses global and regulatory deployment issues – Moves intelligence from network to client, enables client differentiation – Enables deployment flexibility (e.g., live, on-demand, time-shift viewing) – Provide simple interoperability points (profiles) – provides convergence with existing proprietary technologies in this space 10
- 12. What is specified – and what is not? Media Presentation on HTTP Media DASH Client Server Presentation Description Segment DASH Access Engine Resources on-time http located by requests to HTTP-URLs segments Media Engines HTTP HTTP/1.1 Access Client 12
- 13. Information Classification • MPD and Index Information for DASH Access client – Core specification aspects of DASH • Initilialisation and Media Segments for Media engine – Reuse of existing container formats and easy conversion – Small adaptations may be necessary for usage in DASH MPD DASH Media Media Access output Segment MPEG format engine data Client media + timing 13
- 14. Media Presentation Data Model • Media Presentation Description (MPD) describes accessible Segments and corresponding timing Segment Info Initialization Segment http://www.e.com/dash-5 Media Presentation Media Segment 1 Period, start=0s Period, start=0s •start=100 Representation 1 http://www.e.com/dahs-5-1 … … •baseURL=http://www.e.com/ •bandwidth=500kbit/s •width 640, height 480 Media Segment 2 Adaptation Set 1 … start=10s http://www.e.com/dash-5-2 Period, start=100s video Segment Info … duration=10s Media Segment 3 start=20s Adaptation Set 2 Template: http://www.e.com/das-5-3 audio ./dash-5-$Number$ Period, start=295s … … Representation 2 •bandwidth=250kbit/s … … •width 640, height 480 Media Segment 20 Splicing of Selection of start=190s arbitrary content Components/Tracks Select/Switch of http://www.e.com/dash-5-20 14 Bandwidth
- 15. MPD Information • Redundant information of Media Streams for the purpose to initially select or reject Adaptation Sets/Representations – Examples: Role, Codec, DRM, language, resolution, bandwidth • Access and Timing Information – the HTTP-URL(s) and byte range for each accessible Segment – the earliest next update of the MPD on the server – the segment availability start and end time in wall-clock time – the approximated presentation start time and duration of a Media Segment in the media presentation timeline – for live service, playout start instructions such that segments will be available in time for fluent playout in the future • Switching and splicing relationships across Representations • not much more … 15
- 17. Key feature – Common Timeline • Representations in one Period share common presentation timeline – presentation time of access unit within the media streams is mapped to the global common presentation timeline – enables synchronization of different media components and seamless switching of different coded versions of the same media components • Other timelines – segment availability times (mapped to UTC clock) – internal media decode time (not exposed on DASH level) 17
- 18. Adaptation Sets and Representations • Each Period consists of one or more Adaptation Sets. • contains alternate encodings of one or more media content components encapsulated in Representations • Representations encapsulate media streams that are considered to be perceptually equivalent • Representations are a deliverable encoded version – encapsulation in common media formats – segmentation possible for delivery optimization – concatentation of segments of one Representation complies with media format • Typically dynamic switching within one Adaptation Set 18
- 19. Client-Side Process Advertisement Back to video Segments Insertion 250 Kbps 500 Kbps 1 Mbps Segment 30s Period 12s Period 20s Period Process Factors relevant for Representation Client downloads the MPD file selection Client downloads segment by segment based Buffer conditions on playout process Network conditions Bitrate determined by client User change in resolution – ex: full screen Device activity and resources
- 20. Descriptors • Content Protection (2 schemes defined) • Role (1 scheme defined) – caption, subtitle, main, alternate, supplementary, commentary, dub • Accessibility (Role scheme may be used) • Rating • Viewpoint • Frame Packing (2 schemes defined) • Audio Channel Configuration (1 scheme defined) 20
- 21. Example for Role and Viewpoint 2 views, each possibly with different audio 21
- 22. Switching Support Features • Segment Alignment – permits non-overlapping decoding and presentation of segments from different Representations • Stream Access Points (SAPs) – presentation time and position in segments at which random access and switching can occur • Bitstream Switching – concatenation of segments from different Representations results in conforming bitstream • Media Stream Structure Similarity – permits switching at Open GOP boundaries • Alignment and SAPs can also apply for Subsegments 22
- 23. Preferable Switch Points • Segment/Subsegment boundaries for which – alignment holds across Representations – the switch-to Representation starts with a SAP Rep 1 Rep 2 Switch 1 Switch 2 23
- 24. Accessing Segments • Segment information provides URL, time window, type – Support of explicit lists and client side template construction rules • Multiple Base URLs – same information can be accessed at multiple locations – Redundancy, client-side load balancing, parallel download • Byte range access with regular GETs – mapping to byte ranges needs to be done in CDNs 24
- 25. Segments • ISO/IEC 23009 focuses on Segment formats based on MPEG container formats. Specifically, – in 6.3, Segment formats are described for use with Media Segments based on the ISO Base Media File Format as defined in ISO/IEC 14496-12; – In 6.4, Segment formats are described for use with Media Segments based on the MPEG-2 Transport Stream as defined in the ISO/IEC 13818-2; • In both cases the Segment formats are defined such that the Media Segment formats comply with the respective container formats. • Guidelines for adding other Segment formats are provided in Annex F. 25
- 26. ISO BMFF DASH Segments ftyp moov moof mdat moof mdat moof mdat Initialisation styp sidx moof mdat moof mdat Segment • binary information in ISO box structure on Media – Accessible units (one or multiple fragments) Segment – Each unit is described by • Byte range in the segments • Accurate presentation duration (seamless switching) • Presence and timing of stream access positions • Provides a compact bitrate-over-time profile – Can be used for intelligent request scheduling
- 27. Segment Index 27
- 28. Media Segments • Contains the actual segmented media streams • additional information to map segment into media presentation timeline for switching and synchronous presentation with other Representations • For ISO BMFF, contains one or more movie fragments • Can be short (≈1-10 sec) and long (≈10sec – 2h) Segment Advantages Disadvantages duration • Suitable for live • Large number of files • for VoD commonality with live • Large number of URLs Short • High switching granularity on • Fixed request size segment level • switching granularity on segment level • Small number of files • Need for Segment Index • Small number of URLs • Difference from Live Long • High switching granularity • Flexible request sizes • Improved cache performance 28
- 29. Segment and Subsegment Index for MPEG-2 TS 29
- 30. DASH PROFILES 30
- 31. Profiles • Set of restrictions on the offered Media Presentation (MPD & Segments) • can also be understood as permission for DASH clients that only implement the features required by the profile to process the Media Presentation • Profiles defined in ISO/IEC 23009 (as below). More restrictions may be added Full Profile ISO Base media file main MPEG-2 ISO Base ISO Base MPEG-2 TS media file media file TS main format On format simple Demand Live 31
- 32. ISO Base media file format On Demand • intended to provide basic support for On-Demand content. • support for large VoD libraries with minimum amount of content management • permits scalable and efficient use of HTTP servers and simplifies seamless switching. • primary constraints imposed by this profile are – the requirement that each Representation is provided as a single Segment, – Subsegments are aligned across Representations within an Adaptation Set and – Subsegments must begin with Stream Access Points. • Main supporters: Netflix, Qualcomm and few others 32
- 33. Profile identifier: ISO BMFF On-demand Two base URLs Two Audio tracks One Subtitle track One video track Content Protection Five video Representations 33
- 34. ISO Base media file format Live • optimized for live encoding and low latency delivery of Segments consisting of a single movie fragment of ISO file format with relatively short duration. • Each movie fragment may be requested when available using a template generated URL, so it is not normally necessary to request an MPD update prior to each Segment request. • Segments are constrained so that they may be concatenated on Segment boundaries, and decrypted without gaps or overlaps in the media data, regardless of adaptive switching of Representations in an Adaptation Set. • Despite the profile is optimized for live services, the MPD@Type attribute may be set to 'static' to distribute non-live content, for example in case a live Media Presentation is terminated, but kept available as On-Demand service. 34
- 35. Profile identifier: ISO BMFF live Two base URLs Two Audio tracks One video track Closed-GOP SegmentTemplate and Timeline 35
- 36. MPEG-2 TS Profiles • Main: – imposes little constraint on the Media Segment format for MPEG-2 Transport Stream content. • multiplexed Representations (no late binding required) • segment formats are chopped at MPEG-2 TS packet boundaries – HLS content can be integrated with this profile – Indexing and segment alignment is recommended • Simple: – a subset of MPEG-2 TS main profile. – poses more restrictions on content encoding and multiplexing in order to allow simple implementation of seamless switching. – achieved by guaranteeing that a media engine conforming to ISO/IEC 13818-1 can play any bitstream generated by concatenation of consecutive segments from any Representation within the same Adaptation Set. 36
- 37. MPEG-2 TS simple Component Description Template for different segments 37
- 38. Summary: DASH Selected Feature List • Live, On-Demand and Time-shift services • Independency of request size and segment size (byte range requests) • Segment formats – ISO base media FF and MPEG-2 TS – guidelines for integrating any other format – Are codec independent • Support for server and client-side component synchronization (e.g., separate and multiplexed audio and video) • Support for efficient trick mode • Simple splicing and (targeted) ad insertion • Multiple base URLs for the same content • Clock drift control for live sessions • DASH metrics for reporting the session experience • Profile: restriction of DASH and system features (claim & permission) • Content Descriptors for Protection, Accessibility, Rating, etc. – Enables common encryption, but different DRM (DECE-like) 38
- 39. DASH 23009-2 TEST VECTORS AND CONFORMANCE SOFTWARE
- 40. Reference Software • MPEG requires reference/conformance software • Very little is done until now • Objective is: – provide a significant set of good test vectors for positive and negative tests, especially for different profiles – provide conformance software that permits to check compliance of provided Media Presentations • Significant work is necessary between now and the meeting in Geneva end of November
- 41. DASH RELATIONSHIP TO 3GPP TS26.247
- 42. History • Completed Release-9 TS26.234 AHS basis for MPEG DASH – Initially MPEG-DASH attempted to be backward-compatible to Release-9 AHS, but deemed infeasible • 3GPP initiated Rel-10 Work Item on DASH – 3GPP DASH is available in TS26.247 – Close collaboration between MPEG and 3GPP, – always one group was used to define the next step • TS26.247 is more and less – More: • It also defines codecs, protocols metric reporting, etc. • It does have some small additional features (MPD delta) – Less: • it does not use all MPEG DASH features • it does not define profiles or basically is a single profile.
- 43. Remaining Work for 3GPP • Rel-9 AHS is kept incompatible with Rel-10 on the MPD level. – Requires some thoughts on how exactly to separate – Segment formats for Rel-9 and Rel-10 can be used jointly • TS26.247 is currently not compatible to MPEG DASH – purely historical and procedural and timing reasons – Work about to completed to ensure compatibility (this week) – feature and name-space separation – definition as a profile • Rel-11 discussion on integration into 3GPP architecture – support for QoS for DASH users – Using eMBMS to distribute DASH content efficiently – study item to be launched to make DASH even more carrier-friendly
- 45. Typical Uses Cases • MPEG-DASH supports simple and advanced use cases: – On-Demand, Live and time-shift (nPVR) streaming – Dynamic ad-insertion – Dynamic update of program – Delivery of same content on three screens – Delivery of any multimedia content (2D, 3D, animation, graphics, multiview, subtitles, text, etc.), not just AV – Support of multiple languages and different audio configuration – etc. • Simple use cases can be gradually extended to more complex and advanced ones
- 46. Migration Scenarios • Most generated content/production equipment for legacy Adaptive Bitrate Streaming systems can be used for MPEG-DASH: – generic encoders can be reused, DASH adds descriptive metadata for better client operations – Apple HLS Content suitable for DASH M2TS Main profile. – Smooth Streaming Content suitable for DASH ISOBMFF Live profile. • Manifest files can be easily converted to MPD format – XML conversion from m3u8 and Smooth Streaming manifests. – Deployment of two manifest files (legacy and DASH MPD) in parallel (low overhead) • Documentation in preparation … • It’s should not be a competition 46
- 47. DASH Promoters Group • Informal collaboration of companies that have committed or at least significant interest to deploy an open standard for HTTP-based streaming based on DASH. • Effort may include recommending profiles, codecs, DRM • Just started to create PR information, white papers, licensing, attend conferences, demos, interoperability efforts and so on • Current Members: Microsoft, Qualcomm, Netflix, Samsung, Ericsson, Akamai, AEG, Harmonics, Digital Rapids • Significant interest from content providers (globally), operators, device manufacturers, etc. • Interested companies are invited to listen in and/or join • http://dashpg.org 47
- 48. Draft DASH264 Profile • MPEG DASH: – ISO base media FF On-Demand and Live profile – potentially addition of efficient low latency profile • Video: H.264/AVC CBP and High Profile • Audio: MPEG AAC, enhanced AAC • Text and Subtitles: tbd • DRM: Common Encryption based on 23001-7 • Other interop points are tbd 48
- 49. HTML-5 and DASH • Continuous discussions in W3C how to integrate Adaptive Streaming into HTML-5 • Possible Architectures under consideration: – programmable approach: Javascript based download of segments that are fed into video tag – add MPD to video tag – add MPD as URL into the browser (like Apple HLS) • Multiple solution may be deployed 49
- 50. Next steps • Complete standardization work – Completion and publication of all specifications – Conformance, interoperability and reference software • Towards deployments – Generate end-to-end system specs based on DASH including codecs, DRM, profiles, etc. (OIPF, 3GPP, HbbTV, HD Forum, etc.) – Generate guidelines, white papers, test content and software – Promotional efforts: Licensing, interoperability, plug-fests, etc. – Combine it with browsers, the web and HTML-5 • Everyone is invited to contribute 50
- 51. Some Research Challenges • Client side implementation – Rate-adaptation for efficiency, high user experience, robustness – seamless switching experience – Usage and optimization of HTTP, TCP and lower layers – Efficient cloud-based trick modes • Content generation: – Switch points vs. efficiency (how frequent and how) – how many and which bit rates to offer – low-latency optimized content provisioning – caching optimized content generation • Media coding , e.g. efficient switch points • Cloud-support – caching (predictive caching, content management) • Transport and delivery (QoS, multicast, etc.) 51
- 52. More Information • Draft Specifications – 14496-12:2008/FDAM-3: http://www.3gpp.org/ftp/Inbox/LSs_from_external_bodies/ISO_IE C_JTC1_SG29_WG11/29n12310.zip – 23001-7: http://www.3gpp.org/ftp/Inbox/LSs_from_external_bodies/ISO_IE C_JTC1_SG29_WG11/29n12313.zip – 23009-1: http://www.3gpp.org/ftp/Inbox/LSs_from_external_bodies/ISO_IE C_JTC1_SG29_WG11/29n12316.zip • More information from Qualcomm including Qualcomm‘s licensing position – http://www.qualcomm.com/blog/2011/08/16/dash-toward- better-mobile-video-user-experience • Several other companies have declared or expressed willingness to declare favorable licensing conditions 52
- 53. Your Questions • Why is the live profile most popular even for on demand deliver? • How does subtitling work (outside or inside container)? • Is there language support available? • What trick modes are supported in live and on demand profiles? • What are the minimal common features supported for different clients (interoperability between Silverlight, Flash, IOS, etc.)? • Are extra server side components needed for the live or on demand profile? • Is there an impact analysis available from encoding to distribution? • What are the best practices around segmentation? • What are the implications of MPEG TS or ISO base media file format? • What is the preferred profile to distribute both live an on demand (presuming broadcasters want one infrastructure meaning one file base and on streaming platform). • Are there show cases and test streams available? • How much is work is it to support more/all profiles? • What is the latency? • What is the target of the bandwidth, are there limitations to take into account? • Scalability is the same for MPEG DASH (cashing and so on) compared to HLS streaming? • What are the security / encryption possibilities? Probably common encryption format is supported? • If there is no encryption is available, how easy is it to copy/download files? 53
- 54. Comments – Questions - Feedback THANK YOU 54