GPAC Team Research Highlights
1 like•2,584 views
The document presents the activities and research themes of the GPAC team from the Institut Mines-Télécom, focusing on multimedia adaptation, transport, and services. It outlines key research projects, standardization efforts, and the development of the open-source GPAC project, which provides multimedia player and delivery tools. Additionally, it discusses application areas such as interactive multimedia, streaming technologies, and the integration of advanced user interfaces.
GPAC Team Research Highlights
- 1. GPAC Team Séminaire RT4 Cyril Concolato 08/04/2013 Institut Mines-Télécom
- 2. People C. Concolato J.-C. Dufourd J. Le Feuvre J.-C. Moissinac + PhD Students, Post-Docs, S/W Engineers • N. Bouzakaria, R. Bouazizi, V. Gros, M. Sarkis • J. Gorin, A. Shafiei, V. Nguyen • & many former students/engineers Part of the TPT / TSI / Multimedia Group 2 Institut Mines-Télécom
- 3. Activities Current Research Themes • Multimedia Adaptation and Multimodal Interfaces • Multimedia Transport and Delivery • Multimedia Services and Transmedia Teaching • Multimedia Courses (Master, Mastère, Lifelong learning) Our specificities • Open Source Software: GPAC • Standardization: MPEG, W3C, … 3 Institut Mines-Télécom
- 4. Current Research Projects ANR HybRadio FUI Austral FUI 4Ever FUI OptisaT2 FSN M44G ANR ILOT EU COLTRAM EU KIC ICT ANSFM Industrial partnerships 4 Institut Mines-Télécom
- 5. Multimedia Adaptation and Multimodal Interfaces Problems • Adapt the multimedia consumption to the user context (preferences, network, device constraints) • Find representations and architectures for adaptations Latest work (in progress) • Usage of semantic information for adaptation and content generation • V. Gros’s PhD – “Digital Book Publishing” Some other results • W3C recommendation: "Multimodal Architecture and Interfaces" • Multimodality: B. Rodriguez’s PhD. • Adaptation: B. Pellan’s PhD 5 Institut Mines-Télécom
- 6. Multimedia Adaptation and Multimodal Interfaces: Ontologies and Semantic - Interactions Description - Ontologies to describe - How/Where/Which device you interact with, - Tight relationship with W3C MMI group - Description of multimedia adaptation engines 6 Institut Mines-Télécom
- 7. Multimedia Adaptation and Multimodal Interfaces: Adaptation Engine Ontologies WSML example Displayed with Web Services Modeling Toolkit Now: RDF+OWL+OWL-S 7 Institut Mines-Télécom
- 8. Multimedia Transport and Delivery: Context Internet Video Growth • Cisco’s forecast • Netflix Example: 30% of US traffic during peak hours Mobile Video • >= 50% of traffic 8 Institut Mines-Télécom
- 9. Multimedia Transport and Delivery: Requirements Basic constraints are satisfied • Audio/Video Synchronization • Handles Loss and Corruption • Fast start-up • Limited rebuffering • Compatible with Live, VoD, trick modes, … Additional constraints • Adaptive to varying bandwidth conditions (seamlessly, granularity) • IP-compatibility (proxies, firewalls, web servers, caches) • Efficiency 9 Institut Mines-Télécom
- 10. Multimedia Transport and Delivery: « HTTP Streaming » Principles Use a playlist • Continuously download small media files via HTTP ─ Selection btw. alternative files based on bandwidth ─ Constant duration files, cacheable Properties • Time continuity btw. files across quality to guarantee seamless playback and switching • Compatible with live and VoD Some Limitations • Latency for live events • Some overhead (e.g. MPEG-2 TS / HTTP) Proprietary Solutions: Apple, Microsoft, Adobe Standards: MPEG, 3GP, OIPF, W3C, … 10 Institut Mines-Télécom
- 11. Multimedia Transport and Delivery: HTTP Streaming Research Aspects Adaptation strategies and Quality of Experience Cache optimizations Energy efficient streaming (mobile) Low latency live streaming Scalable media streaming Hybrid Broadcast Broadband Multipath streaming … PhD Student: N. Bouzakaria 11 Institut Mines-Télécom
- 12. Multimedia Transport and Delivery: Application Integration How to repurpose broadcast data? How to design HTML5 streaming-aware applications? How to deliver new media data (HEVC, subtitling …)? How to synchronize broadcast/broadband events? Audio Concolato et al., Usages of DASH for rich media services. MMSys Data carousel (500 ms) '11. DOI=10.1145/1943552.1943587 Data carousel (2 s) Live data 12 Institut Mines-Télécom
- 13. Multimedia Services and Transmedia: Research Topics New types of services • Interactive Multimedia ─ Digital TV (HBBTV, …) ─ Digital Radio ─ Web (HTML5, SVG) • Stereoscopic Displays • Second Screens 13 Institut Mines-Télécom
- 14. Multimedia Services and Transmedia: Second Screens User Interfaces are evolving • Integrate advanced rich media ─ 2D/3D, animations, video/audio clips • Possible to have a common user interfaces on many devices ─ TV, Phone, Desktop, Web page Designed around dedicated small applications ⇒ Widgets 14 Institut Mines-Télécom
- 15. Multimedia Services and Transmedia: Transmedia & Widgets Widget Web Server widget 2) Download & activation Internet Home Network 1) Discovery 3) UPnP Control messages 4) UPnP Notification messages DLNA Blu-Ray Player 4) Data exchange DLNA TV 16 Institut Mines-Télécom
- 16. Multimedia Services and Transmedia: Our Approach MPEG-U (Rich Media User Interface) • Lead by Telecom ParisTech ─ Collaboration with Samsung R&D HQ • MPEG Media Type • Simplified representation • Widget Communication Communicating and migratable interactive multimedia documents, MTAP 2011 COLTRAM EU Project • Collaboration with Fraunhofer FOKUS • Use of Web Technologies ─ PhD Student: R. Bouazizi • Authoring of Migratable Web Apps ─ PhD Student: M. Sarkis 17 Institut Mines-Télécom
- 17. Standardization activities MPEG • Since 1996 • Mainly MPEG Systems activities and standards ─ Interactivity: MPEG-4 BIFS, MPEG-4 LASeR ─ File Format: MP4, AVC, SVC, HEVC ─ Streaming: MPEG-2 TS, MPEG DASH • Editors of several standards (+ conformance sequences, reference software, utility software) W3C • Since 2004 • Working Groups: SVG, HTML, MMI, WebApps, … 18 Institut Mines-Télécom
- 18. The GPAC Open Source Project GPAC (http://gpac.wp.institut-telecom.fr) • Multimedia player, encoder, and delivery tools • Distributed Open Source under LGPL License Goals • Platform for research and academic purposes • Dissemination of multimedia standards Development Context • Within our research activities • In sync with industry standards (MPEG, W3C, DVB, ...) • Using academic resources (student projects, internships) 19 Institut Mines-Télécom
- 19. The GPAC Open Source Project: Highlights 13 years-old project! ~ 500 000 lines of code • Cross-platform, ~ANSI C • CPU architectures: ─ x86, ppc, arm, arc, sh4 / 32-/64-bits • OS: ─ Windows (desktop and mobile), Linux, Mac OS X, various UNIXes (BeOS, Solaris, …), Symbian, iOS, Android. Good visibility/community • 5000 downloads per month • Research community ─ 2nd Best OSS @ ACM Multimedia 2007 • End-User community ─ Mostly uses MP4Box 20 Institut Mines-Télécom
- 20. The GPAC Open Source Project: MP4Box Swiss-army knife for the ISO Base Media File Format and derivatives: • MP4,3GP,OMA,MJ2K,MP21, PSP... • Major MP4 packager on the web Media track manipulations • Importers and exporters ─ audio, video, subtitling in different formats • Grouping, disabling, tagging ... Delivery preparation: offline/live • Interleaving for HTTP download • RTP Hinting for 3GP/OMA streaming servers • DASH (segmentation, MPD generation) • MPEG-2 TS generation Encryption/Decryption File analysis Scene Description encoding 21 Institut Mines-Télécom
- 21. The GPAC Open Source Project: Basic Playback Features Classic Audio/Video player • Similar to Quicktime, VLC • Integrated with all browsers (IE, Safari*, Chrome, Firefox, Opera) • Available with different UI: command-line, GUI (Windows, SDL, Qt, ...) Various media codecs • Including HEVC, H.264|AVC, SVC, AAC … • Based on third party libraries Various network stacks • Web protocols: HTTP, IceCast, DASH • Streaming protocols: RTP, RTSP • File Formats: MP4, MPEG-2 TS... • Home Networking protocols: UPnP/DLNA, ... Extensible through plug-in architecture 22 Institut Mines-Télécom
- 22. The GPAC Open Source Project: Advanced Playback Features Enhanced Multimedia Player • Very similar to a Web Browser • Support for several document formats Synthetic Graphics • 2D: SVG, MPEG-4 BIFS, Flash • 3D: VRML, X3D • With some acceleration (OpenGL / OpenGL ES / TinyGL) Advanced Composition • Mixing of languages • Animation & Synchronization • Composition & Synchronization of multiple media elements Interactivity • Support for JavaScript, including AJAX 23 Institut Mines-Télécom
- 23. Merci! 24 Institut Mines-Télécom
Editor's Notes
- #22: Command line tool GUI exists outside of the project (YAMB, MyMP4BoxGUI)