Skymedia
Download as PPT, PDF0 likes606 views
The document discusses the Skymedia project which aims to create immersive media experiences using unmanned aerial vehicles (UAVs) and wireless sensor networks (WSNs). The Skymedia platform uses UAVs to capture live HD and 3D video/images and a WSN to provide supplemental sensor data. This data is fused and processed in real-time by the multimedia service platform to provide enriched multimedia content to end users. Key applications discussed are enhancing the experience of events like marathons and concerts through augmented reality and additional data capture.
Skymedia
- 1. M. Neri, A. Campi , R. Suffritti, F. Grimaccia, P. Sinogas, O. Guye, C. Papin, T. Michalareas, L. Gazdag, I, Rakkolainen SKYMEDIA – UAV-BASED CAPTURING OF HD/3D CONTENT WITH WSN AUGMENTATION FOR IMMERSIVE MEDIA EXPERIENCES
- 2. Skymedia goals Aims at exploring, designing, and demonstrating a novel multimedia end-to-end architecture focused on immersive media experiences to audience during live events Multimedia Service Platform real-time HD/3D processing augmented HD/3D video Unmanned Aerial Vehicle (UAV) and Wireless Sensors Network (WSN) are enabling technologies UAVs capture HD and 3D videos and images from the sky Wireless sensors are able to acquires side-information be fused with the main audiovisual contents
- 3. Key technologies Innovative 3D coding methods with MVD (Multi-view plus Depth), to be executed in real-time Advanced transcoding methods that work in real-time to filter and adapt multimedia content to different distribution networks Mid-air screens materializing high resolution screens By means of vaporized water UAV to capture HD video and images from the sky Wireless Sensors to form an Augmentation Network (WSAN) deployed on ground to enrich the multimedia content with side-information To be fused with the main audiovisual content
- 6. UAV-Based Content Capturing Technical challenges an unobtrusive and silent video/image capturing data from a ground sensor network need to be collected and aggregated Progress beyond the state-of-the-art carries one or more cameras Vibrations, 3D/HD real time video images (H.264/AVC) kinds of sensor devices , payload size, limited resources, power consumption
- 7. Wireless Sensor Augmentation Networks and Data Fusion Technical challenges different types of sensors in dynamic environment with different requirements sport games like soccer, rugby, athletics, or motor sports, etc. aggregate data from both ground sensors and airborne sensors With user generated content Progress beyond the state-of-the-art wireless multi-sensor network multi hop network with either a single or a multi sink structure energy management algorithms improved network protocols data fusion development of tools capable of quickly managing the various datasets and of fusing data
- 8. Multimedia Service Platform Technical challenges Provide real–time immersive services to end-users platforms different constraints, heterogeneous input/output computing power, storage, screen resolution, user interface, sensors and actuators Progress beyond the state-of-the-art Collects pre-formatted multimedia data view point selection and stereoscopic video enrichment and personalization adaptation according to device (smartphone, HD/3D TV screen) real-time management of the HD/3D (MPEG4 MVC, JPEG2000) processing in relation with WSAN data with user interactivity to choose a point of view metadata that are displayed on the HD/3D screen Video-on-Demand is a mandatory multimedia service
- 9. Data path for live and VoD services
- 10. Enabled Services (1/2) Real time HD video content captured from the UAV platfor over the live scene Special real-time transcoding methods is needed to enrich and adapt multimedia content focused on bit-rate reduction, format conversion and resolution optimization Telemetry Data remote measurements regarding live event players parameters delivered to a number of different users Location Based Services Ex. touristic information
- 11. Enabled Services (2/2) Augmented Reality Live capturing, time constraints are particularly stringent Processing center should be coupled with efficent data gathering network Immersive framework for mobile applications Spatial immersion: utilizing the knowledge of users Psychological immersion: offering the user an experience that is updated and related to events Sensory immersion: utilizing state of the art available sensors on the handset (cameras, GPS, gyroscopes, microphones, tough UI, etc.)
- 12. Applications and Use Cases Marathon race different kinds of sensors (biosensor) runner’s shoes or suit, and fixed sensors and cameras placed along the marathon path athlete health status, his relative position in the race, shifting from his best previous performance, deviation from the race record and so on Pre-Test Turin Marathon, November 2011 “ Tutta dritta”, April 2012 Final demonstration Turin Marathon, November 2012
- 14. Applications and Use Cases Outdoor music concert The active part of a concert is a small perimeter event with little movement. The audience of a big concert can, however, cover quite large an area. A freely flying UAV gives more flexibility for filming and other sensors than e.g., cameras on static balloons or on moving wire systems, which can move only very limitedly Motorized sport augment the experience of motorized sports fans avoiding them to become bored during the race
- 15. Conclusion Multimedia architecture able to create immersive media experiences www.ict-skymedia.eu Key roles UAV platform, enabling content capturing from the sky WSAN to collect sensor data advanced Multimedia Service Platform able to manage in real-time HD/3D processing The project is still ongoing Main test event planned in Turin Mrathon 2012 Integration process has been started with preliminary encouraging results
- 16. Thanks