Keyframe-based Video Summarization Designer
1 like1,194 views
The document outlines a thesis on improving keyframe-based video summarization through the Designer Master interface, focusing on methodologies for shot segmentation and keyframe extraction enhancements. It presents results from user tests that assess the quality and representativeness of the generated summaries, demonstrating an improvement in the application. The aim is to develop this work for potential use by the Austrian Broadcasting Station ORF.
![State of the art
Shot segmentation
10
Hierarchical decomposition and representation of video content [1]
[1] http://www.scholarpedia.org/article/Video_Content_Structuring](https://image.slidesharecdn.com/cramospresentationv2-150721131122-lva1-app6892/85/Keyframe-based-Video-Summarization-Designer-10-320.jpg)
![State of the art
Shot segmentation example
11
Shot boundary detection example [2].
[2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).](https://image.slidesharecdn.com/cramospresentationv2-150721131122-lva1-app6892/85/Keyframe-based-Video-Summarization-Designer-11-320.jpg)
![Methodology : Implemented solution
Decision making
The threshold value used in our application is 𝜏 = 0.1 (as defined in [2]).
19
[2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).](https://image.slidesharecdn.com/cramospresentationv2-150721131122-lva1-app6892/85/Keyframe-based-Video-Summarization-Designer-19-320.jpg)
Keyframe-based Video Summarization Designer
- 1. KEYFRAME-BASED VIDEO SUMMARIZATION DESIGNER Carlos Ramos Caballero Advisors: Horst Eidenberger and Xavier Giró I Nieto
- 2. Contents Introduction State of the art Methodology Results assessment Conclusions 2
- 3. The application: Designer Master DEMONSTRATION 3
- 4. Contents Introduction State of the art Methodology Results assessment Conclusions 4
- 5. Introduction Motivation Designer Master: keyframe-based video summarization interface Object Maps: system for automatic video summarization 5 Graphical User Interface (Designer Master) Computer Vision Engine (Object Maps)
- 6. Introduction Goals of the thesis 6
- 7. Introduction Goals of the thesis Improving the keyframe extraction module 7
- 8. Introduction Goals of the thesis Improving the keyframe extraction module Assessing the improvement 8
- 9. Contents Introduction State of the art Methodology Results assessment Conclusions 9
- 10. State of the art Shot segmentation 10 Hierarchical decomposition and representation of video content [1] [1] http://www.scholarpedia.org/article/Video_Content_Structuring
- 11. State of the art Shot segmentation example 11 Shot boundary detection example [2]. [2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).
- 12. State of the art Shot segmentation techniques Pixel-to-pixel methods • Global pixel-to-pixel • Cumulative pixel-to-pixel Histogram-based methods • Simple histogram • Maximum histogram • Weighted histogram Hausdorff method 12
- 13. Contents Introduction State of the art Methodology Results assessment Conclusions 13
- 14. Methodology: Implemented solution System architecture overview 14
- 15. Methodology : Implemented solution Uniform sampling 𝑓𝑝𝑠𝑖: frame rate of the input video. 𝐿𝑖: total number of frames of the input video. 𝑁0: total number of frames we want to keep (𝑁0=100). 15
- 16. Methodology : Implemented solution Gray scale domain 16 Color model transformation RGB to YIQ.
- 17. Methodology : Implemented solution Difference computation Where 𝐼(𝑡,𝑖,𝑗) represents the intensity value at frame t in pixel(𝑖,𝑗). X and Y are the width and height of the video frames, respectively. 17
- 18. Methodology : Implemented solution Normalization Where 𝑑̂ is the normalized value, 256 is the number of grey levels, X and Y are the width and height of the video frames, respectively. 18
- 19. Methodology : Implemented solution Decision making The threshold value used in our application is 𝜏 = 0.1 (as defined in [2]). 19 [2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).
- 21. Contents Introduction State of the art Methodology Results assessment Conclusions 21
- 22. Results assessment TEST 1: Testing the applications + ‘in situ’ survey 11 participants Test data: The intouchables trailer 22
- 23. Results assessment Example: pair of summaries 23 Designer Master v1 Designer Master v2
- 24. Results assessment TEST 2: web-based survey 43 participants Test data: The Intouchables trailer 24
- 25. Results assessment EVALUATION Quality of the generated summaries Representativeness of the generated summaries Mean Opinion Score • 1. Unacceptable • 2. Poor • 3. Good • 4. Very good • 5. Excellent 25
- 26. Results assessment Quality generated summaries “Please, rate summary 1” 26 “Please, rate summary 2”
- 27. Results assessment Quality generated summaries 27 MOS MOS – scores distribution
- 28. Results assessment Representativeness of the summaries “Which summary let you better recognize the video content?” 28
- 29. Results assessment Representativeness of the summaries 29
- 30. Results assessment Ease-of-use of the application “Do you think the application is intuitive and easy to use?” 30
- 31. Results assessment Ease-of-use of the application 31
- 32. Results assessment Execution time 32
- 33. Contents Introduction State of the art Methodology Results assessment Conclusions 33
- 34. Conclusions Accomplishment of the initial goals Improving the keyframe extraction module by integrating both projects. Assessing the improvement. 34
- 35. Conclusions Accomplishment of the initial goals Improving the keyframe extraction module by integrating both projects. Assessing the improvement. Our work has slightly improved Designer Master Users can create better video summaries and easily due the better quality of the extracted keyframes. 35
- 36. Conclusions Accomplishment of the initial goals Improving the keyframe extraction module by integrating both projects. Assessing the improvement. Our work has slightly improved Designer Master Users can create better video summaries and easily due the better quality of the extracted keyframes. It is hoped to develop this work into a product for the Austrian Broadcasting station ORF 36
- 37. Conclusions Accomplishment of the initial goals Improving the keyframe extraction module by integrating both projects. Assessing the improvement. Our work has slightly improved Designer Master Users can create better video summaries and easily due the better quality of the extracted keyframes. It is hoped to develop this work into a product for the Austrian Broadcasting station ORF 37
- 38. Thank you very much for your attention! Danke schön! Moltes gràcies! 38