pivot vector space approach in audio-video mixing
- 1. Guided By:- Presented By:- Ms. Savita Vijay Kanika Rathore B.tech (C.S.E) IIIyr BTBTC10168
- 2. Introduction What is the need? Aesthetic Aspects Video Aesthetic Features Audio Aesthetic Features Pivot Representation Advantages Applications Conclusion
- 3. The PIVOT VECTOR SPACE APPROACH in audio mixing is a novel technique that automatically picks the best audio clip to mix with the given video shot. This technique uses pivot vector space mixing framework & High level perceptual descriptors of audio & video characteristics. It uses a Pivot Vector space mapping method that matches video shots with music segments based on aesthetic cinematographic heuristics . This automatic audio-video mixing technique is suited for Home videos.
- 4. Most videos such as movies and sitcoms have several segments devoid of any speech. Adding carefully chosen music to such segments conveys emotions such as joy, tension ,or melancholy. In a typical professional video production, skilled audio- mixing artists aesthetically add appropriate audio to the given video shots. This process is tedious, time-consuming, and expensive. Many home video users would like to make their videos appear like professional productions before they share it with family and friends.
- 5. Movies comprise :- Images Graphic traces Recorded speech , music and noises Sound effects Roles of music in movies :- Setting the scene Adding emotional meaning Serving as a background filler Creating continuity across shots or scenes Emphasizing climaxes
- 6. The table shows Aesthetic Features that correspond in video & music
- 7. Zettl based these proposed mixing rules on the following aspects :- Tonal matching Structural matching Thematic matching Historical-geographical matching
- 8. A set of attributed features required to describe videos. This consists of features which required to describe videos. Light falloff :- refers to the brightness contrast between the light and shadow sides of an object Color features :- it consist of four features saturation hue brightness energy Motion vectors :- To measure the video segments’ motion intensity.
- 9. We obtained the mean and standard deviation for estimating the confidence level of the Video & audio attributed features for any test shot.
- 10. Low level features :- Spectral centroid (brightness):- measure of a sound’s brightness. Zero crossing :- measure of the frequency content of the signal
- 11. Volume (loudness) :- represents the subjective measure , which depends on the human listener’s frequency response.
- 12. Dynamics :- the volume of musical sound related to the music’s loudness or softness. Tempo features :- that makes the music flow unique and differentiates it from other types of audio signal is temporal organization . (beat rate) Perceptual pitch feature :- it has an important role in human hearing, and the auditory system apparently assigns a pitch to any thing that comes to its attention.
- 13. A vector space P acts as a pivot between the audio and video representation. Independent of any media. This space is defined with some aesthetic features in which music M and videos V are mapped.
- 14. We consider how to represent video and audio clips into their aesthetic spaces V or M In the two spaces, a dimension corresponds to an attributed feature, It includes brightness_high , brightness_low , and so on. One video shot is associated with one vector in the V space. Obtaining the values for each dimension resembles handling fuzzy variables
- 15. The aesthetic feature playing the role of a fuzzy variable and the attribute descriptor acting as a fuzzy value which is represented using diagram. The X-axis refers to the actual computed feature value and the Y-axis simultaneously indicates the aesthetic label and the confidence value.
- 16. In the below figure shows that a) Matching between the video L02_30 & the music T01_5 b)Sample frame the video
- 18. Before the development of the PIVOT VECTOR SPACE APPROACH, audio-video mixing process can be carried out only by professional mixing artists. The Pivot vector space approach enables all the home video users and amateur video enthusiasts to give a professional look and feel to their videos. This technique also eliminates the need for professional mixing artists, thereby significantly reducing the cost, time and labour involved.
- 19. A large amount of home video footage is being produced due to products such as Digital video camcorders , Handicams etc. Hence, this technique will be of great use to all the amateur video enthusiasts and home video users
- 20. This is a technique that all amateur and home video artists can use in the creation of video footage that gives a professional look and feel. Since it is fully automatic, the user need not worry about his aesthetic capabilities.
- 21. http://www-mrim.imag.fr/publications/2003/PM001/v_final.pdf http://ieeexplore.ieee.org www.edutalks.org www.scribd.com