SUMMARY GENERATION FOR LECTURING VIDEOS
0 likes20 views
The document discusses generating summaries for lecturing videos. It proposes a method that uses optical character recognition to extract textual information from each video frame, detects changes in text between frames to find scene changes, and combines key frames to create a highlight summary. The proposed system was tested on PowerPoint-based lecture videos. Future work could focus on expanding it to handle chalkboard-style lectures through improved machine learning models. The goal is to automatically generate concise summaries that extract the most important concepts from long videos to help students learn more efficiently.
![© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2216
SUMMARY GENERATION FOR LECTURING VIDEOS
R Anish1, Shashank P3, Suraiya Anjum3, Sushma K4, Prof. Puneeth P5
*1,2,3,4,5 Department of Information Science and Engineering, Maharaja Institute of Technology,
Mysore, Karnataka, India
---------------------------------------------------------------------***---------------------------------------------------------------------
ABSTRACT
Online lectures and online courses share the same mass conceptual content all stored into one large video. As the world is
progressing towards advancement in the technology so is the production of high volume, high density data. Many universities
adapted to e-learning due to the pandemicand not all will access these videos multipletimestounderstandtheconceptsasthey
are long. Thus the extraction of important and useful topics from the lecturing videos is the area which hasn't been explored in
great yet. Particular area is having a huge potential of research and implementationasfarastherealapplicationsareconcerned.
Video highlights or synopsis is the abstraction of the main events in video or image collection. It is used in order to highlightthe
entire video to easily interpret what it is being tried to conclude. With this highlighting process we can easily understand and
revise on concepts that are importantand parts of videos containing interest. highlight videos of importantconceptswhichwill
ease the process of revision and learning. Content highlights facilitates us in simplifying the learning process. One of our main
objectives is to save time and access the important topics which is required by the viewer as fast as possible.
Keywords: Yake, EasyOCR, frame selection, text detection.
I. INTRODUCTION
The internet is flooded with an enormous amount of videos and textscan quickly scan the text and see if thereisanycontent
in the video. Summary versions of the videos will be a life saving asset. Recorded videos of lectures are gaining popularity as a
basic tool for distance education as well as a supplementary tool for face-to-face education. Students get information from
videos, but the timecost of going through these videos especially forlong lecturevideoswill be high, so to solve this, we needto
automatically capture the gist and essential topics in the videos, the video summary meets this requirement. Video
summarization is defined as the process of generating a summary of along video by selecting the most informative fortheuser.
This thesis emphasizes the survey for generating lecture summaries where weuse CV2 for video to image conversion,easyOCR
for text detection, merging and generating video summary with text generation.
II. LITERATURE REVIEW
[1] In this paper a framework for automatic summarization of videos. The SumBot framework is specially designed for
scenarios where the summarization process follows a semi-structured editing template.
[2] In this paper the anchor-based DSNet approach formulates the video summary as a focus detection problem and the
importance score and position from the generated interest suggestions
[3] In this paper an incremental framework for subset selection. At each point, it updates the set of representatives with
the previously selected set of representatives and the new data stack.
[4] In this paper A PCDL framework for video summarization tasks that generate video summaries using a dual learning
framework and constraints on summarization properties.
[5] In this paper A novel approach to a deep video summary called AD Sum is used to generate the summary.
[6] In this paper, automatic cricket video highlights will be generated by considering some of the criteria like scores,
audience voice and change in the score.
[7] In this paper The static and motion similarity scores of clips with the appropriate adaptation thresholds are used to
merge consistent clips. Use local static and motion similarities to adjust the boundaries between clips.
[8] In this paper A scene change detection algorithm based on position analysis has been proposed for frame rate up-
conversion. The proposed algorithm calculates statistics after generating a 2D histogram to extract the shape of the
histogram.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072](https://image.slidesharecdn.com/irjet-v9i7408-221022113501-1d171c78/85/SUMMARY-GENERATION-FOR-LECTURING-VIDEOS-1-320.jpg)
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2222
X. REFERENCES
[1] “SumBot: Summarize Videos Like a Human” by Hongxiang Gu, Stefano Petrangeli and Viswanathan Swaminathan in
2020.
[2] “DSNet: A Flexible Detect-to-Summarize Network for Video Summarization” by Wencheng Zhu, Jiwen Lu, Jiahao Li and
Jie Zhou in 2021
[3] “Online Summarization via Submodular and Convex Optimization” by EhsanElhamifarandM.Clara DePaolisKaluza in
2017
[4] “ Property-Constrained Dual Learning for Video Summarization “ by Bin Zhao, Xuelong Li and Xiaoqiang Lu in 2019.
[5] “Deep Attentive Video Summarization With Distribution Consistency Learning” by Zhong Ji, Yuxiao Zhao, YanweiPang,
Xi Li and Jungong Han in 2020.
[6] “A Multimodal approach for automatic cricket video summarization” by Aman Bhalla , Arpit Ahuja , Pradeep Pant and
Ankush Mittal in 2019.
[7] “ Unsupervised Video Summarization based on consistent clip generation” By Xin Ai, Yan Song , Zechao Li in 2018.
[8] “Positional analysis-based scene change detection algorithm” by Suk-Ju-Kang in 2015.
[9] “ Video Summarization by learning from unpaired data “ by Mrigank Rochan and Yang Wang in 2020.
[10] “Video Summarization by learning deep side semantic embedding” By Yitian yuan, Tao Mei,PengcuiandWenwuZhu
in 2017.
[11] “Unsupervised Video Summarization Framework using Key-Frame Extraction and Video Skimming” by Shruti Fadon
and Mahmood Jasim in 2020.
[12] “A New Approach to Extracting Sports Highlights ” by Pichet Suksai and Paruj Ratanworabhan in 2016.
[13] “An Efficient Framework for Automatic Highlights Generation from Sport Videos” by Ali Javeed, Khalid BhasirBajura,
Hafiz Malik, Aun Irtaza in 2016.
[14] “Video Summarization via Action Ranking “ by Mohammed Elfek and Ali Baj in 2019.
[15] “Cloud-Assisted Multi-View Video Summarization Using CNN and Bi-LSTM ” by Tanvir Hussain, Khan Mohammed,
Amin Ullah, Zehong Cao, Sungwook Baik and Victor Hugo De Alborquerque in 2019. Summary GenerationforLecturing
Videos 2021-22 Department of ISE, MIT Mysore 23.
[16] “Automatic Tour Video Summarization FocusingonSceneChangeforAdvanceTouristicExperience”bye YukiKanaya,
Shogo Kawanaka, Hirohiko Suwa Yutaka Arakawa and Keiichi Yasumoto in 2019.
[17] “Hybrid Approach for Video Compression Based on Scene Change Detection” by Ankita P. Chauhan, Rohit R. Parmar ,
Shankar K. Parmar , Shahida G. Chauhan in 2013.
[18] “A Novel Key-frames Selection Framework for Comprehensive Video Summarization” by Cheng Huang and Hongmei
Wang in 2018.
[19] “User-Ranking Video Summarization withMulti-Stage Spatio-Temporal Representation”bySiyuHuang,XiLi,Zhongfei
Zhang, Fei Wu, and Junwei Han in 2018.
[20] “Meta Learning for Task-Driven Video Summarization” by Xuelong Li, Fellow, IEEE, Hongli Li, and YongshengDongin
2019.](https://image.slidesharecdn.com/irjet-v9i7408-221022113501-1d171c78/85/SUMMARY-GENERATION-FOR-LECTURING-VIDEOS-7-320.jpg)
SUMMARY GENERATION FOR LECTURING VIDEOS
- 1. © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2216 SUMMARY GENERATION FOR LECTURING VIDEOS R Anish1, Shashank P3, Suraiya Anjum3, Sushma K4, Prof. Puneeth P5 *1,2,3,4,5 Department of Information Science and Engineering, Maharaja Institute of Technology, Mysore, Karnataka, India ---------------------------------------------------------------------***--------------------------------------------------------------------- ABSTRACT Online lectures and online courses share the same mass conceptual content all stored into one large video. As the world is progressing towards advancement in the technology so is the production of high volume, high density data. Many universities adapted to e-learning due to the pandemicand not all will access these videos multipletimestounderstandtheconceptsasthey are long. Thus the extraction of important and useful topics from the lecturing videos is the area which hasn't been explored in great yet. Particular area is having a huge potential of research and implementationasfarastherealapplicationsareconcerned. Video highlights or synopsis is the abstraction of the main events in video or image collection. It is used in order to highlightthe entire video to easily interpret what it is being tried to conclude. With this highlighting process we can easily understand and revise on concepts that are importantand parts of videos containing interest. highlight videos of importantconceptswhichwill ease the process of revision and learning. Content highlights facilitates us in simplifying the learning process. One of our main objectives is to save time and access the important topics which is required by the viewer as fast as possible. Keywords: Yake, EasyOCR, frame selection, text detection. I. INTRODUCTION The internet is flooded with an enormous amount of videos and textscan quickly scan the text and see if thereisanycontent in the video. Summary versions of the videos will be a life saving asset. Recorded videos of lectures are gaining popularity as a basic tool for distance education as well as a supplementary tool for face-to-face education. Students get information from videos, but the timecost of going through these videos especially forlong lecturevideoswill be high, so to solve this, we needto automatically capture the gist and essential topics in the videos, the video summary meets this requirement. Video summarization is defined as the process of generating a summary of along video by selecting the most informative fortheuser. This thesis emphasizes the survey for generating lecture summaries where weuse CV2 for video to image conversion,easyOCR for text detection, merging and generating video summary with text generation. II. LITERATURE REVIEW [1] In this paper a framework for automatic summarization of videos. The SumBot framework is specially designed for scenarios where the summarization process follows a semi-structured editing template. [2] In this paper the anchor-based DSNet approach formulates the video summary as a focus detection problem and the importance score and position from the generated interest suggestions [3] In this paper an incremental framework for subset selection. At each point, it updates the set of representatives with the previously selected set of representatives and the new data stack. [4] In this paper A PCDL framework for video summarization tasks that generate video summaries using a dual learning framework and constraints on summarization properties. [5] In this paper A novel approach to a deep video summary called AD Sum is used to generate the summary. [6] In this paper, automatic cricket video highlights will be generated by considering some of the criteria like scores, audience voice and change in the score. [7] In this paper The static and motion similarity scores of clips with the appropriate adaptation thresholds are used to merge consistent clips. Use local static and motion similarities to adjust the boundaries between clips. [8] In this paper A scene change detection algorithm based on position analysis has been proposed for frame rate up- conversion. The proposed algorithm calculates statistics after generating a 2D histogram to extract the shape of the histogram. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2217 III. SURVEY FINDINGS Generating short summaries or highlights of recorded video content is an essential task not only for publishing content on video sharing platforms, but also for video asset management.Thealgorithmsusedarea videosummarycreatedwithunpaired data and a deep learning framework with unpaired data. Video summaries are intendedtocreatea concisesummarytoextract the most useful parts of the video. This is essential for humans to effectively and efficiently search and understand large amounts of video data in a user-friendly way. This is usually formulated as a supervised learning problem that learns a spatiotemporal mapping function for selecting keyframes or subframes from a video sequence. IV. METHODOLOGY Our algorithm uses the textual information for extraction method. The textual information which is extracted from each frame. First it recognizes the title in the slide and convert the text of the title into a sentence based on the algorithms like OCR and CNN. When textual information is available, title differences are recognizedandinformationabouttopicsthathavechanged for each topic is provided. This forms the basis for detecting changes in the scene. This captures the frames where the scene changes occur and combines them to create highlights. OCR(Optical character recognition) converts the digital image into a machine-coded text electronically. Here, the digital imageisgenerallyanimageincludingaregionsimilartothecharactersofthe language. OCR can be used in artificial intelligence, pattern recognition and computer vision. This is because the new OCR is trained by providing sample data that is executed via machine learning algorithms. This technique of extracting text from an image is usually done in a work environment where you are certain that the image contains text data. Figure 1: System Architecture.
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2218 Figure 2 : Video Snapshot Figure 3 : Video Snapshot.
- 4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2219 V. DATA FLOW DIAGRAMS Figure 4: Dataflow diagram to novel approach of summary generation of lecturing videos
- 5. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2220 VI.USE CASE MODEL Figure 5: Use Case Model VII. RESULTS AND DISCUSSION Here we tried to generate the summary for PowerPoint-based lecturing videos which is very much beneficial for students. Where the user needs to upload a lecturing video using the user interface provided, once the video is uploaded successfully, a summary video with text is generated. Once the video is generated the user should copy the link and pasteitintothebrowserto download the summarized video. Figure 3: User Interface.
- 6. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2221 Figure 4: Summary Video with Text Generation Figure 5: Accuracy Analysis between non-educational and educational videos VII. CONCLUSION The main aim of this project is a summary generationforlecturingvideosVideoSummarizationorsynopsisistheabstraction of the main events in video orimage collection.it is used tosummarize the entire lecturing videoandprovideonlytheimportant concepts that is being covered in that session. With this summarization process, we can easily understand and revise concepts that are important and parts of videos containing interest. The proposed system will generate highlights of PowerPoint presentation videos and blackboard taught videos by extracting the text from the images/frames of the videos and identifying the change in textual information between frames. Some importantconceptsmayormaynotbeidentifiedproperlyinthecaseof poor video/image quality. IX. FUTURE WORK In our proposed system weare implementing it only forthe PowerPoint-based lecturing videos. Inviewoffutureenhancement, wetry to implement the video summarization technique forchalk and board lecturing videos in which the machineneeds to be trained in a very efficient manner to provide the expected output.
- 7. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 2222 X. REFERENCES [1] “SumBot: Summarize Videos Like a Human” by Hongxiang Gu, Stefano Petrangeli and Viswanathan Swaminathan in 2020. [2] “DSNet: A Flexible Detect-to-Summarize Network for Video Summarization” by Wencheng Zhu, Jiwen Lu, Jiahao Li and Jie Zhou in 2021 [3] “Online Summarization via Submodular and Convex Optimization” by EhsanElhamifarandM.Clara DePaolisKaluza in 2017 [4] “ Property-Constrained Dual Learning for Video Summarization “ by Bin Zhao, Xuelong Li and Xiaoqiang Lu in 2019. [5] “Deep Attentive Video Summarization With Distribution Consistency Learning” by Zhong Ji, Yuxiao Zhao, YanweiPang, Xi Li and Jungong Han in 2020. [6] “A Multimodal approach for automatic cricket video summarization” by Aman Bhalla , Arpit Ahuja , Pradeep Pant and Ankush Mittal in 2019. [7] “ Unsupervised Video Summarization based on consistent clip generation” By Xin Ai, Yan Song , Zechao Li in 2018. [8] “Positional analysis-based scene change detection algorithm” by Suk-Ju-Kang in 2015. [9] “ Video Summarization by learning from unpaired data “ by Mrigank Rochan and Yang Wang in 2020. [10] “Video Summarization by learning deep side semantic embedding” By Yitian yuan, Tao Mei,PengcuiandWenwuZhu in 2017. [11] “Unsupervised Video Summarization Framework using Key-Frame Extraction and Video Skimming” by Shruti Fadon and Mahmood Jasim in 2020. [12] “A New Approach to Extracting Sports Highlights ” by Pichet Suksai and Paruj Ratanworabhan in 2016. [13] “An Efficient Framework for Automatic Highlights Generation from Sport Videos” by Ali Javeed, Khalid BhasirBajura, Hafiz Malik, Aun Irtaza in 2016. [14] “Video Summarization via Action Ranking “ by Mohammed Elfek and Ali Baj in 2019. [15] “Cloud-Assisted Multi-View Video Summarization Using CNN and Bi-LSTM ” by Tanvir Hussain, Khan Mohammed, Amin Ullah, Zehong Cao, Sungwook Baik and Victor Hugo De Alborquerque in 2019. Summary GenerationforLecturing Videos 2021-22 Department of ISE, MIT Mysore 23. [16] “Automatic Tour Video Summarization FocusingonSceneChangeforAdvanceTouristicExperience”bye YukiKanaya, Shogo Kawanaka, Hirohiko Suwa Yutaka Arakawa and Keiichi Yasumoto in 2019. [17] “Hybrid Approach for Video Compression Based on Scene Change Detection” by Ankita P. Chauhan, Rohit R. Parmar , Shankar K. Parmar , Shahida G. Chauhan in 2013. [18] “A Novel Key-frames Selection Framework for Comprehensive Video Summarization” by Cheng Huang and Hongmei Wang in 2018. [19] “User-Ranking Video Summarization withMulti-Stage Spatio-Temporal Representation”bySiyuHuang,XiLi,Zhongfei Zhang, Fei Wu, and Junwei Han in 2018. [20] “Meta Learning for Task-Driven Video Summarization” by Xuelong Li, Fellow, IEEE, Hongli Li, and YongshengDongin 2019.