Automated Audio/Visual Summary

About

Introduction: The video streaming market and the audio production services are valued at $103.567 billion and $1.2 billion, respectively. These streaming services use advertising to generate revenue. Advertisements are frequently edited and recorded in industry operators' studios. Different systems exist for capturing significant parts of a musical or visual work of art, but not many of these systems are capable of identifying specific parts that could be tailored to the audiences’ interests and summarized into short preview clips, like a movie trailer that summarizes a handful of key pieces of the entire movie. Technology: Georgia State University researchers developed a software tool that can be used for visual and audio data summarization. This technology focuses on applying the principles of “Deep Neural Networks” to automate music segmentation. A special time of Recurrent Neural Network (RNN), called Long Short Term Memory (LSTM) is being employed to automatically generate “smart song segments” and then coordinate and merge those segments into a musically sensical and concise summary. This software can potentially be applied to visual works as well with further development.