WICED 2022

Permanent URI for this collection

Now showing 1 - 1 of 1

Real-Time Music-Driven Movie Design Framework
(The Eurographics Association, 2022) Hofmann, Sarah; Seeger, Maximilian; Rogge-Pott, Henning; Mammen, Sebastian von; Ronfard, Rémi; Wu, Hui-Yin
Cutting to music is a widely used stylistic device in film making. The usual process involves an editor manually adjusting the movie's sequences contingent upon beat or other musical features. But with today's movie productions starting to leverage real-time systems, manual effort can be reduced. Automatic cameras can make decisions on their own according to pre-defined rules, even in real time. In this paper, we present an approach to automatically create a music video. We have realised its implementation as a coding framework integrating with the fmod api and Unreal Engine 4. The framework provides the means to analyze a music stream at runtime and to translate the extracted features into an animation story line, supported by cinematic cutting. We demonstrate its workings by means of an instance of an artistic, music-driven movie.