Design and Implementation of a plug-in for video-to-audio mapping

As a fast-growing discipline, artificial intelligence has been applied to many fields, especially contributing to product design. How to make artificial intelligence technology enhance human creativity is not only the trend of field development, but also the motivation of the SCORE! project. In this project, we designed and developed a plug-in for electronic music production embedding deep learning method for video-to-audio mapping. And study how this method can be integrated in a specific electronic music production application to assist music creation. Through the survey questionnaire, we obtained the user requirements and preferences for the development of the audio plug-in. After the design and implementation of SCORE! plug-in, we conducted a user study with experts in field of electronic music production and collected feedback with a questionnaire. In the evaluation results, the user found the SCORE! plug-in is a creative support audio plug-in that provides an efficient workflow of video selection and previewing, MIDI clips generation, MIDI clips importing for music production and synthesizer.