Mozart, Madonna or Michael Jackson ?? the human ear recognizes differences between pieces and styles of music very quickly. A computer or Artificial Intelligence (AI) is also able to do this – but not acoustically, but optically. This shows the project? Mozart Jukebox ?? the University of Applied Sciences Würzburg-Schweinfurt (FHWS). A second project, the augmented reality app, integrates the individual projects of the Mozart anniversary year.
Both FHWS projects are integrated into two current concepts: On the one hand, Würzburg is celebrating the 100th anniversary of the Mozart Festival, in which the university project “Mozart & KI” is part of a quartet. On the other hand, scientists from Würzburg and Weimar have been going since 2020 the question of how Mozart’s musical creativity can be combined with algorithms. According to their press release, the Julius Maximilians University of Würzburg, the Steinbeis Institute New Media and Data Science (Prof. Dr. Frank Deinzer , FHWS) as well as a freelance composer from Weimar.
Software can differentiate 484 pieces of music
Christoph Raab, employee at the Digital Engineering Institute at the Faculty of Computer Science and Business Informatics at FHWS, has the? Mozart-Jukebox ?? Developed together with students with the aim of visually demonstrating the learning process of artificial intelligence. The approach: So-called acoustic frequency representations are used and these are represented as a spectrogram. The software of the jukebox has been ?? trained ?? and can differentiate 484 pieces of music by 15 composers and musicians. The AI was programmed in such a way that it represents its interpretation of the music in three dimensions and is thus visible to the human eye.
The project pursues two approaches ?? those of the monitored and unsupervised AI. What they have in common is that they try to summarize pieces of music in a meaningful way. While the monitored AI had access to artist information, the unsupervised variant focused on category assignment.
Works much like a brain
“The FHWS jukebox”, as Raab explains, “makes it possible to understand the perspective of an artificial intelligence (AI) on music. The technology used here is called Convolutional Neural Network, or CNN for short: people perceive the environment with their eyes. This perception is sent from the eyes to the brain and finally processed. However, a CNN is a program and therefore of course has no eyes. In principle, however, it works in a similar way to the brain: the person gives the CNN images for processing and finally reports the result back to the person.
Christoph Raab shows the relevance of the results: ?? For this project, the AI was programmed in such a way that a person can see the classification of songs by artists with the naked eye. It is thereby possible to see how well or how badly she is doing it. The result is heavily dependent on the programming and the settings that the person makes. Interested parties can try out the jukebox (https://mozart.fiw.fhws.de/) themselves and make different filters and selections, also via the “Mozart and More” app in the app and play store.
App integrates individual projects
The augmented reality app was developed by Prof. Dr. Frank Deinzer (FHWS) at the Steinbeis Institute New Media and Data Science as part of a technology transfer or a spin-off. The app integrates individual projects of the overall concept ?? Mozart and KI ?? and makes history through eg posters, photos, videos and 3D objects with additional information and a content management system “alive”. On the basis of augmented reality, for example, posters can be provided with additional information via an app and a content management system. These are able to recognize the content of the poster and to virtually superimpose the prepared content on this poster.
The overall project ?? Mozart & KI ?? by 100 years of the Mozart Festival in Würzburg, the Vogel Foundation, Würzburg Web Week and the Bavarian Sparkasse Foundation.
The ?? Mozart & KI ?? projects can be viewed until Sunday, June 27th, in the ?? M Pop-Up ?? (https://www.mozartfest.de/das-mozartfest/m-popup/index.html) in Würzburg (Plattnerstraße 14) can be viewed or tried out.
Further information under Mozart Jukebox application (https://mozart.fiw.fhws.de/) and https://christophraab.github.io/Jukebox/
https://app-mozartfest.augmented-art.de
–
