Microsoft files new patent for AI-generated sound in games and movies

Microsoft has filed a new patent for the use of AI-generated music/soundtracks/audio in a wide range of media, including movies, video games, live recordings and related fields. The patent is titled “ ARTIFICIAL INTELLIGENCE MODELS FOR CREATING AUDIO MATERIALS ” and was published on November 17, 2022. MICROSOFT TECHNOLOGY LICENSING , LLC is the patent applicant.

The patent description explains how real-time sound can be generated with an artificial intelligence system built with large datasets that will include machine learning techniques using visual, audio and text (cues). You can read the exact description of the patent below.

A method for training one or more AI models to generate audio scores accompanying visual datasets includes receiving training data containing a plurality of audiovisual datasets and parsing each of the multiple audiovisual datasets to extract a plurality of visual features, textual features, and audio features. The method also includes matching multiple visual and text features to multiple audio features through a machine learning network. Based on correlations between visual, text, and audio features, one or more AI models are trained to produce one or more audio scores to accompany a given data set.

According to the patent, this new technology will help the system generate sound in real time depending on the situation, or, more simply, help generate dynamic/adaptive sound. Interestingly, this technology will separate the experience of each person based on their choice and situation in a video game, if we consider video games as an example of the implementation of this technology.

Microsoft HoloLens in action | Image: US Army

Microsoft’s new AI for sound could go far beyond the usual use of dynamic/adaptive music in games. Player actions can be dynamically evaluated in real time with appropriate audio cues and music. As a result, the sound experience will differ from person to person.

For example, we use pre-recorded background scores and sound in video games and movies that have been recorded according to a predetermined situation that a user will encounter in a particular game or movie. However, video games use more AI technology than movies; In video games, many areas are already implementing AI, whether it’s player interaction with NPCs or the primary level of dynamic sound based on player movements.

On the other hand, movies are more rigid compared to video games because every aspect of the movie is pre-determined and recorded, and nothing changes in real time for the viewers. Thus, as described in the patent, this new technology could be revolutionary in the field of media. This will change everything and players or viewers will feel more involved and immersed than ever in the media they consume.

It’s also not that far-fetched if we think about it realistically, as AI technology has evolved a lot in recent years, from simply using AI for targeted ads to creating ultra-realistic photos and videos with a single line of text; technology has come a long way and sooner or later will be introduced into all areas of the media industry to automate long processes.

It will be interesting to see something like soundtracks created by artificial intelligence in real time. So what do you think of this? Are you looking forward to experience something like this? Let us know in the comments section below.

Leave a Reply

Your email address will not be published.