Sony Titles Could Feature Sound Generation Based On User Environment

VR and AR games have grown immensely in recent years, becoming a new phenomenon in the gaming industry. The upcoming Sony VR experience Horizon Call of the Mountain and the popular AR games like Pokemon Go are ideal instances of the technical capabilities of these new technologies. 

Besides staggering developments in VR and AR technology, immersion still feels lacking, but that may change soon in the future. One significant aspect of immersion in video games is the sound generation, and game studios are finally toiling to improve the stagnant game mechanic.

We stumbled across a new patent published by Sony named “IMPULSE RESPONSE GENERATION SYSTEM AND METHOD,” which will produce in-game sounds by considering the user’s actual environment. For instance, for a player inside a room or a park, the gameplay audio will be induced to reflect their current background.

Major Takeaway:

• Sony has published a new patent that will produce in-game sounds based on the user’s current personal environment; it primarily discusses AR and VR titles but extends beyond them. 
• It is a big upgrade over the current “surround-sound” system that only makes you feel surrounded by speakers in a closed room. This system will instead reflect your unique background to provide an immersive experience.
• The gameplay sound will differ depending on whether a player is in a closed or open space, and other similar variables will also be considered.
• The VR or AR device (a phone or a headset) will capture images of the player’s current background to deduce how in-game audio should be produced.

The patent by Sony serves as an upgrade over the current “surround-sound” technology, which was created to upscale the immersion. However, that method is not viable when the user’s environment is to be considered. So the patent proposes a new system for AR and VR games to increase immersion on a user’s personal level.

Sony elaborates, “when augmented reality content is provided that is associated with an audio output, this audio output will be expected to sound different depending upon the environment in which the augmented reality content is reproduced.”

Moreover, the patent cites the need for the proposed system to produce in-game audio to reflect the player’s environment automatically. It continues, “A user that views this content in a small room at home will expect the audio to differ with respect to a user that views the content in an echoey cavern.”

It is impractical to manually generate audio by considering the environment, as every player might indulge in varied, diverse settings. Some users may play AR games in their rooms, while others may sit on a bench in a park while playing.

It elaborates, “for a popular product, this may be hundreds of thousands of different environments as each user is likely to view content in a unique environment.” Moreover, manually collecting data for each environment will be very tough and time-consuming, per the patent.

The system will use something called a “dynamic impulse response generation process,” which will be allowed for an environment to be performed. It further continues, “This is enabled by using the properties of a known sound within the environment along with a recording of the sound, with the recording being captured within the environment being considered.”

It is dynamic because the process can be performed whenever desired. For instance, “whenever a user enters a new environment or whenever a new content is initialised.” Moreover, the process can be performed for a “number of locations within an environment.”

Sony’s proposed system will work perfectly for gaming, as stated. “application may be particularly suitable given that immersive games may provide content that is displayed so as to appear to be in the room (that is, the environment) with the user.” The user will feel that the game takes place within their environment.

One of the most examined examples is AR and VR technology. The AR or VR device (which could be a headset or a phone) will capture images of the environment to deduce the user’s background. After figuring out the captured images, the location will be identified to produce the relevant audio to the setting.

“When using an AR arrangement there may be an outward-facing camera that is operable to capture images of the environment. From these images, it may be possible to identify the size of the user’s local environment or some context that indicates a likely location (such as large green areas indicating the user being outdoors in a park).”

Many variables will be measured, such as the environment type, size, and more. It continues, “Based upon indicators such as environment size or type it may be possible to identify expected characteristics or impulse responses for the environment.”

Moreover, the dynamic impulse response generation can be controlled to keep it relevant. The patent continues, “deviations greater than a particular threshold may be seen to indicate an incorrect impulse response being generated. This can then be the trigger for repeating the impulse response generation process.”

In some cases, the user may be directed to make sounds, sometimes of various pitches, to help identify the environment. The patent states, “the user may be directed to mimic a particular sound that is provided—for example, to whistle or clap (in some cases, with the aim of reproducing a sound that is played to the user) […] which may enable a more accurate generation of the impulse response.”

Sony’s patent mainly discusses the patent in the context of gaming but is stated to apply to other industries. Furthermore, this “impulse response generation method” can be employed in VR and AR games. However, the patent also hints at including other games and interactive experiences, as explicitly stated.

The patent mentions, “Such an implementation may be suitable for use with a wide range of applications, as a controller may be provided for use with a number of different VR, AR, and interactive experiences.”

That is not all; more studios have begun focusing on upgrading the overlooked mechanics like the audio generation in video games. Sony is also exploring ways to revamp the haptic feedback for VR titles. Moreover, using artificial intelligence, Activision is attempting to generate in-game music unique to each player.

What are your thoughts about Sony games, especially AR and VR titles, featuring in-game sound generation based on the user’s unique environment? Do let us know your opinions in the comments below.

Similar Reads: Kratos Vs Thor Leaked Fight Video Garners Over 2 Million Views In 24 Hours