[unable to retrieve full-text content]
Working on VR audio technology I often hear, “you mean 3D audio?” or “how is it different from existing 3D audio technology?”. If VR audio is defined as creating and delivering audio signals that are suited for VR content, then the term 3D audio does not fully reflect the capabilities of the audio in a virtual environment.
Listening to interactive, three dimensional audio in VR
Having the the x, y and z axis information is what makes the sound three-dimensional. Traditional ‘surround sound’ or ‘3D audio’ lacks the height information. The 5.1 channel systems, for example, are 2D audio with speakers placed at the height of human ears, which covers front, back, left and right.
True 3D audio became possible in VR with the adoption of binaural rendering technology. Binaural rendering replicates what we hear in the real by transforming three dimensional audio signals into stereo outputs. This allows you to hear true 3D sounds, even through regular headphones. VR audio can immerse users in the scene more than ever before because of one important factor – user interaction.
Previously when there was no user interaction in the story, video and audio were pre-rendered on the production side to a fixed flat screen and a fixed loudspeaker layout. This 2D content was simply played back on the end-user side. On the other hand, user’s head orientation and interaction constantly change in VR, so everything has to be configured on the end-user side. Sound cannot be pre-rendered during VR production because it has to change according to a listener’s viewpoint or movement in real time. Performing real-time binaural rendering on the end-user mobile device is not an easy task.
Binaural rendering by nature requires high computational power to calculate how a sound travels from a sound source position to a listener’s ears, using complex functions. Most mobile bandwidth is used to process video. This is why effective, high-quality binaural rendering technology that alleviates the burden on the mobile device is key in the VR audio industry.
Where VR Audio is At
Overwhelming immersiveness is what completes the ‘being there’ experience in VR. In order to enhance audio immersiveness, efforts were made to increase the audio signal resolution on both the ‘time’ and ‘spatial’ domain. Since resolution in the time domain (frequency) has already exceeded the level that humans can physically recognize, spatial domain is where further developments are needed.
Before the age of VR, the 2D video story was not influenced by end-user’s interaction. The spatial resolution for audio improved just by adding more speakers around the end-user’s frontal rectangular screen. The biggest hurdle for immersion was instead ‘present room effect.’ One could never fully be there in the story because the virtual world was limited by the screen size. This is now a different story because the presence of the real world is blocked by wearing HMD and headphones. VR certainly helps the content consumer be completely transported to a different world, but again it delivers different levels of immersiveness depending on type of content.
In 360 video or […]