Binaural sound or binaural audio in virtual reality reproduces sound as you would hear it in the real world. Practically, this means that as you move your head to look around the scene your sound changes dynamically as do your visuals. Binaural sound is different to regular pinned sound – we don’t just make sound louder from objects in certain directions, we perfectly recreate the source by having phase, loudness and time differences between the ears too; this is one of the main ways we pinpoint sound in the real world.
Firstly, in virtual reality we are always aiming to create a feeling of ‘presence’, this is when the user is truly lost in the scene, forgetting reality and absorbed and immersed in the virtual. Presence can only be achieved if all of the cues received are completely natural. If the sound is flat and non-directional the spell will be broken, hence binaural/360 sound is vital for experiences to feel real.
Secondly we can use binaural sound/3D audio to aid us in storytelling, audio cues can ensure users are seeing everything you want them to. We used binaural sound during our work with Meantime Brewery to highlight animated statistics that were appearing around the scene.
360 audio/binaural sound is often overlooked because it is very hard to capture and playback well. Henrik Oppermann, head of sound at Visualise is a specialist in creating VR sound environments and has built dedicated capture devices for doing this. We have also developed our own plugins and techniques to track sound sources and reposition these in dynamic, interactive, VR environments. To do this well is incredibly hard and is often an overlooked part of virtual reality experiences.
We capture sounds from multiple sources and directions on set or location. Using binaural microphones, directional microphones and a whole assortment of specialist studio quality equipment depending on the specifics of the shoot. During the 360° music gig with Kasabian we produced with Sony Music, we captured 53 audio sources allowing us to rebuild the soundscape exactly how it would have felt in each of the camera positions.
Experience directional audio in our latest 360° video for MSF-USA Forced From Home campaign.