Long gone are the days when virtual reality was considered nothing more than a way to enhance computer games. VR technologies continue to gain in importance in research fields such as psychology and cognitive science because of the wealth of opportunities they offer, for example, when it comes to the development of new learning methods or of new approaches in psychotherapy.

What has yet to be explored is the vast spectrum of opportunities within audio-visual projects that explicitly focus on the use of VR in the realm of audio. From our daily life experience, we know that our spatial orientation is influenced by what we hear. Furthermore, film studies have shown that audio can enhance people’s sense of immersion in audiovisual works.

In our project, we want to find out whether it’s possible to use VR technology and 360° audio to render an environment to such a realistic extent that we’re able to develop computer games designed to train people’s hearing. The film and sound recordings used in our project were made in outdoor spaces and buildings in Berlin and Bayreuth. For the sound recordings, we used a microphone designed at the TU Berlin. At each level of the game, players are required to solve perception-based tasks, such as connecting the direction of a sound with the corresponding film clip. The game is intended to be motivating and entertaining while simultaneously training players’ auditory perception.

Based on this game, which is currently in development, our project explores questions that extend beyond the game itself:

What can be and should be achieved with spatial audio in the field of VR?
How can spatial perception be influenced by sound and imagery in VR?
How can VR be used for training and knowledge transfer?