Interactive geometric sound propagation

• Sound propagation: the computation of impulse responses (IRs) that represent an acoustic space.

• Audio rendering: the generation of spatialized audio signal from the impulse responses and dry (anechoically recorded or synthetically generated) source signals.

Sound propagation from a source to a listener conveys information about the size of the space surrounding the sound source, and conveys the source position to the listener even when the source is not directly visible. This considerably improves the immersion in virtual environments. For instance, in a first-person shooter game scenario, the cries of a monster or the steps of an opponent approaching can alert the player to danger.

Sound propagation is also used for acoustic prototyping for computer games, architectural buildings, and urban scenes. Audio rendering also provides sound cues which give directional information about the 3D position of the sound source relative to a listener. For example, in a VR combat simulation it is critical to simulate the 3D sounds of machine guns, bombs, and missiles. Another application of 3D audio is user interface design, where sound cues are used to search for data on a multi-window screen.

Main Results: We present an algorithm and system for interactive sound rendering in complex and dynamic virtual environments. Our approach is based on high-frequency approximations of sound waves. Many such algorithms have been proposed for interactive sound propagation [6, 12, 23]. However, they are either limited to static virtual environments or can only handle propagation paths corresponding to specular reflections.

In order to perform interactive sound rendering, we use methods that can be parallelized across many processors. Our propagation algorithms use a hybrid ray-based representation that traces discrete rays [14] and ray-frusta [18]. Discrete ray tracing is used for diffuse reflections, and frustum tracing is used to compute the propagation paths for specular reflections and edge diffraction. We fill in the late reverberations using statistical methods. We also describe an audio rendering pipeline combining specular reflections, diffuse reflections, diffraction, 3D sound, and late reverberation.

Our interactive sound rendering system can handle models consisting of tens of thousands of scene primitives (e.g., triangles), as well as dynamic scenes with moving sound sources, listener, and scene objects. We can perform interactive sound propagation including specular reflections, diffuse reflections, and diffraction of up to 3 orders on a multi-core PC.

PREVIOUS WORK

Sound Propagation
Sound propagation deals with modeling how sound waves propagate through a medium. Effects such as reflections, transmission, and diffraction are the important components. Sound propagation algorithms can be classified into two approaches: numerical methods and geometric methods.

Numerical Methods: These methods [7, 21] solve the sound wave equation numerically to perform sound propagation. However, despite recent advances [22], these methods are too slow for interactive applications, and are limited to static scenes only.

Geometric Methods: The most widely used methods for interactive sound propagation are based on geometric acoustics (GA). GA techniques represent acoustic waves as rays and can accurately model the early reflections (up to 3 - 6 orders). They compute propagation paths from a sound source to the listener. Specular reflections of sound are modeled with the image-source method [2]. Image-source methods recursively reflect the source point about all of the geometry in the scene to find specular reflection paths. Some methods compute exact visibility [11, 15], some use sampling [14], and some fall in between [6, 17].

There has also been work on complementing specular reflections with diffraction effects. Diffraction effects are very noticeable at corners, as the diffraction causes the sound wave to propagate in regions that are not directly visible to the sound source. Diffraction effects have been used in several interactive simulations [4, 25, 27].

Another important effect that can be modeled with GA is diffuse reflections. Diffuse reflections have been shown to be important for modeling sound propagation [8] and can be modeled with ray tracing based methods [13].

The GA methods described thus far are used to render the early reflections. The reset of the acoustic response, late reverberation, must also be calculated. This is often done through statistical methods [10] or ray tracing [9]. Sound propagation deals with modeling how sound waves propagate through a medium. Effects such as reflections, transmission, and diffraction are the important components. Sound propagation algorithms can be classified into two approaches: numerical methods and geometric methods.

Audio Rendering
Audio rendering is the final step that generates audio for output over headphones or speakers. In the context of geometric sound propagation, it involves using the propagation paths to create impulse responses which represent how an input sound is changed by the environment. Once the impulse response is known, it can be convolved with a dry input audio signal to produce the output audio.

Rendering dynamic scenes is challenging, and there has been work on generating artifact-free audio rendering in such scenes [26, 28]. Introducing 3D cues in the final audio signals is an important effect, and requires convolution of an incoming sound wave with a Head Related Impulse Response (HRIR) [16].