3D Audio

Sound

elastic wave propagation in some medium

Immersive environments

imitation of reality in a computer generated world

Foley artists

manually make and record sounds from real world interactions

Immersive audio

realistic auralization

Localization

user should know where sound source is located

Front-back confusion

position of sound source at front or back is ambiguous

Steps of sound rendering

synthesis and modeling, propagation, 3D audio rendering

Acoustic vs graphics

low vs high geometric detail

Modeling acoustic geometry

surface simplification

Modeling sound material

absorption and scattering coefficient

Source modeling

area source, emitting characteristics, sound signal

Sound propagation

how sound travels in a scene and has complex interactions with boundary

Diffraction

waves tend to bend around obstacles when λ > s, low-frequencies tend to bend more

Scattering

closely related to diffraction, how sound spreads out

Sound propagation in games

strict time budget, moving sources and listeners, tradeoff between speed and accuracy, static environment effects

Sound synthesis

how sound is produced locally due to an object’s vibration, using geometry to figure out correct sound on impact

Sound perception

psycho-acoustics, how sound is processed by the human auditory system

Source

point from which sound is coming

Listener

position where sound is heard

Objects

capable of blocking or reflecting the sound

Soundscape

modeling of acoustic environment

Mono

audio sounds the same for both ears

Stereo

audio can differ between ears

Spatialized audio

can tell where a sound is coming from, simulating a virtual acoustic environment

Nonspatialized audio

can’t tell where sound is in 3D space, sound is not simulated in acoustic environment

Waveforms

signal strength of air pressure over time

Spectrograms

dB of each frequency/pitch over time

Low frequency examples

whale sounds, background noise

Mid frequency examples

human voice

High frequency examples

bird chirps, snapping fingers

Audio unmixing

finding source components from a spectrogram

HRTF

head-related transfer function, estimates how sound will travel through the ear and head

Attenuation

volume at various distances estimated by a curve, commonly logarithmic

Falloff

how audio objects are heard based on distance

Filters

controls sound that survives to output, fast but not always accurate

Low pass filter

allow low-pitch sounds through but suppress high-pitch

High pass filter

allow high-pitch sounds, suppress low-pitch

Occlusion

sound when something is in front of it, often achieved with raycast and LPF or propagation

Directional source

sound is much louder when in front of the source, weighed towards direction

Point source

sound radiates uniformly from all directions

Ambient source

uniform strength applied to all surface e.g. background noise

Direct audio

sound that’s in line of sight

Early reflections

sounds that bounce and are heard soon after

Late reflections

sounds that bounce and sound very delayed

Reflection order

how many bounces until reaches listener

Diffraction

how a sound is heard when not in line of sight

Density

how many rays are traced, fullness of reverb

Diffusion

how far rays get from each other when bouncing

Reflections delay

how long it takes rays to hit objects and reflection to reach listener

Gain

volume before first bounce

Decay time

how quickly sound attenuates

Reflections gain

how much volume survives after hitting obstacle

Late gain

how much volume survives for late reflections

Late delay

how long it takes for late reflections

Air absorption gain HF

how much air absorbs HF sounds

Room rolloff factor

size of reverb zone * attenuation

Acoustic materials

contain scalar parameters describing how the surface affects rays

Distance compression

sound usually sounds too close or far

Cocktail party effect

hearing specific audio cues when lot of sources

Google Resonance

open source, lower requirements, no geometric propagation

SteamAudio

best documentation and good geometric propagation

Oculus Audio

similar to SteamAudio, more preset materials/filters

GSound/MSound

predecessor to many propagation libraries by GAMMA group

UE4/Unity built

in audio

Modal sound synthesis

figure out features of the audio, apply to other geometry and simulate

Challenges of sound synthesis

huge precompute step, trouble with continuous contact and damping, performs poorly realtime