PSYC 212 FINAL

Sound

Comes from pressure fluctuations in the air

• speed through air: 340 m/s (air pressure is related to the amplitude of the sound wave)

• speed through water: 1500 m/s

Sound Pressure (Pa)

“Pascals”

• measures force exerted by air molecules

Loudness

the psychological perception of sound intensity

• measured in dB (decibels)

Decibels (dB)

smallest perceivable pressure

• 0dB ≠ no sound (represents the minimum audible level)

• scale: 0 (threshold of human hearing) —> 140 (Gunshot, fireworks)

Logarithmic Scale

+10dB = 10x increase in intensity

• ex. sound of a helicopter x10 more intense than a hairdryer

• each 10dB increase represents a tenfold increase in sound intensity

  • ex. difference between 20dB & 30 dB

    • 20 dB = 10² = 100

    • 30 dB- = 10³ = 1000

    • difference = 1000 - 100 = 900

Pitch

Psychological aspect of sound related mainly to the fundamental frequency

Frequency (Hz)

The number of cycles per second

Pure tones

Sounds that only have ONE frequency

Equal loudness curve

a graph plotting sound pressure level (dB SPL) against the frequency for which a listener perceives constant loudness

• “phon”: unit used which corresponds to the dB value of the curve at 1k Hz

Harmonic Spectrum

the spectrum of a complex sound in which energy is at integer multiples of the fundamental frequency

• set of frequencies that make up a complex sound

• acts as a “recipe” that determines its unique timber or tone color

• typically caused by a simple vibrating source (ex. string of guitar)

Fundamental Frequency

the lowest-frequency component of a complex periodic sound

• the lowest frequency of a vibrating object or sound wave that represent the base note or pitch we hear

Timber

psychological sensation by which a listener can judge that two sounds with the same loudness and pitch are dissimilar

• timber quality is conveyed by the profile of the harmonics

Auditory canal

tube-like structure that directs sound waves from the OUTER ear to the tympanic membrane (eardrum)

Eardrum (Tympanic membrane)

thin, vibrating membrane that separates the outer ear from the middle ear

• transmits sound vibrations to the ossicles

Ossicles

three small bones in the middle ear (malleus, incus, stapes)

• amplify and transmit sound vibrations to the inner ear

Cochlea

spiral-shaped, fluid-filled structure in the inner ear

• converts sound vibrations into neural signals for hearing

Oval window

membrane-covered opening that connects the middle ear to the cochlea

• transmits vibrations from the ossicles

Round Window

flexible membrane in the cochlea

• helps relieve pressure from sound waves travelling through the cochlear fluid

Cochlear (auditory) nerve

nerve that carries auditory information from the cochlea to the brain

• used for sound processing

Organ of Corti

a structure on the basilar membrane of the cochlea

• composed of hair cells and dendrites of auditory nerve fibers

Basilar membrane

structure within the cochlea

• vibrates in response to sound

• plays a key role in frequency discrimination by supporting hair cells

Tectorial membrane

gelatinous membrane in the cochlea

• interacts with hair cells

• aid in the conversion of mechanical sound vibrations into electrical signals

Hair cells

sensory receptor cells in the cochlea

• detects sound vibrations and convert them into neural signals transmitted to the brain via the auditory nerve

Cochlear Place Code

• inner cochlear APEX:

  • low frequency

  • low pitch (200Hz)

• outer cochlear BASE:

  • high frequency

  • high pitch (20,000 Hz)

Temporal Coding

neural strategy where auditory information is encoded by the precise timing and patterns of neuronal spikes rather than just their firing rate

• Auditory nerve firing is “phase-locked” (neurons are systematically fired at a given time point of the cycle)

• BUT above 4000-5000 Hz: the refractory period of auditory nerve fibers does NOT allow fibers to fire fast enough

Volley Principle

even if individual auditory neurons cannot keep the pace, the whole population of neurons can still temporarily encode frequency

Cochlear Nucleus

first brainstem region that receives auditory signals from the cochlea

• where initial sound processing occurs

Superior olive

brainstem structure

• involved in sound localization by comparing timing and intensity differences between ears

Inferior colliculus

midbrain structure

• integrates auditory information from various brainstem nuclei

• plays a role in reflexive responses to sound

Medial Geniculate Nucleus (MGN)

relay station in the thalamus

• processes and transmits auditory information to the primary auditory cortex

Primary auditory cortex

region of the cerebral cortex (temporal lobe)

• responsible for processing and interpreting sound information

Tonotopy

spatial organization of sound frequency processing in the auditory system

• where different frequencies are mapped to specific locations along the cochlea and auditory cortex

Belt region

secondary auditory area surrounding the primary auditory cortex

• processes more complex sounds (ex. speech)

Parabelt region

higher-order auditory processing area adjacent to the belt region

• processes more complex sounds (ex. speech)

Dorsal “where” pathway

connects auditory areas to the parietal lobe

• helps determine the location and movement of sound

Ventral “what” pathway

connects auditory regions to temporal lobe

• identifies and categorizes sounds (ex. speech, music)

Conductive hearing imparment

loss of sound conduction to the cochlea (occurs when sound is blocked from reaching the inner ear due to issues in the outer or middle ear)

• Frequently caused by wax, fluid build-up, infections, etc.

• Not a neural hearing loss, but rather a temporary one

Sensorineural hearing impairment

hearing loss due to damage to the cochlea

• ex. congenital (condition from birth), drugs, age, chronic/phasic exposure to loud noise

Interaural Time Difference (ITD)

The difference in time between a sound arriving at one ear versus the other

• Listeners can detect interaural delays of as little as 0.01 ms.

Azimuth

the angle of sound source relative to the center of the head

• very precise

Physiology of ITD

specialized, high-precision neural circuits in the auditory brainstem (mainly superior olive) that detect the subtle differences in arrival time of a sound wave between the two

• with the help of coincidence detector neurons

Interaural Level Difference (ILD)

The difference in level (intensity) between a sound arriving at one ear versus the other

• frequency does not matter

• longest ms. —> where the sound is coming from

coincidence detector neurons

specialized cells that fire only when they receive multiple excitatory inputs within a very narrow, synchronized time window

Inverse Square Law

the intensity of sound decreases as a function of the inverse of the square of the distance

• formula: 𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦_𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑡 = 𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦_𝑠𝑜𝑢𝑟𝑐𝑒 /𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒²

• sound decreases with distance

  • harder to tell small differences in distance between 2 objects if they’re both far away than closer

Sound localization Problem

How can you tel if a source is loud and far away vs. close but quiet?

Solution: the spectral composition of sound changes with distance

• long wavelengths are always more resistant to obstacles (sound, lights, etc.)

• sources that are far away are likely to have encountered more obstacles

• air also has “sound-absorbing” qualities

—> therefore, the intensity of higher frequencies decreases as a function of distance (distal sounds have more reverberated than direct energy)

Cones of confusion

Problem: Auditory localization issue where sounds originating from different locations are perceived as coming from the same place

• have identical ITD and ILD

• Often makes it hard to distinguish if a sound is in front, behind, or above

Solutions:

• moving your head (this will change cones of confusion & the only point that will retain its ITD and ILD is the “real” source)

• the pinna (also ear canal, head, and torso) slightly distorts the amplitude of certain frequencies as a function of elevation

Directional Transfer function

A specialized type of Head-Related transfer function that describes how the pinna, head, and torso modify sound from a specific direction before it reaches the eardrums

• focuses on spectral cues for sound localization by removing the average spectral content

Auditory Stream Segregation

the perceptual organization of a complex acoustic signal into separate auditory events for which each stream is heard as a separate event

Segregation cues

• the location of sounds

• the frequency (pitch) of the sounds

• the timing of the sounds

• the timbre of the sounds

• the onset of the sounds

• rule of “good continuation” (continuity effects)

• higher-order information (restoration effects)

Grouping by frequency (pitch)

tones that have similar frequencies will tend to be grouped together

Grouping by timber

tones that have similar timbre will tend to be grouped together

Grouping by onset

when sounds begin at different times they appear to be coming from different sound sources

Continuity effect

Despite interruptions, one can still “hear” a continuous sound if the gap is filled with noise

• In that case, the sound is perceived as continuing behind the noise

• However, if the gaps aren’t filled with noise, the sound is perceived as separate “chunks”

Grouping by Time

tones that are close together in time will tend to be grouped together

Restoration effect

Despite interruptions, one can still “hear” a sentence if the gaps are filled with noise

• in this case, higher-order semantic/syntactic knowledge is used to “fill the blanks”

• as for continuity effects, the effect vanishes if the gaps are not filled with noise

Place Coding

Because of how the cochlea and basilar membrane are constructed, acoustic stimuli of different frequencies cause different amounts of movement along the basilar membrane

• In the neural tissue, different locations encode specific aspects of sound

  • Specifically, frequency or pitch

    • The bigger vibration = lower frequency

    • Narrower basilar membrane = higher frequency 

      • higher frequency sounds cause bending of the basilar membrane closest to the stapes, resulting in more hair cell activity in that area

Rate coding

low frequency sounds are processed using a rate coding system

• the pattern of neurotransmitter release from the hair cells deepest in the cochlea (furthest from the stapes) determines the perception of low frequency sounds

Time coding of sound

Frequency will generate a firing of neurons at the same frequency throughout time

• a process where auditory nerve fibers fire in a specific pattern in response to sound waves

Hearing Aids

Acoustic amplifiers that you put in the ears to make sounds louder and then restore audibility to softer sounds

• “micro-amplifiers”

• If hearing loss is severe → cochlear implants

Types of hearing aids

• In-the-ear (ITE) hearing aids

• Behind-the-ear (BTE) hearing aid

  • Much smaller because it is detached from the amplifier (leaves room to allow air to go through the ear canal and leaves it open)

  • Used when we want to address hearing loss in the high frequencies only (Frequency-specific amplification)

Hearing aid fit check

validation of acoustic output at the tympanic membrane

Electro-acoustic stimulation (EAS)

the use of a hearing aid and a cochlear implant technology together in the same ear

• Output of a speakerphone (hearing aid)

• A cord to receive sounds and turn them into an electrode (cochlear implant)

  • Stimulate the nerve for the high frequencies

—> We use the software on those machines to adjust to the needs of the patients

Hearing aids (How they work)

1. a microphone captures sound waves and converts them into digital signals

2. a computer processor amplifies and modifies these signals to match specific hearing loss needs

3. a speaker (receiver) converts the signals back into sound waves, delivering them into the ear canal

Cochlear implants

An electrode array that is installed along the cochlear nerve to restore hearing 

• Includes two pieces:

  • External: There is an external piece with a microphone that picks up a sound

  • Internal (under the skin): Then signals the sound to a subcutaneous piece that itself trans converts the sound into an electrical signal along the cochlea

    • Placed surgically

Candidacy for device

Hearing aid candidate:

• < 60 dB of hearing loss

• > 60% of speech recognition

Cochlear implant candidate:

• > 60 dB of hearing loss

• < 60% of speech recognition

The Superior Temporal Sulcus

auditory analogue of the fusiform face area in the visual system 

• region that is selective for voices

Wernicke’s area

Most important area to understand language

• posterior section of the superior temporal gyrus (left hemisphere)

Wernicke’s aphasia

Fluent language but with a lack of sense/meaning (inability to recognize sounds as words)

• Impairment in meaning understanding

• Reading is impaired (able to read the words but lack understanding of the meaning of words)

• Speech does not make sense because they do not know the literal meaning of words

Broca’s area

Controls the motor organization of speech sounds

• In the inferior frontal gyrus (left hemisphere)

Broca’s Aphasia

Not fluent language (difficulty finding the words to communicate) but comprehension is intact

• Opposite of wernicke’s aphasia

• Trouble repeating words

• Patients can recover to some degree

Phonemes

a unit of sound that distinguishes one words from another in a particular language 

• Ex. kill vs. kiss

• Required when speaking and understanding speech

Respiration (#1)

The diaphragm pushes air out of lungs → through trachea → up to the larynx (contains vocal folds or vocal cords)

• 1st step to speech production

Phonation (#2)

• The process through which vocal folds are made to vibrate when air pushes out of the lungs

• Vocal folds will vibrate as the air pushes through these folds (on each sides)

  • More tension = higher-pitched sounds

  • Small vocal cords, high-pitched voices → children < women < men

  • The type of sound that is being produced by the vocal folds will have a harmonic spectrum (amplitude/frequency)

Articulation (#3)

• The act or manner of producing a speech sound using the vocal tract

  • Oral tract (vocal and nasal) is the area above the larynx

  • Humans can change the shape of their vocal tract by manipulating the jaws, lips, tongue, etc. (“articulation”)

    • This allows the change in sounds

—> Changing size and shape of vocal tracts will increase/decrease energy at different frequencies

• includes formants and spectograms

Formants

Peaks in the speech spectrum (helps identifying the phoneme)

• Labelled by number (from lowest to highest) 

• Shorter vocal tracts = higher frequencies

Spectrograms

Represent the sentence said by someone (based on amplitude, frequency and time)

• The spectrum of speech sounds changes over time

  • x: time

  • y: frequency

  • color: energy (amplitude)

Coarticulation

Phenomenon when our oral tract anticipate the next phoneme as we are pronouncing the current phoneme

• Occurs to experienced talkers (they position their tongues in antiticipation of the next consonant or vowel) → causing a change in pronunciation (overlap)

• Makes it difficult for the auditory system to identify the phoneme

• Ex. “di” → “du”

Categorial Perception

Occurs when a stimulus changes continuously but we perceive it as belonging to discrete categories

• Because of this → differences across category boundaries appear LARGER than equally large differences within category

• We do not perceive the sounds as continuously varying BUT there are sharp categorical boundaries between stimuli

  • Perceived as more different 

—> resolution to coarticulation

Learning to Listen

4-day old French babies prefer hearing French over Russian

• sound distinctions are specific to various languages (ex. “r” and “l” are not distinguished in Japanese)

• infants begin filtering out irrelevant acoustics long before they start to say speech sounds

  • ex. english-speaking infants <10 months can distinguish between two types of “t” sounds that are different phonemes in Hindi but lose that ability after 10 months, while Hindi infants still continue to make distinction

—> Babies aren’t just learning words—they’re reshaping how they hear the world.

• At first: broad, flexible perception

• Over time: specialized for their native language

“Motor theory” of speech perception:

Motor processes used to produce speech sounds are used in reverse to understand the acoustic speech signal

• Supported by the McGurk Effect: showed that what someone sees can affect what they hear

• Problem: Speech production is as complex, if not more complex, than speech perception

  • solution: reduce the number of phonetic categories

Tone height

a sound quality corresponding to the level of pitch

• Tone height is monotonically related to frequency

Tone Chroma

a sound quality shared by notes that have the same octave interval

• Each note on the musical scale (A-G) has a different chroma

Octave

the interval between two sound frequencies having a ratio of 2:1

• Western music has 13 notes separated by equally spaced pitch intervals (semitones)

• There are 10 octaves within the audible range (piano has 7 octaves)

Consonance

When two or more notes are played simultaneously (a chord) or sequentially

• Consonance: the combination of sounds is pleasant, as if the notes “go together”

  • Happens when the fundamental frequency of the two notes has a simple ratio

  • Many harmonics of the two sounds will coincide

→ study shows that even people as young as 2-month old infants prefer consonant conditions over dissonant ones

Dissonance

When two or more notes are played simultaneously (a chord) or sequentially

• Dissonance: the combination of sounds is unpleasant or “off”

  • Happens when the fundamental frequencies of the two notes have a complex ratio

  • Very few harmonics will coincide

Scale

A particular subset of the notes in an octave

Key

the scale that functions as the basis of a musical composition

• Ex. a composition in the key of C major contains notes mostly from the C major scale

Major and Minor scales

Major and minor scales are differentiated by the pattern of intervals (number of semitones) between successive notes

• Major scales: 2-2-1-2-2-2-1

  • Sound “happy”

• Minor scales: 2-1-2-2-1-2-2

  • Sound “sad”

Tonic

the root note of the key

• Acts as the gravity point of the key

• Moving away from and back at resting point → what makes music interesting and why it has a pleasing effect on us

Melody

a sequence of notes or chords perceived as a single coherent structure

• Ex. row, row, row your boat

• defined by contours

• can change octaves or keys and still be the same melody even if they have completely different notes

Contours

The pattern of rises and declines in pitch (rather than by an exact sequence of sound frequencies)

Right auditory cortex

Music is mostly processed in the RIGHT auditory cortex

• speech is mostly processed in the left auditory cortex

Music and Brain study

Study:

• Presented two different pitches to participants:

  • 1st group: stable, still pitch

  • 2nd group: changing pitch

• Result: 

  • Fixed spitch sequence vs silence → primary auditory cortex is active

  • Changing pitch sequence vs. fixed pitch sequence → melodies are being processed by the right belt and para belt regions of the right auditory cortex

Temporal resolution

Speech requires fine temporal resolution

• Production is very fast (10-15 consonants & vowels per second)

  • Can be doubled if you talk fast

• Experienced talkers position tongue in anticipation of next consonant or vowel → causing coarticulation

  • Coarticulation will cause overlap in articulatory or speech patterns

Spectral Resolution

music requires fine spectral resolution

• The left auditory cortex is able to distinguish the different lyrics but not the different melodies

• The right auditory cortex is able to distinguish the different melodies but not the different lyrics

Congenital Amusia

inability to perceive music 

• perfectly normal otherwise, only have a problem with identifying music

• Unaware of how they sing

• Fail to recognize popular tunes, without the help of lyrics

• Akin to dyslexia, prosopagnosia, etc. 

• Seems to have problems with their arcuate fasciculus (on the right side)

Congenital Amusia Diagnosis

Developed a battery of musical tests 

• In our brains we have different specific modules/regions for processing the melody. 

  • When we process music → we combine melody and the temporal structure to recognize the musical excerpt

Arcuate Fasciculus

bundle of white matter that connects the auditory cortex to frontal regions (inferior frontal gyrus and superior temporal gyrus)

Absolute Pitch

capacity to actually name or produce a pitch without comparisons to other notes

• 1/1500 have it; 

• Heritable (has genetic basis)

• requires exposure to musical training in early life (environment)

Reward Prediction Error

concept used to describe how dopamine neurons in your brain (specifically in the ventral tagmental area of brainstem) signal the difference between the reward you expect and the reward you actually get

• Works in 3 main ways:

  • Positive surprise: if a reward happens and you didnt see it coming → dopamine neurons show a sudden burst of activity 

  • No surprise: if you see a cue that predicts a reward is coming → dopamine neurons fire when they see that cue 

    • However when the reward arrives there is no extra bursts 

  • Negative surprise: if you expect a reward but it does not happen → your dopamine neurons fire at the initial cue but activity drops or pauses at the moment the reward was supposed to appear

→ Reward circuitry is interested by learning what causes the reward and not necessarily the reward itself