PSYC Week 7 - Sensation and Perception

Hearing: Learning Objectives

Hearing: Learning Objectives

- Describe the basic auditory attributes of sound.

- Describe the structure and general function of the auditory pathways from the outer ear to the auditory cortex.

- Discuss ways in which we are able to locate sounds in space.

- Describe various acoustic cues that contribute to our ability to perceptually segregate simultaneously arriving sounds.

Describe the basic auditory attributes of sound

3 Main Categories:

1. Loudness

2. Pitch

3. Timbre

Basic auditory attributes of sound: Loudness

- A direct physical correlate of loudness is sound intensity (or sound pressure) measured close to the eardrum

Other factors that influence the loudness of a sound:

- frequency content

- duration

- the context in which it is presented

Measurement methods involve techniques such as:

- magnitude estimation

- where a series of sounds (often sinusoids, or pure tones of single frequency) are presented sequentially at different sound levels, and subjects are asked to assign numbers to each tone, corresponding to the perceived loudness

- How loudness changes as a function of the frequency of a tone, resulting in the international standard iso-loudness-level contours which are used in many areas of industry to assess noise and annoyance issues

- Such studies have led to the development of computational models that are designed to predict the loudness of arbitrary sounds

Basic auditory attributes of sound: Pitch

- Pitch is crucial in acoustic communication

- Pitch variations over time provide the basis of melody for most types of music

- Pitch contours in speech provide us with important prosodic information in non-tone languages, such as English, and help define the meaning of words in tone languages, such as Mandarin Chinese.

- Pitch is the perceptual correlate of waveform periodicity, or repetition rate: The faster a waveform repeats over time, the higher perceived pitch

- Most common pitch-evoking sounds = harmonic complex tones

- They are complex because they consist of more than one frequency, and they are harmonic because the frequencies are all integer multiples of a common fundamental frequency (F0)

- Ex: a harmonic complex tone with a F0 of 100 Hz would also contain energy at frequencies of 200, 300, 400 Hz, and so on

- These higher frequencies are known as harmonics or overtones, and they are important in determining the pitch of a sound

- Ex: even if the energy at the F0 is absent or masked, we generally still perceive the remaining sound to have a pitch corresponding to the F0

- This phenomenon is known as the “pitch of the missing fundamental,” and it important in the formation of theories and models about pitch

Basic auditory attributes of sound: Timbre

- Timbre refers to the quality of sound

- described using words like bright, dull, harsh, and hollow

- timbre includes anything that allows us to distinguish two sounds that have the same loudness, pitch, and duration

- Ex: a violin and a piano playing the same note sound very different, based on their sound quality or timbre.

- spectral content of a sound: Sounds with more high-frequency energy tend to sound brighter, tinnier, or harsher than sounds with more low-frequency content, which are deep, rich, or dull

- temporal envelope (or outline) of the sound, especially how it begins and ends

Ex: a piano has a rapid onset, or attack, produced by the hammer striking the string, whereas the attack of a clarinet note can be much more gradual

- the overall spectral content and the temporal envelope can provide a good first approximation to any sound, but it turns out that subtle changes in the spectrum over time (or spectro-temporal variations) are crucial in creating plausible imitations of natural musical instruments

Describe the structure and general function of the auditory pathways from the outer ear to the auditory cortex: 1. OUTER EAR

Outer Ear:

- consists of the pinna, the ear canal and the tympanic membrane

- our brain can compare the subtle differences in the signals at the two ears to localize sounds in space.

- However, this trick does not always help: a sound directly in front or directly behind you will not produce a difference between the ears

- In these cases, the filtering produced by the pinnae helps us localize sounds and resolve potential front-back and up-down confusions

- the folds and bumps of the pinna produce distinct peaks and dips in the frequency response that depend on the location of the sound source

- The brain then learns to associate certain patterns of spectral peaks and dips with certain spatial locations.

- this learned association remains malleable, or plastic, even in adulthood

Ex:

- a study that altered the pinnae using molds found that people could learn to use their “new” ears accurately within a matter of a few weeks

- Because of the small size of the pinna, these kinds of acoustic cues are only found at high frequencies, above about 2 kHz

- At lower frequencies, the sound is unchanged whether it comes from above, in front, or below

- The ear canal itself is a tube that helps to amplify sound in the region from about 1 to 4 kHz—a region important for speech communication.

Describe the structure and general function of the auditory pathways from the outer ear to the auditory cortex: 2. MIDDLE EAR

- The middle ear consists of an air-filled cavity, which contains the middle-ear bones, known as the incus, malleus, and stapes, or anvil, hammer, and stirrup (their shapes)

- smallest bones in the body

- Primary function: transmit the vibrations from the tympanic membrane to the oval window of the cochlea through a lever action, to match the impedance of the air surrounding the tympanic membrane with the fluid in the cochlea.

Describe the structure and general function of the auditory pathways from the outer ear to the auditory cortex: 3. INNER EAR

- The inner ear includes the cochlea in the temporal bone of the skull

- the mechanical vibrations of sound are transduced into neural signals that are processed by the brain

- The cochlea is filled with fluid and is also spiral shaped

- Along the length of the spiral runs the basilar membrane: vibrates in response to the pressure differences produced by vibrations of the oval window

- Sitting on the basilar membrane is the organ of Corti, which runs the length of the basilar membrane from the base (by the oval window) to the apex (the “tip” of the spiral)

- The organ of Corti includes three rows of outer hair cells and one row of inner hair cells

- The hair cells sense the vibrations due to their stereocillia

- The outer hair cells function mechanically and amplify the sound-induced vibrations

- The inner hair cells form synapses with the auditory nerve and transduce those vibrations into action potentials, or neural spikes, which are transmitted along the auditory nerve to higher centers of the auditory pathways.

Describe the structure and general function of the auditory pathways from the outer ear to the auditory cortex: 4. COCHLEA

- frequency analysis (very important part of hearing) is established in the cochlea

- action of the cochlea = a prism: the frequencies that make up a complex sound are broken down into their constituent frequencies:

1. low frequencies creating maximal basilar-membrane vibrations near the apex of the cochlea

2. high frequencies creating maximal basilar-membrane vibrations near the base of the cochlea

- This decomposition of sound into its constituent frequencies, and the "tonotopic" representation, is a very important organizational principle

- It is maintained in the neural representation of sounds from the cochlea to the primary auditory cortex.

- This decomposition into constituent frequency components is what allows us to hear more than one sound at a time

- Frequencies are also represented by the timing of spikes within the auditory nerve = "phase locking,"

- This is crucial in comparing time-of-arrival differences of waveforms between the two ears

Describe the structure and general function of the auditory pathways from the outer ear to the auditory cortex: 5. AUDITORY CORTEX

- auditory signals go through many stages of processing before they reach the primary auditory cortex

- primary auditory cortex is in the temporal lobe

- Understanding of the processing accomplished by higher stages of the auditory pathways is minimal

Overview of Auditory System

***Refer to the Hearing (Audition) section of Sensation and Perception for the specific pathway that sound takes as it travels through the ear***

- Auditory perception depends on how sound is processed through the ear

- Two ears can be useful to determining location of the source of sound

- Outer ear: pinna + ear canal (auditory meatus) + tympanic membrane

o Folds + bumps of pinna make distinct peaks and dips in frequency response (depending on location of sound source) which is then interpreted by the brain

- Tympanic membrane vibrates in response to sound

- Middle ear: air-filled cavity, contains ossicles

- Ossicles: (malleus, incus, stapes) are the smallest bones in the body

o Function to transmit vibrations from tympanic membrane to oval window of cochlea

- Inner ear: cochlea encased in temporal bone of skull

- Cochlea: spiral bone filled with fluid and auditory hair cells

- Basilar membrane runs along the spiral of the cochlea and vibrates in response to

pressure differences made by oval window vibrations

- Organ of Corti runs along the length of the basilar membrane from base (by oval

window) to apex (the “tip” of spiral)

- Includes 3 rows of outer hair cells and 1 row of inner hair cells

- Outer hair cells mechanically amplify the sound-induced vibrations

- Inner hair cells form synapses with auditory nerve and transduce

vibrations into action potentials (to be sent to higher centers of auditory

pathways

- Frequency analysis: cochlea breaks up the many frequencies of a sound

o Low frequencies = maximal basilar-membrane vibrations near apex of cochlea

o High frequencies = maximal basilar-membrane vibrations near base of cochlea

o Frequency-to-place mapping = tonotopic representation

- Frequency analysis allows us to hear more than one sound at once

- Phase locking: frequencies are represented by the timing of spikes in the auditory nerve

- PL compares time-of-arrival differences of waveforms between two ears

- There may be a pitch center in auditory cortex, but more studies are required on which parts of the cortex are responsible for specific features (like pitch)

Discuss ways in which we are able to locate sounds in space.

- 360 degrees of hearing, but acuity is an order of magnitude less than vision in locating object in space

- Two main sources of information are from comparison of sounds at two ears:

1. Interaural time differences:

- sound source on the right reaches right ear slightly before it reaches the left ear

- We are more sensitive to ITDs at low frequencies (below 1.5 kHz) because we can still perceive changes in timing for high frequencies

2. Interaural level differences:

- at higher frequencies, the head casts an acoustic shadow so that when sound is presented from the right, the sound level at the right ear is somewhat higher than the sound level at the left

- ILDs at high frequency are more useful because head shadow is the greatest

- Perception of spatial location is due to ITDs in low-frequency temporal fine structure

- Perception of distance depends on context:

- As speaker moves further away, direct sound level decreases

- However, sound level of reverberation remains the same

- (so ratio of direct-to-reverberant energy decreases)

Describe various acoustic cues that contribute to our ability to perceptually segregate simultaneously arriving sounds

- Normal range of frequencies: ~20 Hz to 20 000 kHz

- Normal range of loudness: 1 to 4 kHz to sounds with a factor of 1 000 000 000 000 less

intense

- Due to wide scale, we use decibels to describe sound pressure/intensity

- 0 dB sound pressure level (SPL) = 20 micro-Pascals (quietest sound level)

- 120 dB SPL = Dangerously loud sound

- Masking: presence of one sound makes another more difficult to hear as far as the frequencies of the sounds overlap

- Suppression: when response to the masker reduces neural (and maybe mechanical) response to the target sound (maybe due to failure to separate target from masking sounds)

- At least some forms of masking originate in auditory cortex, not before

- Upward spread of masking: low frequency sounds are more likely to mask high frequency sounds (especially at higher intensities)

Cochlea

Snail-shell-shaped organ that transduces mechanical vibrations into neural signals.

Interaural differences

Differences (usually in time or intensity) between the two ears.

Pinna

Visible part of the outer ear.

Tympanic membrane

Ear drum, which separates the outer ear from the middle ear.

Gregory and Peter are both violinists who play with their city’s orchestra. As they are warming up for a concert, Peter says to Gregory: “I think you’re A is a bit flat.” Peter is suggesting that the ______ of Gregory’s violin needs to be adjusted.

purity

.

intensity

.

timbre

.

pitch

.

amplitude

pitch

Different musical instruments give a different quality of sound. A trumpet, for example, may be described as “tinny,” while a cello might be described as producing a “rich” sound. These sound qualities refer to the ______ of the instrument.

pitch

.

timbre

.

intensity

.

frequency

.

amplitude

timbre

Tympanic membrane is:

an ear drum, which separates the outer ear from the middle ear.

an ear drum, which separates the outer ear from the concha.

.

an ear drum, which separates the inner ear from the cochlea.

.

an ear drum, which separates the outer ear from the inner ear.

.

an ear drum, which separates the outer ear from the pinna.

an ear drum, which separates the outer ear from the middle ear.

Which of the following is a primary function of the malleus, incus, and stapes?

transmitting vibrations from the tympanic membrane to the oval window

swaying gently up and down to create innervation of auditory neurons

.

expanding and contracting when “struck” by sound waves

.

enhancing sound waves that are over 20,000 hz in frequency

.

blocking certain sound stimuli to protect the auditory receptors from damage caused by sounds that are too loud

transmitting vibrations from the tympanic membrane to the oval window

Mathilda is studying the parts of the brain and their various functions. When she gets to the sense of hearing, she should probably realize that the primary auditory centers are located in the ______ lobes of her brain.

corporeal

.

parietal

.

temporal

.

occipital

.

frontal

temporal

If you were to go to a concert of your favorite band, you’d want to avoid sitting right next to the powerful speakers. Based on your reading, you know that a sound amplitude over ______ decibels (db) sound pressure level (SPL) is considered dangerously loud.

100

.

50

.

120

.

70

.

145

120

Brian is listening to his son, Abel, talk about what he did in school today. Suddenly Brian has difficulty hearing the story because Abel's brother--who has a very similar voice--starts talking on the telephone nearby. Which phenomenon describes Brian's difficulty?

auditory shadowing

.

tinniting

.

masking

.

fusion

.

priming

masking

top down vs bottom up processing

• However, during the time you first eat something or hear a band, you process those stimuli using bottom-up processing. This is when we build up to perception from the individual pieces. Sometimes, though, stimuli we’ve experienced in our past will influence how we process new ones. This is called top-down processing. The best way to illustrate these two concepts is with our ability to read. Read the following quote out loud (do not notice the 2 “the”s)

Sensory adaptation:

• Finally, it should be noted that when we experience a sensory stimulus that doesn’t change, we stop paying attention to it. This is why we don’t feel the weight of our clothing, hear the hum of a projector in a lecture hall, or see all the tiny scratches on the lenses of our glasses. When a stimulus is constant and unchanging, we experience sensory adaptation. This occurs because if a stimulus does not change, our receptors quit responding to it. A great example of this occurs when we leave the radio on in our car after we park it at home for the night. When we listen to the radio on the way home from work the volume seems reasonable. However, the next morning when we start the car, we might be startled by how loud the radio is

measuring absolute thresholds

• The way we measure absolute thresholds is by using a method called signal detection. This process involves presenting stimuli of varying intensities to a research participant in order to determine the level at which he or she can reliably detect stimulation in a given sense. During one type of hearing test, for example, a person listens to increasingly louder tones (starting from silence). This type of test is called the method of limits, and it is an effort to determine the point, or threshold, at which a person begins to hear a stimulus (see Additional Resources for a video demonstration). In the example of louder tones, the method of limits test is using ascending trials. Some method of limits tests use descending trials, such as making a light grow dimmer until a person can no longer see it.

webers law

• A similar principle to the absolute threshold discussed above underlies our ability to detect the difference between two stimuli of different intensities. The differential threshold, or just noticeable difference (JND), for each sense has been studied using similar methods to signal detection. To illustrate, find a friend and a few objects of known weight (you’ll need objects that weigh 1, 2, 10 and 11 lbs.—or in metric terms: 1, 2, 5 and 5.5 kg). Have your friend hold the lightest object (1 lb. or 1 kg). Then, replace this object with the next heaviest and ask him or her to tell you which one weighs more. Reliably, your friend will say the second object every single time. It’s extremely easy to tell the difference when something weighs double what another weighs! However, it is not so easy when the difference is a smaller percentage of the overall weight. It will be much harder for your friend to reliably tell the difference between 10 and 11 lbs. (or 5 versus 5.5 kg) than it is for 1 and 2 lbs. This is phenomenon is called Weber’s Law, and it is the idea that bigger stimuli require larger differences to be noticed.

sensation vs perception

Sensation: the raw sensory data activating peripheral sensors

Perception: the organized, coherent reality that you are aware of

How vision works:

• We are actually seeing light bounce off that object and into our eye. Light enters the eye through the pupil, a tiny opening behind the cornea. The pupil regulates the amount of light entering the eye by contracting (getting smaller) in bright light and dilating (getting larger) in dimmer light. Once past the pupil, light passes through the lens, which focuses an image on a thin layer of cells in the back of the eye, called the retina.

why do we have depth perception

• Because we have two eyes in different locations, the image focused on each retina is from a slightly different angle (binocular disparity), providing us with our perception of 3D space (binocular vision). You can appreciate this by holding a pen in your hand, extending your arm in front of your face, and looking at the pen while closing each eye in turn. Pay attention to the apparent position of the pen relative to objects in the background. Depending on which eye is open, the pen appears to jump back and forth! This is how video game manufacturers create the perception of 3D without special glasses; two slightly different images are presented on top of one another.

Photoreceptors

It is in the retina that light is transduced, or converted into electrical signals, by specialized cells called photoreceptors. The retina contains two main kinds of photoreceptors: rods and cones. Rods are primarily responsible for our ability to see in dim light conditions, such as during the night. Cones, on the other hand, provide us with the ability to see color and fine detail when the light is brighter. Rods and cones differ in their distribution across the retina, with the highest concentration of cones found in the fovea (the central region of focus), and rods dominating the periphery (see Figure 2). The difference in distribution can explain why looking directly at a dim star in the sky makes it seem to disappear; there aren’t enough rods to process the dim light!

Optic nerve → higher brain centres:

• Next, the electrical signal is sent through a layer of cells in the retina, eventually traveling down the optic nerve. After passing through the thalamus, this signal makes it to the primary visual cortex, where information about light orientation and movement begin to come together (Hubel & Wiesel, 1962). Information is then sent to a variety of different areas of the cortex for more complex processing. Some of these cortical regions are fairly specialized—for example, for processing faces (fusiform face area) and body parts (extrastriate body area). Damage to these areas of the cortex can potentially result in a specific kind of agnosia, whereby a person loses the ability to perceive visual stimuli. A great example of this is illustrated in the writing of famous neurologist Dr. Oliver Sacks; he experienced prosopagnosia, the inability to recognize faces. These specialized regions for visual recognition comprise the ventral pathway (also called the “what” pathway). Other areas involved in processing location and movement make up the dorsal pathway (also called the “where” pathway). Together, these pathways process a large amount of information about visual stimuli (Goodale & Milner, 1992). Phenomena we often refer to as optical illusions provide misleading information to these “higher” areas of visual processing.

Dark and Light Adaptation

• Dark and Light adaptation: Humans have the ability to adapt to changes in light conditions. As mentioned before, rods are primarily involved in our ability to see in dim light. They are the photoreceptors responsible for allowing us to see in a dark room. You might notice that this night vision ability takes around 10 minutes to turn on, a process called dark adaptation. This is because our rods become bleached in normal light conditions and require time to recover. We experience the opposite effect when we leave a dark movie theatre and head out into the afternoon sun. During light adaptation, a large number of rods and cones are bleached at once, causing us to be blinded for a few seconds. Light adaptation happens almost instantly compared with dark adaptation. Interestingly, some people think pirates wore a patch over one eye in order to keep it adapted to the dark while the other was adapted to the light. If you want to turn on a light without losing your night vision, don’t worry about wearing an eye patch, just use a red light; this wavelength doesn’t bleach your rods.

Colour vision:

• Our cones allow us to see details in normal light conditions, as well as color. We have cones that respond preferentially, not exclusively, for red, green and blue

• This trichromatic theory is not new; it dates back to the early 19th century (Young, 1802; Von Helmholtz, 1867). This theory, however, does not explain the odd effect that occurs when we look at a white wall after staring at a picture for around 30 seconds. (Canada flag green to red example)

• This is where the opponent-process theory comes in (Hering, 1920). This theory states that our cones send information to retinal ganglion cells that respond to pairs of colors (red-green, blue-yellow, black-white). These specialized cells take information from the cones and compute the difference between the two colors—a process that explains why we cannot see reddish-green or bluish-yellow, as well as why we see afterimages. Color deficient vision can result from issues with the cones or retinal ganglion cells involved in color vision.

Pitch and volume:

• People are capable of getting a large amount of information from the basic qualities of sound waves. The amplitude (or intensity) of a sound wave codes for the loudness of a stimulus; higher amplitude sound waves result in louder sounds. The pitch of a stimulus is coded in the frequency of a sound wave; higher frequency sounds are higher pitched. We can also gauge the quality, or timbre, of a sound by the complexity of the sound wave. This allows us to tell the difference between bright and dull sounds as well as natural and synthesized instruments

ear diagram

How Hearing works:

In order for us to sense sound waves from our environment they must reach our inner ear. Lucky for us, we have evolved tools that allow those waves to be funneled and amplified during this journey. Initially, sound waves are funneled by your pinna (the external part of your ear that you can actually see) into your auditory canal (the hole you stick Q-tips into despite the box advising against it). During their journey, sound waves eventually reach a thin, stretched membrane called the tympanic membrane (eardrum), which vibrates against the three smallest bones in the body—the malleus (hammer), the incus (anvil), and the stapes (stirrup)—collectively called the ossicles. Both the tympanic membrane and the ossicles amplify the sound waves before they enter the fluid-filled cochlea, a snail-shell-like bone structure containing auditory hair cells arranged on the basilar membrane (see Figure 4) according to the frequency they respond to (called tonotopic organization). Depending on age, humans can normally detect sounds between 20 Hz and 20 kHz. It is inside the cochlea that sound waves are converted into an electrical message.

How do we localize sound in a 3D space

Because we have an ear on each side of our head, we are capable of localizing sound in 3D space pretty well (in the same way that having two eyes produces 3D vision). Have you ever dropped something on the floor without seeing where it went? Did you notice that you were somewhat capable of locating this object based on the sound it made when it hit the ground? We can reliably locate something based on which ear receives the sound first. What about the height of a sound? If both ears receive a sound at the same time, how are we capable of localizing sound vertically? Research in cats (Populin & Yin, 1998) and humans (Middlebrooks & Green, 1991) has pointed to differences in the quality of sound waves depending on vertical positioning.

Cochlear nerve and higher brain centres:

After being processed by auditory hair cells, electrical signals are sent through the cochlear nerve (a division of the vestibulocochlear nerve) to the thalamus, and then the primary auditory cortex of the temporal lobe. Interestingly, the tonotopic organization of the cochlea is maintained in this area of the cortex (Merzenich, Knight, & Roth, 1975; Romani, Williamson, & Kaufman, 1982). However, the role of the primary auditory cortex in processing the wide range of features of sound is still being explored

Balance and the Vestibular System:

The inner ear isn’t only involved in hearing; it’s also associated with our ability to balance and detect where we are in space. The vestibular system is comprised of three semicircular canals—fluid-filled bone structures containing cells that respond to changes in the head’s orientation in space. Information from the vestibular system is sent through the vestibular nerve (the other division of the vestibulocochlear nerve) to muscles involved in the movement of our eyes, neck, and other parts of our body. This information allows us to maintain our gaze on an object while we are in motion. Disturbances in the vestibular system can result in issues with balance, including vertigo.

Touch

Our skin, the body’s largest organ, provides us with all sorts of information, such as whether something is smooth or bumpy, hot or cold, or even if it’s painful. Somatosensation—which includes our ability to sense touch, temperature and pain—transduces physical stimuli, such as fuzzy velvet or scalding water, into electrical potentials that can be processed by the brain.

Tactile Sensation

Tactile stimuli—those that are associated with texture—are transduced by special receptors in the skin called mechanoreceptors. Just like photoreceptors in the eye and auditory hair cells in the ear, these allow for the conversion of one kind of energy into a form the brain can understand.

somatosensory map

• After tactile stimuli are converted by mechanoreceptors, information is sent through the thalamus to the primary somatosensory cortex for further processing. This region of the cortex is organized in a somatotopic map where different regions are sized based on the sensitivity of specific parts on the opposite side of the body (Penfield & Rasmussen, 1950). Put simply, various areas of the skin, such as lips and fingertips, are more sensitive than others, such as shoulders or ankles. This sensitivity can be represented with the distorted proportions of the human body shown in Figure 5.

Pain

Most people, if asked, would love to get rid of pain (nociception), because the sensation is very unpleasant and doesn’t appear to have obvious value. But the perception of pain is our body’s way of sending us a signal that something is wrong and needs our attention. Without pain, how would we know when we are accidentally tou

Phantom Limbs

• Records of people experiencing phantom limbs after amputations have been around for centuries (Mitchell, 1871). As the name suggests, people with a phantom limb have the sensations such as itching seemingly coming from their missing limb. A phantom limb can also involve phantom limb pain, sometimes described as the muscles of the missing limb uncomfortably clenching. While the mechanisms underlying these phenomena are not fully understood, there is evidence to support that the damaged nerves from the amputation site are still sending information to the brain (Weinstein, 1998) and that the brain is reacting to this information (Ramachandran & Rogers-Ramachandran, 2000). There is an interesting treatment for the alleviation of phantom limb pain that works by tricking the brain, using a special mirror box to create a visual representation of the missing limb. The technique allows the patient to manipulate this representation into a more comfortable position (ie itch phantom limb)

Smell and Taste: (Connected)

• The two most underappreciated senses can be lumped into the broad category of chemical senses. Both olfaction (smell) and gustation (taste) require the transduction of chemical stimuli into electrical potentials. I say these senses are underappreciated because most people would give up either one of these if they were forced to give up a sense. While this may not shock a lot of readers, take into consideration how much money people spend on the perfume industry annually ($29 billion US Dollars). Many of us pay a lot more for a favorite brand of food because we prefer the taste. Clearly, we humans care about our chemical senses.

• During the process of eating we are not limited to our sense of taste alone. While we are chewing, food odorants are forced back up to areas that contain olfactory receptors. This combination of taste and smell gives us the perception of flavor. If you have doubts about the interaction between these two senses, I encourage you to think back to consider how the flavors of your favorite foods are impacted when you have a cold; everything is pretty bland and boring, right?

Olfaction (smell)

Unlike any of the other senses discussed so far, the receptors involved in our perception of both smell and taste bind directly with the stimuli they transduce. Odorants in our environment, very often mixtures of them, bind with olfactory receptors found in the olfactory epithelium. The binding of odorants to receptors is thought to be similar to how a lock and key operates, with different odorants binding to different specialized receptors based on their shape. However, the shape theory of olfaction isn’t universally accepted and alternative theories exist, including one that argues that the vibrations of odorant molecules correspond to their subjective smells (Turin, 1996). Regardless of how odorants bind with receptors, the result is a pattern of neural activity. It is thought that our memories of these patterns of activity underlie our subjective experience of smell (Shepherd, 2005). Interestingly, because olfactory receptors send projections to the brain through the cribriform plate of the skull, head trauma has the potential to cause anosmia, due to the severing of these connections. If you are in a line of work where you constantly experience head trauma (e.g. professional boxer) and you develop anosmia, don’t worry—your sense of smell will probably come back (Sumner, 1964).

Gustation (Taste)

Taste works in a similar fashion to smell, only with receptors found in the taste buds of the tongue, called taste receptor cells. To clarify a common misconception, taste buds are not the bumps on your tongue (papillae), but are located in small divots around these bumps. These receptors also respond to chemicals from the outside environment, except these chemicals, called tastants, are contained in the foods we eat. The binding of these chemicals with taste receptor cells results in our perception of the five basic tastes: sweet, sour, bitter, salty and umami (savory)—although some scientists argue that there are more (Stewart et al., 2010). Researchers used to think these tastes formed the basis for a map-like organization of the tongue; there was even a clever rationale for the concept, about how the back of the tongue sensed bitter so we would know to spit out poisons, and the front of the tongue sensed sweet so we could identify high-energy foods. However, we now know that all areas of the tongue with taste receptor cells are capable of responding to every taste

Multimodal Perception

• Though we have spent the majority of this module covering the senses individually, our real-world experience is most often multimodal, involving combinations of our senses into one perceptual experience. This should be clear after reading the description of walking through the forest at the beginning of the module; it was the combination of senses that allowed for that experience. It shouldn’t shock you to find out that at some point information from each of our senses becomes integrated. Information from one sense has the potential to influence how we perceive information from another, a process called multimodal perception.

• Interestingly, we actually respond more strongly to multimodal stimuli compared to the sum of each single modality together, an effect called the superadditive effect of multisensory integration. This can explain how you’re still able to understand what friends are saying to you at a loud concert, as long as you are able to get visual cues from watching them speak. If you were having a quiet conversation at a café, you likely wouldn’t need these additional cues. In fact, the principle of inverse effectiveness states that you are less likely to benefit from additional cues from other modalities if the initial unimodal stimulus is strong enough (Stein & Meredith, 1993).

• Because we are able to process multimodal sensory stimuli, and the results of those processes are qualitatively different from those of unimodal stimuli, it’s a fair assumption that the brain is doing something qualitatively different when they’re being processed. There has been a growing body of evidence since the mid-90’s on the neural correlates of multimodal perception. For example, neurons that respond to both visual and auditory stimuli have been identified in the superior temporal sulcus (Calvert, Hansen, Iversen, & Brammer, 2001). Additionally, multimodal “what” and “where” pathways have been proposed for auditory and tactile stimuli (Renier et al., 2009). We aren’t limited to reading about these regions of the brain and what they do; we can experience them with a few interesting examples including the McGurk Effect and Double Flash Illusion.

The smallest amount of stimulation needed for detection by a sense.

Absolute threshold

Loss of the ability to perceive stimuli.

Agnosia

Loss of the ability to smell.

Anosmia

Ability to process auditory stimuli. Also called hearing.

Audition

Tube running from the outer ear to the middle ear.

Auditory canal

Receptors in the cochlea that transduce sound into electrical potentials.

Auditory hair cells

Difference is images processed by the left and right eyes.

Binocular disparity

Our ability to perceive 3D and depth because of the difference between the images on each of our retinas.

Binocular vision

Building up to perceptual experience from individual pieces.

Bottom-up processing

Our ability to process the environmental stimuli of smell and taste.

Chemical senses

Spiral bone structure in the inner ear containing auditory hair cells.

Cochlea

Photoreceptors of the retina sensitive to color. Located primarily in the fovea.

Cones

Adjustment of eye to low levels of light.

Dark adaptation

The smallest difference needed in order to differentiate two stimuli. (See Just Noticeable Difference (JND))

Differential threshold

Pathway of visual processing. The “where” pathway.

Dorsal pathway

The combination of smell and taste.

Flavor

Ability to process gustatory stimuli. Also called taste.

Gustation

The smallest difference needed in order to differentiate two stimuli. (see Differential Threshold)

Just noticeable difference (JND)

Adjustment of eye to high levels of light.

Light adaptation

Mechanical sensory receptors in the skin that response to tactile stimulation.

Mechanoreceptors

The effects that concurrent stimulation in more than one sensory modality has on the perception of events and objects in the world.

Multimodal perception

Our ability to sense pain.

Nociception

Chemicals transduced by olfactory receptors.

Odorants

Organ containing olfactory receptors.

Olfactory epithelium

Theory proposing color vision as influenced by cells responsive to pairs of colors.

Opponent-process theory

A collection of three small bones in the middle ear that vibrate against the tympanic membrane.

Ossicles

The psychological process of interpreting sensory information.

Perception

Area of the cortex involved in processing auditory stimuli.

Primary auditory cortex

Area of the cortex involved in processing somatosensory stimuli.

Primary somatosensory cortex

Area of the cortex involved in processing visual stimuli.

Primary visual cortex

The finding that, in general, for a multimodal stimulus, if the response to each unimodal component (on its own) is weak, then the opportunity for multisensory enhancement is very large. However, if one component—by itself—is sufficient to evoke a strong response, then the effect on the response gained by simultaneously processing the other components of the stimulus will be relatively small.

Principle of inverse effectiveness

Cell layer in the back of the eye containing photoreceptors.

Retina

Photoreceptors of the retina sensitive to low levels of light. Located around the fovea.

Rods

The physical processing of environmental stimuli by the sense organs.

Sensation

Decrease in sensitivity of a receptor to a stimulus after constant stimulation.

Sensory adaptation

Theory proposing that odorants of different size and shape correspond to different smells.

Shape theory of olfaction

Method for studying the ability to correctly identify sensory stimuli.

Signal detection

Ability to sense touch, pain and temperature.

Somatosensation

Somatotopic map

Organization of the primary somatosensory cortex maintaining a representation of the arrangement of the body.

Changes in air pressure. The physical stimulus for audition.

sound waves

The finding that responses to multimodal stimuli are typically greater than the sum of the independent responses to each unimodal component if it were presented on its own.

Superadditive effect of multisensory integration

Experience influencing the perception of stimuli.

Top-down processing

Theory proposing color vision as influenced by three different cones responding preferentially to red, green and blue.

Trichromatic theory

Thin, stretched membrane in the middle ear that vibrates in response to sound. Also called the eardrum.

Tympanic membrane

Pathway of visual processing. The “what” pathway.

the ventral pathway

Parts of the inner ear involved in balance.

Vestibular system

States that just noticeable difference is proportional to the magnitude of the initial stimulus.

Weber’s law

what is masking?

• Masking is the process by which the presence of one sound makes another sound more difficult to hear. We all encounter masking in our everyday lives, when we fail to hear the phone ring while we are taking a shower, or when we struggle to follow a conversation in a noisy restaurant. In general, a more intense sound will mask a less intense sound, provided certain conditions are met. The most important condition is that the frequency content of the sounds overlap, such that the activity in the cochlea produced by a masking sound “swamps” that produced by the target sound. Another type of masking, known as “suppression,” occurs when the response to the masker reduces the neural (and in some cases, the mechanical) response to the target sound. Because of the way that filtering in the cochlea functions, low-frequency sounds are more likely to mask high frequencies than vice versa, particularly at high sound intensities. This asymmetric aspect of masking is known as the “upward spread of masking.” The loss of sharp cochlear tuning that often accompanies cochlear damage leads to broader filtering and more masking—a physiological phenomenon that is likely to contribute to the difficulties experienced by people with hearing loss in noisy environments (Moore, 2007).

• Although much masking can be explained in terms of interactions within the cochlea, there are other forms that cannot be accounted for so easily, and that can occur even when interactions within the cochlea are unlikely. These more central forms of masking come in different forms, but have often been categorized together under the term “informational masking” (Durlach et al., 2003; Watson & Kelly, 1978). Relatively little is known about the causes of informational masking, although most forms can be ascribed to a perceptual “fusion” of the masker and target sounds, or at least a failure to segregate the target from the masking sounds. Also relatively little is known about the physiological locus of informational masking, except that at least some forms seem to originate in the auditory cortex and not before (Gutschalk, Micheyl, & Oxenham, 2008).