Perception exam

only the final chapters - no cumulative info

define an event

segment in time at a particular location that observers perceive as having a definable start/end point

* event boundary = the point in which one event ends and another begins

Research by Heider + Simmel 1994 that sows the benefit of motion in perception

Demonstrated that motion alone can be enough to provide interesting narratives too simple geometric shapes that appear to be interacting with one another

why is motion important to perception

* provides information about objects
* attracts attention
* allows us to take action

Research by Hamlin, Wynn and Bloom (2007) that shows importance of motion to perception

6 - 10month old infants are shown videos of shapes helping or Hindering

Infants preferred helping shapes

Suggesting that the motion add meaning

Akinetopsia

Akinetopsia is a rare neurological disorder that results in the inability to perceive fluid motion

* Inconspicuous Akinetopsia
* Gross Akinetopsia

Inconspicuous Akinetopisa

experience motion as a series of still snapshots

* common form of akinetopisa
* persistant afterimages

Gross Akinetopisa

experience the world as snapshots of objects when they are still - unable to see any form of motion

* rare form of akinetopsia
* caused by damage to the middle temporal area of the brain, which is responsible for processing motion information.

Patient LM

LM has bilateral lesions that destroyed areas V5/MT

saw world in snapshots

couldn’t cross the street - can not perceive the motion of cars

all known info about of akinetopsia is because of LM

under what conditions do we perceive real motion

The perception of real motion occurs when something moves across our field of vision

under what conditions do we perceive illusionary motion

the perception of illusionary motion occurs when perceive motion but the stimuli aren’t actually moving

* there are 3 types of illusionary motion - Apparent motion, Induced motion, Motion after effects

illusory motion: apparent motion

the illusion of motion resulting from a sequence of still images

illusory motion: induced motion

Occurs when motion of one object (like a background) causes another object to appear to move or move in a different way than it actually is moving.

* can occur with a stationary or moving target

induced motion with a stationary vs moving target

stationary = creates the illusion of motion in the stationary target

moving = creates the illusion that the target is

moving faster or in a different direction than it actually is

illusionary motion: double drift

The double drift illusionary motion is a visual illusion where two superimposed gratings drift in opposite directions, creating the perception of a single grating moving back and forth.

illusory motion: after effects

Create the perception of motion where there is none

* occur when the brain continues to perceive motion after the stimulus that caused the motion has stopped.
* The perception of motion in after effects is an illusion created by the brain, rather than an actual physical movement.

apparent motion and tv/movies

Our brains can recognize 10-12 frames per second (fps) as separate images.

* Early movies had frame rates between 16-24 fps, but lower frame rates were perceived as jerky movement.
* Many early television shows used 30fps, which is one reason TV shows “look/feel different” from movies.
* Modern movies have a frame rate standard of 24 frames/second - “cinema look”
* screens also have “refresh rate”

Why does Gemini Man look strange/amateurish compared to other movies?

The movie was shot at 60fps, which reduces motion blur

* cinema look = 24 fps

refresh rate

When screen is being “redrawn” whenever there is motion/changes to the display.

* This results in a flicker where the pixels are turned off and then on again in a new arrangement

critical flicker fusion

where flicker lights are perceived as a continuous light, range from 35 – 60hz, depending on a number of factors.

games vs movies Why games aim for

higher FPS/Hz - psychological reasons

Gamers often prefer higher monitor refresh rates of 60HZ (60fps) to 144HZ (144fps)

* We’ve been conditioned to find 24fps “cinematic” - The Hobbit was filmed at 44fps and reviewers found it “jarring” and “hyper-realistic” which took them out of the movie experience.


* We have the opposite expectation with games -many want the games to feel as real as possible.

games vs movies Why games aim for

higher FPS/Hz - technological reasons

Gamers often prefer higher monitor refresh rates of 60HZ (60fps) to 144HZ (144fps)

* Movies are passively consumed, while games are interactive.
* Higher FPS allows less of a delay in reaction time and your “movements” to be translated to the screen.
* Less lag in the response from the game control to be rendered on the screen

Two Situations where motion is perceived

1. The eyes move to follow a moving object and the moving object’s image is stationary on the retina.
2. The eyes are stationary and the moving object moves across the retina.

J. J. Gibson (ecological approach)

Gibson argued that perception evolved so that we can move within, and act upon, the world.

* He thought that perception was best studied in the natural environment, rather than artificial settings within laboratories.
* “Perception is based on information, not sensation”

not enough sensory info in the environment

top down processing

Gregory

enough sensory information in the environment

direct perception

* perception & action

Gibson

too much sensory information in the environment

selective attention

Triesman

motion perception - ecological approach (Gibson)

Motion perception is based on info gathered from the Optic Array - motion is perceived based on the local disturbance in the optic array.

Movement through the world creates “Optic flow”

The point to where you are moving is called the Focus of Expansion (F.O.E.).

why we don’t perceive motion when we scan a scene or walk through our environment and objects move across the retina (gibson)

When there is global optic flow, where everything moves across the retina, the objects in the optic array do not appear to be moving.

according to gibson motion is..

* Motion is perceived when part of the optic array moves and other parts remain still
* Motion is not perceived when 1) all parts of the optic array move (in the same direction) or 2)none of the optic array moves.

The Ecological Approach: Limitations

* The ecological approach ignores Topdown influences and doesn’t really offer a physiological explanation.
* The Ecological Approach does not try to explain illusions, and dismisses them as artificial phenomenon.
* It may also have difficulty explaining more common things like movies being perceived as “real motion”.

Corollary discharge theory

movement perception depends on three signals:


1. Image displacement signal (IDS): movement of image stimulating receptors across the retina.
2. Motor signal (MS): signal sent to eyes to move eye muscles.
3. Corollary discharge signal (CDS): copy of the motor signal
* A mechanism called the “comparator” compares the signals that have been sent from the retina and eye muscles.

According to Corollary discharge theory potion is perceived when

Motion is perceived when the comparator receives a corollary discharge signal (CDS) or Image displacement signal (IDS).

According to Corollary discharge theory potion is NOT perceived when

Motion is not perceived when the comparator receives both the corollary discharge signal (CDS) and the image displacement signals (IDS) at the same time

Corollary Discharge Theory - When you keep your eyes still

As an object moves across the retina, the IDS is sent to the comparator.

Corollary Discharge Theory - When you track a moving object

If you follow the object with your eyes, to image is stationary, the CDS is sent to the comparator

Neurons sensitive to motion

complex cells

* receptive field is sensitive to direction and orientation of a bar of light moving across the RF.

The Reichardt Detector - neural circuit sensitive to rightward motion

The output unit will only send a motion signal if it receives information from A and B simultaneously.

* motion to the right will create a signal, motion to the left will not.


* The circuit can also be sensitive to different speeds, depending on how long the delay unit delays the signal.
* can cover large areas

why can the reichardt detector cover large areas

The Receptive fields of cells get larger as we move on from early stages of visual processing. Cells in the later stages of visual processing probably consider information from several “simpler” Reichardt detectors

The Brain Area for Motion

The Middle Temporal Cortex (V5/MT)

The MT cortex and motion: Lesioning and Transcranial Magnetic Stimulation

* Lesioning the monkey MT cortex increases the coherence threshold to detect the direction of motion.


* Applying TMS to the human MT cortex impairs the ability to discern direction of movement in random dot patterns.
* Applying TMS to the human MT cortex induces a form of temporary Akinetopsia.

Stimulating direction sensitive neurons in the MT cortex - Britten et al. (1992)

Microstimulation: Small wire electrode causes specific neurons in the cortex to fire.

* Stimulating neurons in MT that are sensitive to downward motion shift the monkey’s responses from “to the right” to “downward and to the right”.

Motion signaled by single neurons can be ambiguous. The Aperture Problem

Observation of small portion of larger stimulus leads to misleading information about direction of movement.

* Activity of a single complex cell does not provide accurate information about direction of movement.
* Several receptive fields combine their information - Hypercomplex (endstopped) and complex cells pool their responses

Single Neurons as a solution to the Aperture Problem

* The ends of the objects help determine movement direction.
* Some cells are selective for this type of stimulus (i.e., end-stop cells/hypercomplex cells).
* A number of neurons pool their responses to motion to determine the overall characteristics of the motion being perceived.
* Also, not all stimuli are uniform in colour and shape.

Motion and The Human Body: Apparent Motion

* Shortest Path Constraint: Apparent motion occurs along the shortest path between two stimuli.

The Human Body and Apparent Motion: Shiffrar & Freyd (1990, 1993)

1. The visual system needs time to process the motion of meaningful complex stimuli and
2. Motion produced by the human body may be special.
* Other objects do not demonstrate a change in motion path contingent on rate of alternation.

fMRI results of Stevens et al. (2000)

showed mirror neurons in the motor cortex only activated when participants perceived movement around the head.

Motion and The Human Body: Point Light Walkers

Biological Motion: motion of a person or other living organism.

* Point-Light stimuli can be used to isolate the movement from the organism.
* The dots become perceptually organized and recognized as biological motion when they move (Gestalt principle of common fate).
* We are good at recognizing actions presented with Point-Light stimuli because of our everyday experience with it

Motion and The Human Body: Brain areas for biological motion - Grossman & Blake (2001)

The Superior Temporal Sulcus (STS) was more active when viewing biological motion as compared to scrambled motion.

* Areas that contain mirror neurons are also involved in the perception of biological motion (i.e., the FFA and parts of the Prefrontal Cortex).
* The striate cortex, medial temporal area, medial super temporal area also have roles in motion perception

Grossman et al. (2005) - is motion biological or scrambled

Noise was added to dots so they could only achieve 71% accuracy.

* Transcranial magnetic stimulation applied to STS caused a decrease in the ability to detect biological motion.

The Perception of Motion in Still Photos

* Implied motion occurs when a still image depicts an action involving motion.
* Representational Momentum: the idea that motion depicted in a picture tends to continue in the observer’s mind.

Kourtzi and Kanwisher (2000) - Implied Motion and Representational Momentum

had participants view different type of pictures, some with implied motion.

* Implied motion cause a greater degree of responding within the MT cortex.

Implied Motion and Motion After Effects - Winawer et al. (2008)

hypothesized that still images with implied motion might produce a motion after effect if MT neurons are involved in the perception of the images.

* Had participants adapt to the images before presenting them with moving dot displays.
* After viewing implied rightward motion, participants were more likely to say the dots were moving to the left.
* This is believed to be for the same reason as the waterfall illusion (neural adaptation).
* supports the idea of unconscious inference, Bayesian inference, or Predictive coding.
* Your brain appears to be running simulations.

Motion Illusions based on peripheral drift: Fujiwara’s Black hole illusion

Peripheral drift is an optical illusion that occurs when stationary objects appear to move due to the arrangement of their surrounding patterns.

* In the Black Hole Illusion, a spiral pattern is used to create the illusion of a rotating black hole. The peripheral drift effect causes the spiral pattern to appear to rotate, giving the impression of a black hole sucking in everything around it.

Rotating snakes and saccadic eye movements Otero-Millan et al. (2012)

Otero-Millan et al. (2012) measured blink-rate and saccadic eye movements while participants viewed two displays” A) illusory motion B) real rotation

* Participants indicated when they perceived motion by releasing a button and when the motion stopped by pressing the button
* There was a large increase in saccadic eye movement and blink rate prior to perceiving the illusory motion in the “rotating snakes illusion”
* There was no such increase prior to the perception of real motion in the non-illusion stimulus

Luminance profiles and stimulus structure: Fraser and Wilcox (1979)

For each section of the illusion, luminance gradually increases and then drops to black.

* The direction of motion tends to go from black to dark grey and from white to light grey.
* Staggered gradience  creates more motion – even more with rounded edges
* Secede eye movement creates some of the motion

hearing basics

Hearing provides information when vision might not be helpful

Sound is Caused by Vibrations

Sound occurs because of change in air pressure - your voice is moving air - brain puts the sound there

sound vs light

Sound must travel through a medium (i.e., air molecules, water molecules, etc.) and cannot exist in a vacuum.

* Light can travel through a vacuum. If light does not encounter obstacles, like in a vacuum, it will travel forever.
* Both light and sound will lose energy when travelling through a medium.

sound

the stimulus of hearing

* can be understood as either a physical stimulus or a perceptual response.

perceptual definition of sound

\
Sound is the experience we have when we hear. 

Physical definition of sound 

Sound is pressure changes in the air or other medium.

describing the physical aspects of sound

Sound wave: a change in air pressure patterns that is ultimately received by the ear. Travels 340 m/sec through air and 1500 m/sec through water. 

Sound waves are described in terms of their: 


1. Frequency: The number of cycles per second that the pressure changes repeat. 
2. Amplitude: The size of the pressure change. 
* Periodic vs Aperiodic Sound Waves

sound: frequency

The number of cycles per second that the pressure changes repeat.

* Frequency is measured in Hertz (Hz).1Hz = 1 cycle per second
* Frequency is associated with the pitch of a sound. Higher frequency = Higher pitch

sound: amplitude

refers to the distance from “Rest to Crest”. It can also be measured from rest to trough.  

* Amplitude is associated with the loudness of a sound.
* Higher amplitude = Louder sound.
* decibles

amplitude - decibles

Decibels are used because the range of possible amplitudes are too large and plotting differences between soft and loud noises would be impractical in other units of measurement.

* Cant hear anything = 0 decibels
* Average speaking voice - 60 decibels
* Pain = 120 decibels - start to feel sound - can start to cause damage

Sound: Periodic vs Aperiodic Sound Waves

Periodic waveforms repeat a pattern, aperiodic waveforms do not. 

pure tone

any tone that can be represented by a sine wave.

* periodic


* rare in the environment. Most periodic tones are actually 

complex tones

created by combining multiple pure tones. 

* Most periodic tones are actually complex tones.

Complex Tones: The Fundamental Frequency

The Fundamental Frequencies equal to the greatest common divisor of each of the pure tones that make up the complex tone.

* Lets say we had a complex tone with 200Hz, 400Hz, 600Hz, and 800Hz - The fundamental frequency is 200Hz.

Complex Tones: Harmonics

\
**First Harmonics** a pure tone equal to the fundamental frequency. 

* All others are called **Higher Harmonics**, which are whole number multiples of the fundamental frequency.
* An **Overtone** is any tone with a frequency greater than the fundamental frequency and may or may not be a whole number multiple.    

Representing a Complex tone with a Frequency Spectra

The Frequency Spectra(to the right of the waveforms) show us the frequencies that are present in the tone, as well as the amplitude of each tone.

Perceptual Aspects of Sound 

* Loudness
* Pitch
* Timbre

perceptual aspects of sound: Loudness level

the perceptual quality related to level or amplitude.

* There is a not a linear relationship between the sound stimulus intensity (dBs) and its perceived loudness. 
* Response Expansion occurs when increasing the intensity of a pure tone. 

loudness across the frequency rage: the audibility curve

\
Threshold of hearing: The point in which a sound can just be heard. 

Auditory response area:The light green area showing sounds that can be heard by humans.

Equal loudness curves indicate the sound levels that produce the same perception of loudness at each frequency. 

Threshold of feeling: The point where you can “feel” sound. 

perceptual aspects of sound: pitch

the perceptual property of sound that we describe as high or low

* tone height & tone Chroma
* the missing fundamental effect

Pitch: tone height

\
The perceptual experience of increasing pitch that accompanies increases in a tone’s fundamental frequency.

Pitch: Tone Chroma

Notes with a similar sound that can be grouped together (i.e., A0, A1, A2, A3, etc.)

pitch: the missing fundamental effect

\
occurs when your brain perceives a sound that is lower than any of the frequencies that are present in a complex wave. 

* Problem: Cell Phones only register sounds that are above 300Hz and the fundamental frequency of most adult speaking voices is lower than 300 Hz (male = 85Hz to 180Hz; Female = 165Hz to 255Hz). 

solution to the problem in the missing fundamental effect

Your brain provides the missing fundamental by determining the greatest common factor of the overtones and harmonics present. 

* Example 1: 200Hz, 400Hz, 600Hz, 800Hz, 1000Hz Fundamental frequency = 200Hz


* Example 2: 400Hz, 500Hz, 600Hz, 700Hz, 800Hz, 900Hz, 1000Hz Fundamental Frequency = 100Hz

\

perceptual aspects of sound: timbre

all other perceptual aspects of a sound besides loudness, pitch, and duration

* Timbre is determined by the specific combination of Harmonics produced by each instrument.


* Removing higher harmonics (or adding them) does not affect pitch of a tone, but it does affect its timbre. Example: 200Hz, 400Hz, 800Hz, 1000Hz
* One way that timbre is determined is by the specific combination of higher harmonics present in the complex tone. 

parts of the outer ear

Malleus

pinna

auditory canal

Parts of the middle ear

ossicles: 3 bones in the middle ear


1. malleus
2. incus
3. stapes

oval window

round window

parts of inner ear

semicircular canals - 3 chambers

auditory nerve

cochlea

malleus (hammer)

\
transmits vibrations from the tympanic membrane to the Incus.

incus (anvil)

\
transmits vibrations from the malleus to the stapes.

stapes (stirrup)

\
transmits vibrations to the inner ear by pushing on a membrane covering the oval window.

oval window

\
a membrane covered opening through which vibrations are transmitted to the inner ear from the middle ear. 

round window

A type of pressure relief for the inner ear. 

cochlea

hollow, liquid filled, space in the temporal bone.

It has 3 chambers: scalavestibuli, scalamedia/cochlear duct, scalatympani (blue arrows) 

* Uncoiled the cochlea would be approximately 2mm in diameter and 35mm long.


* Coiled, it makes approximately 2 . turns around the modiolus (central axis of the cochlea). 

\

cochlea chamber: scalavestibuli 

\
begins at the oval window and ends at the apex, where it meets with the scalatympani.

cochlea chamber: scalamedia/cochlear duct 

\
contains the cochlear partition(*basiliarmembrane* and *organ of corti*).

cochlea chamber: scalatympani

\
begins at the apex and terminates at the round window.

The Outer, Middle, and Inner Ear

\
When the stapes pushes on the oval window, it creates a wave that travels through the liquid of the cochlea. 

* This wave causes the basilar membrane to move up and down and the tectorial membrane to move side to side. 

Transduction of waves to electrical signals: The Organ of Corti - parts

Cilia

Inner “hair cells”(approx. 3500)

Outer “hair cells” (approx. 12,000)

Transduction of waves to electrical signals: The Cilia

\
The cilia (“hairs”)are connected by structures called Tip Links.

* Movement towards the tallest cilia causes depolarization and movement towards the shortest cilia causes hyperpolarization. 

Vibrations from the Basilar Membrane cause excitatory and inhibitory responses. 

\
The tallest row of cilia (of only the outer hair cells) are embedded into the tectorial membrane. 

* When the basilar membrane moves upward, the cilia are bent into the excitatory position; cell firing increases.
* When the Basilar membrane is at rest, the cilia are not bent; the hair cells fire at baseline.
* When the basilar membrane moves down, the cilia are bent to the inhibitory position; cell firing is inhibited. 

The Hair Cells Fire in Timing with the pure tone

\
Phase Locking: The hair cells fire in timing with the rise and fall of pressure of the pure tone

How the brain makes sense of frequency

* Lower frequencies cause lower rates of firing, while higher frequencies cause faster rates of firing. 
* The refractory period of hair cells create an issue for tones that are higher frequency. 
* This is resolved by having a large number of hair cells all firing at different peaks of the tone.  

How is Loudness Encoded? Explanation 1

intensity is encoded by the firing rate - loudness is just intensity

* Larger amplitude waves cause greater displacement of the hair cell cilia, causing greater depolarization. This leads to an increase in the amount of neurotransmitter released. 
* More neurotransmitter = higher rate of fire in the spiral ganglion cell during each peak of the sound wave travelling along the cochlea.     

\
How is Loudness Encoded? Explanation 2

encoded by the number of hair cells that are stimulated at a particular point on the cochlea. 

* Increasing amplitude cause a larger area of the cochlea to vibrate as the amplitude (peak of the vibration) increases. 
* This results in more hair cells being stimulated by a greater degree. 

How is Frequency encoded?  Georg von Bekesy (1899-1972) 

Vibration of the Basilar Membrane

* different frequencies made different areas of membrane vibrate 
* higher frequency vibrated the base and low frequency vibrated the apex

Awarded the Nobel prize in 1961 for his research on the cochlea. 

How the Basilar Membrane Vibrates to different Frequencies

* The base of the basilar membrane is more narrow and stiff, and will only vibrate to high frequency sounds.
* The apex is wider and more flexible, and will vibrate to low frequency sounds. 
* like a reverse piano