Perception

Paradox of perception

There is much more information than we can actually process. At the same time, there is also not enough information to fully and unambiguously grasp the world (this is why we rely on assumptions and cues).

Perception seems easy

There is a vast array of complex dedicated computational machinery that makes this possible. Phenomenal experience of an activity as ‘easy’ or ‘natural’ often leads scientists to assume that the processes that give rise to it are simple.

Unconscious inference

involuntary, rapid mental process in which the brain uses prior experiences, context, and environmental cues to make sense of incomplete or ambiguous sensory information, such as interpreting depth, shadows, or hidden objects

-for example, the Ponzo illusion (railroad)

Ambiguity of brightness and depth

Depth is ambiguous without certain external cues! Both the big and small circle are imprinted on the retina in the same way so depth can only be discerned by cues like the leading lines.

This applies to brightness too. The two squares look like different colors but they’re actually the same color. The brightness and shadows make it ambiguous.

Cornea

transparent front surface of the eye that allow light in and help focus

Iris

colored part of the eye; controls light entering through the pupil

Lens

adjustable focusing elements of the eye - many transparent layers

retina

layer of cells at the back of the eye

cones

responsible for color vision; less numerous; concentrated in the fovea

rods

responsible for night vision and monochromatic vision; much more numerous; 100x more sensitive than cones; located in periphery of retina

backwards design of the eye

light is captured in the back of the retina which then goes forward through the ganglion and bipolar cells for information processing. This information is then sent to the brain.

lens focusing

First problem of light in the eye: the lens must focus the light on the back of the retina correctly

Many individuals struggle with myopia (nearsighted), hyperopia (farsighted), or astigmatisms

myopia

commonly known as nearsightedness, is a vision condition where you can see nearby objects clearly, but distant objects appear blurry

hyperopia

commonly known as farsightedness, is a refractive eye condition where distant objects are seen clearly, but close-up objects appear blurr

astigmatism

cornea or lens is irregularly shaped (more like a football than a round basketball). This causes light to bend unevenly, leading to blurred or distorted vision at any distance

blindspot

second problem of light in the eye: it is backwards!

-brain “fixes” it by filling in your perception with what it thinks is most likely in that spot

Caused by optic nerve which is where visual information exits the eye and travels to the brain

-the optic nerve has NO photoreceptors which leads to a _

fading

if something is faint and never changes, you stop seeing it (adaptation)

-blood vessels in your eyes are static and sit directly in front of the retina, the brain rapidly adapts and filters them from your conscious sight

motion induced blindness

stationary objects in a moving world tend to fade from our awareness, as our brain thinks those objects are in our eyeball, and not important for us to perceive

color afterimages

Staring at a color fatigues the specific cone cells in your retina responsible for it, causing your brain to see the inverse colors when you look away

photoreceptors, back, optic nerve, LGN, primary visual cortex

Pathway of visual information:

Enter the eye, captured by ? located at the ? of the retina. Passes through other cells in the retina (on/center, off/surround cells), then exits the eye via the ?. Travels through the _ (on/center, off/surround cells) and lands in ? (V1)

LGN cells

on-center off-surround cells = cells only fire when light shines on the middle of the receptive field (cells won’t fire if light shines on its entire receptive field)

off-center on-surround cells = only fires if light shines on perimeter ring of receptive field

primary visual cortex

aka V1

cells in V1 care about particular orientations of lines, in specific spots in the world.

V1 simple cells

orientation and position selectivity (predicted from arrangement of of off and on areas)

V1 complex cells

generalizes orientation selectivity over an extended region, unlike a simple cell that only responds to a bar in exactly the right spot

hypercomplex (end-stopped) cells

lines that end at exactly the right place codes stimulus size

-sensitive to orientation, motion, and direction as well

feature hierarchy

you can build a simple cell by making it fire if and only if all of the LGN cells are properly arranged

you can build a complex cell by making it fire if any of the simple cells, arranged in a line, fire

photoreceptors send light to retinal ganglion cells

LGN →simple cells→complex cells→hypercomplex cells

Area MT

specific area for smooth motion processing; treats moving objects different from static objects. Project motion about 100ms into the future

-for moving objects, area MT extrapolates motion into the future

Akinotopsia: MT can be damaged which selectively breaks motion processing

Motion in object determination

-Special motion processing machinery creates a strong presumption that things that move together are probably the same object. This especially applies if there is a rigid 3D shape that can explain the motion.

Depth

Most depth perception is monocular (from one eye)

Depth from shadow

Depth from perspective

Depth from occlusion

Depth from motion

Amodal completion

visual ability to perceive a whole, continuous object even when parts of it are hidden or occluded by another object in the foreground. The human brain automatically fills in the missing information so you perceive the object as a coherent shape.

Speed of object recognition

When hooked up to EEG, participants were asked if there was an animal in a picture

<150ms of visual processing (only time for a single sweep through of ~10 neurons)

Grandmother cells (feature hierarchy)

there is a detector for every different object in every pose and from every angle in every lighting condition… leads to computational explosion†

Geons/alphabet view

allow for viewpoint invariance and ability to identify many many object types in many different combos. a combination of component geons defines an object. just 3 neons can make 154 million objects. solves the computational explosion problem

ventral pathway

ventral, to inferior part of temporal lobe for object recognition

Lateral occipital complex (responds mostly to intact and lightly scrambled images that show object ‘parts’

-when damaged leads to visual agnosia (person can see but cannot visually recognize objects)

-can recognize object if they bypass their damaged LOC and pick it up

-disrupted LOC: no issue recognizing faces but struggle to recognize objects

“What?” for conscious perception and object recognition

dorsal pathway

dorsal, to parietal lobe for location, action, navigating, grasping

-parietal lobe responsible for landmark discrimination

Patient DF: damaged in ventral but not dorsal pathway. she could not name object (object agnosia) and also could not when objects placed in her hands

-suggests dorsal stream has it’s own shape analysis meant to accurately guide attention and actions

-patient DF was good at actions, not perception

ventral vs dorsal

temporal (ventral), dorsal (parietal)

TMS (transcranial magnetic stimulation)

giant magnet that disrupts neural firing temporarily. can allow researchers to localize function. researchers may selectively disrupt neurons. shows that face area and LOC have dissociable functions.

action patterns

my visual system perceives the world and as soon as I notice the stimuli appear, I begin moving my hand. There is a dynamic interaction between my visual system and accumulating evidence about where the items are and what they are and the motion of my hand.

Kitten Carousel (action and perception interplay)

tested if it mattered that your own actions cause your visual experience? is it enough to be a passive viewer?

-active cat saw normally with health visual development and depth perception

-passive cat was severely impaired and walked off cliff

Most movement on your retina is caused by your own motion. This shows the interplay of action and perception. It reveals how important it is that we’re in control of the visual motion we see.

Dissociation of perception and action

Even when people perceived the center disc as a different size, their fingers still opened to the correct actual size when they reached out to grasp it.

It supports the idea that perception and action use somewhat different visual processing systems. The system you use to consciously judge what something “looks like” can be fooled by context, while the system guiding your grasp is more tuned to the object’s real physical size.

Signal detection theory

It is far more consequential to overlook the face of a predator in the wild than it is to misinterpret/false flag a nonface. This shifts the decision point to the left so we may have more false alarms but at the benefit of never missing faces (less likely to be eaten)

FFA (fusiform face area)

responsible for the holistic (configural) information, meaning that it puts all of the processed pieces of the face together

-processes and judges the spacing and configuration of the parts of the face to determine identity

-ONLY WORKS FOR UPRIGHT FACES!!! (Margaret Thatcher illusion)

-Inversion screws up the FFA

OFA (occipital face area)

recognizes parts of the faces at the early stages of recognition

-primarily processes basic facial features (like eyes, nose, and mouth)

fSTS (superior temporal sulcus)

involved in perception of gaze and biological motion

-plays a major role in detecting where others are looking (averted vs. direct gaze)

-STS has abnormalities in autism which leads to less eye contact

innate face perception

Babies only hours old have preferences for faces and face-like stimuli over many other kinds of stimuli

Testing face-specific brain regions

-OFA, use a task where people detect or identify individual face parts, like matching eyes, noses, or mouths, without needing to recognize the whole person.

-FFA, use a task requiring identity recognition or spacing/configuration judgments, such as recognizing the same person across images, using upright versus inverted faces

-fSTS, use a task involving gaze direction, such as deciding whether someone is looking left/right or watching dynamic facial movements.

Attention is capacity limited

brain's remarkable ability to focus your auditory attention on a single conversation while filtering out a cacophony of background noise and other chatter

Attention is selective

we have to select what we attend to

-may feel like we can see this whole picture but we actually can’t process everything about this image at once

-“how many ducks are there?” cant be answered without directing our attention selectively

Cocktail party effect

you only hear your own conversation. You only see what you explicitly attend

Inattentional blindness

(Gorilla illusion) when we attend one thing, we often miss other things. Especially if they are unexpected. We are ‘blind’ to anything not within the focus of our attention

Attentional set

-Cognitive filter that biases your brain to focus on specific features, stimuli, or rules in your environment while actively ignoring others

groups that were told to look for white shapes were more likely to see white cross. Groups that were told to look for black shapes were more likely to see black cross.

-more likely to see things that align with what we’re looking for

Experts and attention

Even radiologists missed the gorilla image because they were focused on the white cancerous nodules

Involuntary attention

some features or events grab our attention reflexively. Attention may also be involuntary

-a loud sound may make us involuntarily orient towards this sound

Voluntary attention

focusing on something in a controlled manner; even without moving your eyes

-visual search (where’s Waldo?)

Covert attention

moving our attention without moving our eyes (object of your attention is in your peripheral vision)

Overt attention

eyes (LOS) align with what we’re attending to

Attention changes neural processing

When you attend to something, your brain devotes a lot of cognitive resources towards processing that thing…which means that other stuff in our visual field doesn’t get processed as well

Attention module V4 activity

-researchers recorded from a single neuron in V4 (likes horizontal red lines and not vertical green lines)

-within borders of V4 neuron’s receptive field, a horizontal red line and vertical green line

-when attending red bar, neuron responds a lot even if green bar is still in its receptive field

-when attending to green bar, neuron responds way less even if red bar is still in its receptive field

-when attention is not directed to either bars, there’s an intermediate/moderate neuron response

-neuron’s response is biased towards attended stimulus (individual neuron response is impacted by attention)

Pop-out search

only need to pay attention to one feature (color only suffices)

Conjunction search

need to pay attention to two features (color AND orientation)

Feature integration theory

attention is necessary to integrate or bind multiple features together

Ensemble perception

brain's ability to rapidly extract summary statistical information (such as the average size, average facial expression, or overall "gist") from a group of similar objects, rather than processing each item individually

Visual crowding

you can shrink the area of selection to be small but not as small as the smallest detail you can see

-It’s very hard to focus on the center dot and count the # of lines on the left

Low vs high prevalence

-TSA agents are more likely to find threats when prevalence is high

-when prevalence is low, miss rate increases and false alarms are more rare

-when prevalence is high, miss rate decreases and false alarms are more common

Signal detection theory in low prevalence

In low prevalence, the decision point shifts to the right and there are more misses (more threats flagged as safe) and fewer false alarms (safe bags flagged as threats)

-this is because in low prevalence, the standard for what’s a threat increases

Physical definition of sound

sound is pressure changes in the air (or other medium)

Perceptual definition of sound

sound is the experience we have when we hear

Amplitude

Lower amplitude = quieter sound

Higher amplitude = louder

Pitch and frequency

Lower freq = lower pitch

Higher freq = higher pitch

Pinna

collects sound & funnels it

into external auditory canal

• Helps with sound localization

Ear canal

conducts sound from pinna

to tympanic membrane

Tympanic membrane (aka eardrum)

elastic sheet that vibrates in response to

sound coming through external auditory

canal

• Damaging the membrane can lead to

hearing loss

Ossicles

amplify sound by concentrating energy from larger to smaller surface areas: Necessary so that air vibrations in outer and middle ear can lead to fluid vibrations in inner ear

Eustachian tube

connects the middle ear with the pharynx and helps equalize air pressure on either side of the tympanic membrane

tectorial membrane and basilar membrane

move in response to the waves caused by the liquid in the cochlea which cause stereocilia to move and bend. causes action potentials and the release of neurotransmitters

stereocilia

tectorial membrane and basilar membrane

move in response to the waves caused by the liquid in the cochlea. This causes stereocilia to move and bend. This

movement causes action potentials and the release of neurotransmitters

Cochlear nerve

The cochlear nerve (the auditory nerve) converts sound vibrations into electrical impulses and sends these action potentials to the brain.

Base

stiff, high frequencies (as we age, we lose higher frequencies)

-Different locations along the basilar membrane respond to different frequencies

Apex

floppy, low frequencies

-Different locations along the basilar membrane respond to different frequencies

Tonotopic organization

Neurons are organized according to

the frequencies to which they respond

Hierarchal organization of auditory cortex

A1 responds broadly to sound, while higher auditory areas like the belt and parabelt respond more to complex tones and meaningful sounds, such as speech

“what” auditory pathway

helps identify sounds, like recognizing speech, music, or a dog bark

“where” auditory pathway

helps locate where a sound is coming from

Binaural cues

Azimuth (left-right)

Monaural cues

Elevation (up-down) & distance

Interaural time difference

Uses the difference in time it takes sound to reach the left and right ears to localize sound

useful for low frequencies where there is a clear difference in time

-particularly effective for low frequency

sounds that are coming from the left or

right of us

Interaural level difference

difference in sound level (volume) that reaches the two ears (because your head blocks sound) to localize sound

-Higher frequencies are more disrupted by the head than lower frequencies (so it’s easier to locate high frequency sounds using interaural level difference cues!)

Auditory continuity

When a brief, louder sound (like a tone or a burst of white noise) completely interrupts a softer, continuous sound, the brain automatically "fills in" the missing acoustic information. As a result, you perceive the softer sound as playing continuously underneath the louder one.

Phonemic Restoration Effect

When our mind uses context to ‘fill

in’ missing sounds in speech (phonemes)

Auditory transduction pathway

Information travels along the auditory nerve to the:

Cochlear nucleus: Determines the

direction of sound and only receives info from one ear

The superior olive: Receives info from both ears, helps to determine sound localization

Next, information passes through the

inferior colliculus and the medial geniculate nucleus (MGN) (in the thalamus!)

MGN relays information to primary auditory cortex in the temporal lobe