Perception: Object Recognition

Template theory

Matching the neural representation of an image with an internal representation of the same “shape” in the brain

Storage → Fixed templates for each object

Recognition process → Direct matching to a single internal representation

Flexibility → Limited (sensitive to variations)

Scalability → Requires many templates for different views

Dog example → Matches input to a stored dog template (e.g., a side-view of a Labrador)

Exemplar Theory

Comparing the neural representation of an image to multiple stored examples rather than a single template

Storage → Multiple stored examples (exemplars)

Recognition process → Comparison with multiple previously seen instances

Flexibility → High (handles variability well)

Scalability → Stores many exemplars but generalizes well

Dog example → Compares input to multiple stored dog examples (various breeds, angles, and contexts)

Prototype theory

people form an average of the category of objects they have experienced (prototype represents the “best” example of a given category)

Storage → A single, abstracted prototype per category

Recognition process → Matching to the most representative prototype

Flexibility → Moderate (allows for some variation but relies on an “average”

Scalability → Requires storing only one prototype per category

Dog example → Compares input to an idealized “average dog” that represents the category

Generalize Context Model (CGM)

You store many specific faces you’ve seen before, when you see a new face: compare to the stored examples and assign a category

General Recognition Theory

You rely on perceptual dimensions (e.g., face shape, jaw, eye size) and decide based on the statistical boundaries between two categories

Recognition by components

We recognize objects using an alphabet of shape (36 geons - combined can form any given object)

Grandmother cell theory

We have one single neuron for every concept that exists in the physical world

• Meant to highlight the absurdity of the idea that individual concepts are attached to specific cells

  • Jennifer Anniston cell discovered in the 2000s

Deep neural network (DNN)

A multilayer network that can be trained to recognize objects - over time learned to recognize new instances of an object it has not been trained on

Deep learning in object recognition

Representations inside DNN’s comparable to inferior temporal cortex in monkeys on an object recognition task

Some layers of the model could predict whether you will see an image presented very quickly to you (attentional blink

Recognizing objects

Retinal ganglion cells and lateral geniculate nucleus detect spot (localized contrast), the primary visual cortex detects edges and bars (orientation selectivity)

Intermediate-level vision

V2, V3, V4, …

Groups features into contours, textures, surfaces

• receptive fields of extrastriate cells respond to visual properties crucial for object perception

High-level vision

Inferior temporal cortex

Recognizes complex shapes, objects, categories

Intermediate-level vision functions

Loosely defined stage of visual processing that occurs after low-level feature extraction (edges, contrast) but before high-level object recognition/scene understanding

• Perception of edges and surfaces

• Determining which regions of an image should be grouped

• Bridging low-level feature detection and high-level object recognition

Gestalt Theory

Suggests that perception is holistic – we naturally organize elements into meaningful wholes rather than processing each part independently

The “committee” metaphor for how perception operates:

• Committees must integrate conflicting inputs to reach consensus

• Many different and sometimes competing principles influence perception

• Perception emerges from the dominant interpretation agreed upon by these processes

Grouping Similarity

Similar objects (color, shape, size) appear grouped

Grouping proximity

Elements close to each other tend to be grouped

Grouping good continuation

Lines and edges are perceived as following the smoothest path

Grouping closure

The mind fills in missing information to perceive complete shapes

Grouping common fate

elements moving together are grouped

Grouping figure ground

The brain separates objects from the background

Grouping common region

Elements locaed within a shared boundary or enclosed areal are perceived as a group

Grouping connectedness

elements visually connected by lines tend to be grouped

5 principles on intermediate vision

1. Group what should be grouped together

2. Separate what should be separated

3. Use prior knowledge (constantly compare what we are experiencing now to that stored internal model)

4. Avoid accidents

5. Seek consensus and minimize ambiguity

Dorsal stream

• “Where” pathway

• Processes shapes and locations of objects, but does NOT encode object names and functions

  • Extends from the occipital lobe to the parietal lobe

Ventral stream

• “What” pathway

• Processes object names and functions (no matter where they are located)

• Stems from the occipital lobe to the temporal lobe

• As we move from V1 through to V4 – neurons respond to more abstract/complex information

Posterior IT (PIT)

starts combining features/parts of objects

• Responds to key components, does not require the full object to be present

• Becomes a key transition area between V4 and anterior IT (AIT) that

  encodes whole objects

Lateral occipital complex (LOC)

First stage in visual hierarchy, where full objects are explicitly represented

• Bridges mid-level feature processing with high-level object recognition

• Supports invariant object recognition, which is important for recognizing

  objects across different contexts

Fusiform Face Area (FFA)

Highly tuned to faces – could say category preferential but not selective (prefers responding to faces, but also responds to other object categories)

• Located in fusiform gyrus of central temporal lobe

• Helps recognize faces across different angles, lighting, expressions

• Damage to FFA linked to prosopagnosia

Parahippocampal Place Area (PPA)

Responds preferentially to places

• Dedicated scene-processing region – challenged the idea that object recognition alone explains scene perception

Decoding method

• Collect fMRI scans of participants viewing images from known categories

• Train computer model to recognize brain activity patterns associated with each

category

• Can it decode what the brain just saw?

Encoding method

• Collect fMRI scans of participants viewing images from known categories

• Train computer model to predict responses to new, unseen stimulus

• Is the predicted brain activity correlated with actual brain activity?

Second order isomorphism

Similar objects in the physical world have similar representations in the mind