Kap. 3 - Perception and Mental Imagery

What does it mean to say that perception is a “construction of the mind”?

• What is the distinction between top-down and bottom-up processing?

Perception

• “The ability to recognize and interpret information from the senses“

.

Construction of the mind

• “The very act of perceiving involves making sense of the raw stimulation conveyed by our senses“

  • Making sense by:

    • piecing together a coherent, stable, mental representation

    • 
      

      

      
      

• The mind fills in missing pieces

  • Like perceiving the whole picture with only a few puzzle pieces

• E.g.

  • Modal completion

    • Where view of the illusory object is not obstructed, and there is no objective boundary between the illusory object and the background

    • a)

  • Amodal completion

    • See/to perceive an object despite an apparently obstructed view

    • b)

  • 
    

.

Top-down processing

• Knowledge and expectations influence and enhance our interpretation of sensory input

• Feedback

  • Memories, expectations and experiences modulate/bias the activity in the early sensory areas

• E.g. - interpreting ambiguous images

  • Context shapes perception

  • Experience shapes perception

    • Unconscious inference —> Educated guesses based on visual cues

      • (top-down guesses bases on bottom-up information)

• E.g. - making predictions

  • Predictive coding

    • Visual brain makes predictions about what input the eyes are about to receive

      • Ongoing comparison of expectations with bottom-up information

.

Bottom-up processing

• Sensory input (“raw data”) influences perception

• Feedforward

  • 
    

How fast can an object be recognized?

Within 150 ms!

Give examples that illustrate the challenges of object segmentation, occlusion, and inverse projections in everyday vision

Object segmentation

• “Visually assigning the elements of a scene to separate objects and backgrounds”

  • To recognize objects and their location, you need to separate them from the background

• Figure-ground organization

  • “The perceptual segmentation of a visual scene into objects (“figure“) and backgrounds (“ground”)

  • Problem: - not always evident what boundary belongs to object vs background

    • E.g.

      • Rubin vase

        

      • Camouflaged animals

        

    • 
      

    • Rules to solve the problem:

      • Enclosure

        • The region completely surrounded by the other is perceived as the figure

      • Symmetry

        • The more symmetrical region is perceived as the figure

      • Convexity

        • The side that is convex (bulges out) is perceived as the figure, and the concave (caves) as the background

      • Meaningfulness

        • The meaningful side (determined by experience) is perceived as the figure

.

Object occlusion

• “Our views of objects are often partially blocked by other objects“

• Amodal completion

  • Brain fills in/infer the missing information during object occlusion

  • Infering the missing information can be easier with the occlusion than without but with the same amount of information

  • E.g

    • Seeing a truck behind a tree

    • “Amodal nudity“

• Related phenomenon: boundary extention

  • Tendensy to remember pictures as having extended beyond their actual edges

.

Inverse projection

• The fact that we live in 3D, but input to the eyes are 2D

• Problem: Different 3D objects can produce the same 2D image - bottom-up information is incomplete

  • E.g.

    • Different objects producing the same shadow

    • 3D objects producing 2D image on the retina

    • .

  • Solving the problem:

    • Top-down information completes the image - disambiguating the multiple possibilities:

      • Binocular depth cues

        • (result of binocular disparity —> disparity/difference between what the different eyes see)

          • i.e. binocular depth cues require both eyes

        • Greater disparity = object is closer to you

      • Monocular depth cues

        

        • Linear perspective

          • Parallell lines converge on single point when receding into distance

        • Texture gradient

          • Textual elements that are of same size get smaller and densely packed when receding into distance

      • Constancy - to perceive the world as stable

        • Constancy mechanisms require binocular and/or monocular depth cues

          • The distance/depth cues activate the constance in size mechanism

            • i.e. - If no distance/depth cues (monocular or binocular), we won't activate the constance in size mechanism

          • 
            

            

          • 
            

        • Object constancy

          • Recognizing objects despite orientation/ despite the image on the retina changing dramatically

        

        • Size constancy

          • Perceives sizes of objects are stable despite radical differences in their image size on retina -

          • Smaller retinal image often infer distance and not size

          

What are the different types of visual constancy that perception achieves?

Lightness/brightness Constancy

• Ability to factor in illumination conditions - in order to perceive a lightness as constant

  

  • The brain compensates for the squares in the shadow

    • Making the perception of the squares in the brain seem brighter than the ones outside of the shadow, even if they actually are the same brightness

      
      

.

Color Constancy

• Ability to factor in differences in illumination - in order to perceive a color as constant

  • i.e. - So, the shadow information/cues influences how we perceive the brightness of colors!

    • e.g.:

      • We are used to colors in the shadow being darker than in the light:

        • and so even if the color at the top/in the light in reality is the same (see picture under) as the color in the bottom/shadow, we compensate for the shadow, - ending up thinking that the bottom color is a brighter color and therefore another color than the one on the top

What is object recognition? - explain in terms of Riddoch and Humphreys (2001) hierarchical model of object recognition

• What is the distinction between apperceptive and associative agnosia? At which steps in the hierarchical model do they disturb the process of object recognition?

Object recognition

• Ability to match perceptual images with stored representations

.

Riddoch and Humphreys (2001) hierarchical model of object recognition

• Recognizing an object is not a single process - different stages transform visual information into increasingly meaningful representations:

  

  • 1. Early visual processing

    • Basic visual information is extracted - e.g. edges, orientation, color, size, shape

      • Processes what the visual impact looks like

  • 2. Viewpoint-dependent object descriptions

    • The visual features are combined into a more structured description of the object as it appears from a particular viewpoint

      • Processes what the object’s parts and spacial arrangements are

  • 3. Perceptual classification

    • The perceptual description is compared with stored structural knowledge to determine what kind of object is being seen

      • Processes whether the visual pattern corresponds with a familiar object shape

  • 4. Semantic classification

    • The recognized object is linked to stored knowledge about what the object is/means and what it is used for

      • Processes what the object is and what to do about it

  • 5. Naming

    • The appropriate verbal label is retrieved once the object has been identified and its meaning accessed

      • Labels the object

.

Agnosia

• Inability to recognize objects

  • Can occur in different modalities:

    • e.g. visual agnosia, tactile or auditory

• Often caused by lesions/damage in brain, but functioning eyes and ability to see in general

  • i.e. not explained by sensory deficits or memory loss

  • i.e. also not loss of the knowledge about the objects - the semantic system is intact

• Visual agnosia

  • Indicates that object recognition involves both early perceptual stages (low-level) and late perceptual stages (high-stages)

  • e.g. visual agnosia: matching images to their correct categories/lables

.

• Apperceptive (visual) agnosia

  • Impaired early vision - cannot perform even simples visual feature tasks (e.g. copying images)

    • Cannot recognize objects’ shape and say if it is familiar or not,

      • This inability can interfere with different levels of the object recognition process

    • Often involves prosopagnosia

  • Brain lesion impacting the early visual stage

• Associative (visual) agnosia

  • Intact early vision - can (ish) copy images

  • Impaired late vision (stage of recognition and categorization) - inability to name objects

    • i.e. Cannot say what the object is used for, but can recognize the object as familiar or not

      • They can copy it, describe it, but don't know what the object actually is - cannot assign meaning to object

  • Brain lesion impacting later visual stages

    • i.e. disturbing the connection between the perceptual and semantic system

.

Define and contrast view-based approaches and structural description approaches to object and face recognition

View-based - approaches to object recognition

• Claims: “we match images to representations that are like 2D pictures / “templates””

  • “Template” - representation that fully describes the shape of an object

• Critique of this approach:

  • How much noise or deviation can a matching process tolerate?

.

Structural description - approaches to object recognition

• Structural descriptions: “models that represent objects as sets of 3D parts standing in spatial relationships to each other“

  • The “models” - have have volumetric parts that can be analyzed quickly regardless of orientation

  • Recognition by components / analysis by parts

• Essential for reading and object recognition

.

Holistic perception

• Claims: “process a whole object at once - including the relations of the individual parts to each other“

• Evidence:

  • Visual expertise is linked with holistic processing across a range of visual stimuli

• Face recognition is dependent on holistic analysis

• 
  

What area of the brain is associated with face recognition?

Fusiform Gyrus

• Fusiform face area (FFA)

• ERP N170:

  • Show increased negativity for faces compared to objects

    

.

Prosopagnosia

• Visual agnosia specific for faces

• Can be caused by lesion in FFA

What is the evidence distinguishing what and where pathways for object processing?

• What evidence suggests that the what/where distinction might be better thought of as a perception/action distinction?

What and where pathways - evidence

• What pathway

  • Ventral path

    • From V1 and 2 —> through V4 in inferial temporal lobe

  • More active when focusing on distinguishing objects

    • e.g. by color, texture, shape, detail

• Where pathway

  • Dorsal path

    • From V1 and 2 —> through V3 and V5 in parietal lobe

  • More active when performing location task (even with same objects as in the object task)

    • e.g. location, movement, spatial relation

.

• Dissociating the ventral and dorsal streams

  • E.g. discrimination tasks and lesion

    • Object discrimination

      • Task: to pick the correct shape

      • Lesion in temporal lobe —> makes the task difficult

    

    • Landmark discrimination

      • Task: pick the food tray closer to the cone

      • Lesion in parietal lobe —> makes the task difficult

      • 
        

        

  • .

  • E.g. object and position task and fMRI

    

    • Object task:

      • Task: Participants need to say if the two displays (a and b) contained the same objects

      • Associated with activity in the ventral pathway (temporal lobe) - (red color)

    • Position task:

      • Task: Participants need to say if the objects in the two displays are presented in the same location

      • Associated with activity in the dorsal pathway (parietal lobe) - (green color)

  .

  • E.g. Ebbinghaus illusion

    

    • Even if the ventral pathway can be tricked by sizes of surrounding circles (see picture for reference), the dorsal pathway (action pathway) is not

.

Perception / action pathway

• Evidence for this way of thinking about the two paths:

  • Perception and action use information differently:

    • Case D.F. —> Agnosia with lesion in lateral occipital cortex (ventral/what path):

      

      • Unable to report the orientation of a slot, but could place a card in the slot

        • So, from the findings you can infer that visual information may influence action (dorsal stream) directly, independent of the recognition pathway (ventral stream)

    • Frogs hunt without recognizing the shape of the prey - only by detecting flylike movement

• Perception pathway

  • Allows us to determine what is located where

• Action pathway

  • Uses perceptual information to guide ongoing actions

How does the brain build complex visual perception from simple features over time?

Stage 1 (Features): Lower-level neurons detect basic orientations

• Feature detectors:

  • Neurons that respond best to specific types of stimuli (e.g. a certain orientation)

• Hubel & Wiesel

  • recorded electrical activity from a single neuron in the primary visual cortex of an anesthetized cat

.

Stage 2 (Hierarchy): Information moves from lower to higher cortices

• Hierarchical processing

  • Information progresses sequentially from lower to higher brain regions

• Gross et al., 1972

  • Recorded single-neuron activity in the temporal lobe while presenting different visual images to the subject

  

  • Findings:

    • A specific neuron fired strongly when looking at a shape resembling a hand

      • This response was invariant (i.e. it didn't matter if it was a real hand, a drawing, or tilted).

    • However, the neuron became inactive or fired weakly when shown abstract rectangles or shapes that did not look like a hand

    • Disproven Hypothesis:

      • If a visual stimulus is more complex, then more total neurons should fire to process it.

      • The Reality:

        • The visual system uses a structural hierarchy where specialized neurons along the ventral stream recognize whole configured objects rather than individual parts.

.

Stage 3 (Coding): Higher-level neurons recognize complex objects like tables

• Sensory coding

  • The way neurons represent the environment

  • Specificity coding

    • Suggests that highly complex stimuli are recognized by dedicated, single neurons "grandmother neurons"

    • However! the grandmother neuron hypothesis is not true!:

      • The brain relies on a neural network (population coding/distributed representation) rather than isolated cells

        • If we relied on single grandmother neurons, the visual system would be too sensitive and exposed to damage.

        • So, instead: A distributed network ensures resilience; if a few cells are damaged, the rest of the network can still accurately reconstruct and recognize the object.

.

Stage 4 (Dynamics): Successive images propagate simultaneously through the network

• (King & Wyart)

• Experimental Setup:

  • The experiment uses very simple visual stimuli that differ just slightly by tilting their orientation. These are flashed rapidly one after the other

  

  • 
    

• The flux of visual information propagate across time

  

  • Visual representations propagate continuously across a timeline from early sensory to late associative cortices, tracking cumulative processing scores

    • i.e. Network Propagation = Information does not stay isolated; it propagates through the whole brain like a network to help us make sense of the complete image/event

  

  • The 2-Second Window: The colorful graph tracks about 2 seconds of information processing. At any given second, the brain is actively processing the current stimulus while simultaneously processing what was presented just before

    • i.e. What was experienced just a moment ago still lingers in the brain and is actively integrated with what we are currently seeing.

• Implications:

  • Proves that we process things over time to create a whole combined world.

    • Instead of experiencing fragmented, free-floating stimulations, our brain merges these slightly different stimuli into a single, continuous perception.

What is mental imagery?

The act of forming a percept in mind without sensory input

.

Aphantasia

• Inability to engage in mental imagery

What is the mental rotation test, and how does performance change as a function of the angle of rotation?

Mental rotation

• Comparing and matching rotated images

• Performance changes as a function of the angle of rotation:

  • The time it takes to mentally rotate and match two objects is directly proportional to the amount of rotation that separates the objects

  • Like turning the object at a fixed speed in the mind

Define and contrast the depictive view and the propositional view of mental imagery

Depictive view of mental imagery

• Suggests: “the brain represents mental images like it represents real images coming through the eyes“

.

Propositional view of mental imagery

• Suggests: “mental images are held in a post-perceptual, abstract way, more like a linguistic description than a picture“

What is the neuropsychological patient evidence supporting a tight relation between mental imagery and perception? - i.e. supporting a depictive view of mental imagery

Evidence for relationship between mental imagery and perception

• “Machinery” (in brain) used for perception is also used in mental imagery

  • Brain scans showing similar activity in both conditions

    • e.g. Subtraction method in fMRI Ganis et al. (2004)

      • Findings:

        • Visual imagery and visual perception draw on mostly the same neural machinery, showing massive spatial overlap.

          • In frontal and parietal regions, the subtraction (Perception − Imagery) leaves almost no difference, proving cognitive control processes function comparably

          • In temporal and occipital (sensory) regions, differences appear, indicating that actual sensory inputs are engaged differently than purely mental ones.

  • Lesions:

    • Hypothesis: impairments in visual perception should have similiar impairments in mental imagery

    • E.g.

      • Color-vision impairments

        • Impairments in reporting colors from memory of common objects

      • Select regions of occipital lobe damage in one hemisphere

        • Impaired in visual perceiving & mental imagining in select region

        • Occipital lobe is topographic

          • Items in visual space is represented by adjacent close-together-neurons in cortex (like a map)

      • Spatial neglect

        • Fails to visually attend to objects on one side of their visual field

        • Test: asked to report a famous building from perception and then from memory —> showing similar patterns

          • “The minds eye turn”: Then asked to turn their body a bit in their minds eye —> now able to report parts of the building they weren’t able to before

            • Conclusion: Mental imagery uses the exact same coordinate system and attentional brain networks as physical vision

      • What/where pathways

        • Visual imagery performance correspond with visual perception performance

          • So, poor ability on what tasks visually correspond with poor ability on what tasks imaginary (and vice versa)

Why does mental imagery have a profound impact on mental illness compared to verbal thought?

Mental imagery elicits stronger emotions than verbal thinking does

• PTSD & Anxiety:

  • Patients rate mental images as feeling "more real" than healthy controls do

• Depression Risk:

  • Imagining suicidal acts is heavily linked to an increased risk of actual suicide

• Therapeutic Interventions:

  • Learning to intentionally imagine positive future events acts as a treatment for depression

  • Playing visuospatial video games (like Tetris) distracts the brain's visual working memory, blocking the rehearsal of trauma and leading to weaker, less vivid emotional memories

What is the brain imaging evidence supporting a tight relation between mental imagery and perception?

Imagining specific things activates areas in brain corresponding to areas activated when actually seeing the same things:

• Faces

• Scenes

• Motion

• Topographic areas show larger activity for mental imagery of large objects

  • As if the images were stimulating larger areas of the cortex

• Using TMS on visual area in brain also disrupts visual imagery