Perception: The Story of Shape

Perception: The Story of Shape

Ways of Seeing

• Developmental differences in 3D space perception, specifically related to Strabismus.

Strabismus

• Caused by impairment in the functioning of eye muscles, often congenital (infantile strabismus) with a genetic component.
• Historically, it was assumed that strabismics have severely impaired depth perception and can't "see 3D".
• There was a tendency to equate "seeing 3D" with binocular disparity (stereopsis), and a belief in a single 3D spatial encoding.
• Recent research focuses on identifying which aspects of 3D vision are impaired or preserved in strabismus.
• Notably, many famous artists and scientists, such as da Vinci and Picasso, were strabismic.

Distance Cues

• Near Space
  • Lens accommodation (effective up to 1 meter).
  • Ocular convergence angle (effective up to 6 meters).
• Far Space
  • Declination from eye level (effective from 5 to 20 meters).

Relative Depth Cues

• Provide information about 3D shape and layout.
• Examples include:
  • Shading
  • Interposition
  • Relative size
  • Perspective convergence
  • Texture
  • Binocular Disparity (Left eye & Right eye differences)
  • Motion Parallax
• In strabismus, binocular disparity is the primary cue that is compromised.
• Traditionally, binocular disparity is considered the most important 3D cue, leading to the assumption that strabismics have compromised depth perception.

Depth Cues

• Contour
• Shading
• Interposition
• Relative size
• Height in the field
• Texture
• Perspective
• Blur
• Binocular disparity
• Motion Parallax

Distance Cues (Near Space)

• Accommodation
• Convergence
• Depth of Focus Blur

Distance Cues (Far Space)

• Declination from eye level
• Ground-plane information

Strabismus and Distance Perception in Ambulatory Space (Blind Walking)

• Strabismics can perform blind walking tasks as well as non-strabismics when the target object is on the ground.
• However, they underestimate distance when the object is suspended in mid-air.
• This suggests that accuracy depends on declination from eye level but is underestimated when the task relies more on binocular disparity.

Blind Walking in Strabismus

• Accurate when it depends on declination from eye level.
• Underestimated when the task depends more on binocular disparity.

Relative Depth Judgement (3D)

• Task: Determining if black bars are equally spaced in 3D space; adjusting until they appear equal.
• Bias: Depth separations are underestimated for farther depth intervals.
• Typical observers will set farther intervals to be larger to perceive them as equal in 3D.
• Individuals with impaired 3D perception are expected to show even greater underestimation.

2D Judgement

• Task: Determining if black bars are equally spaced on the picture plane (screen); adjusting until they appear equidistant.
• Pictorial depth biases judgement and makes farther intervals seem larger.
• Typical observers adjust farther intervals to be smaller to perceive them as equal on the picture plane.
• Observers with impaired 3D perception are expected to show less bias.

Examining Relative Depth Perception in Strabismus

• Compare performance of strabismic and non-strabismic observers.
• Assess both monocular and binocular viewing conditions.
• Measure the ratio.

Results from Zlatkute et al., 2020 (2AFC staircase and method of adjustment)

• Comparison of non-strabismics and strabismics under one-eye and two-eyes conditions.
• Analysis of 2D and 3D depth perception tasks.

Depth Perception Compromised in Strabismics

• Strabismics have difficulty with precision motor tasks in near space (e.g., threading a needle).
• Grasping in near space can be compromised.
• These tasks require precise perception of exocentric and egocentric distance in near space.

Depth Perception in Strabismus

• Perception of 3D shape and layout (relative depth) is similar to non-strabismics.
• Egocentric distance estimation using blind walking is accurate for objects with ground contact.
• Motor tasks needing exocentric (scaled) distance and egocentric distance in near space are compromised due to loss of binocular disparity and ocular convergence information.
• Strabismics can perceive depth, but specific aspects are compromised.
• There is evidence that strabismics can perceive monocular stereopsis.
• The compromise primarily affects the binocular contribution to exocentric and egocentric distance perception.
• Supports the idea that 3D perception is based on multiple encodings, not a single one.

Integrated Model of Colour Perception

• Perceptual Realm
  • Analytic Description of phenomenology.
  • Abstract Model Based on phenomenology.
• Physical Realm
  • Physical description of stimulus.
• Psychophysical or Neural Model
  • Description of phenomenology.
  • Neural / behavioural response.

3D Perception

• Perceptual Realm
  • Analytic Description of phenomenology.
  • Abstract Model Based on phenomenology.
  • Phenomenology of egocentric distance.
  • Phenomenology of relative (unscaled) depth structure.
  • Phenomenology of exocentric (scaled) distances.
• Physical Realm
  • Depth cues.

Physicalist Approach to Object Shape Perception

• Outer Psychophysics
  • External world.
  • Retinal image.
• Inner Psychophysics
  • Neural computations.
  • Computational processing.
  • Perceptual judgment (as measured in an experiment).
  • Neural response to stimulus.

Cortical Regions

• Areas within the brain responsible for:
  • Primary somatic sensory cortex
  • Primary motor cortex
  • Primary visual cortex
  • Parietal lobe
  • Occipital lobe
  • Temporal lobe
  • Frontal lobe

Physicalist Approach to Object Shape Perception (Cont.)

• Hierarchical neural construction of shape.
• Early neural structures (early visual cortex) encode simple shape components (points, lines, curves).
• Later neural structures (temporal cortex) encode progressively more complex shapes—shape “features” to whole shape (face, object).
• Shape perception is mainly construed as shape recognition or categorization.

Primary Visual Cortex (V1)

• V1 receives input from the LGN (lateral geniculate nucleus).
• Contains:
  • Circular centre-surround cells.
  • Elongated 'simple' cells.
  • Ideal stimuli.

Orientation-Selective Simple Cells in V1 (Hubel & Weisel, 1956)

• Cells respond to bars of specific orientations.
• Tuning curve illustrates cell's response relative to stimulus orientation.

Response to Visual Patterns in Temporal Cortex

• Specific cells are selective for certain shapes.
• Example: Cell selective for a green star shape exhibits a distinct firing pattern when the stimulus is on.

Shape Selectivity in Monkey Temporal Cortex

• Cells demonstrate selectivity for specific shapes, such as a right-pointing triangle.

Cells in Monkey Temporal Cortex (STS) Responding to Diverse Face Images

• Cells respond to complex shapes such as faces.
• Study by Foldiak et al. 2003.

Shape Selectivity in the Ventral (Temporal) Pathway of Monkey Temporal Cortex

• Hierarchical processing: Features in V1 and V2 lead to Shape Selectivity in the temporal cortex (STS) and eventually, responsiveness to Faces.
• From Front to Back.

Parallel Results from Human Inferior Temporal Cortex from fMRI Data

• Brain regions identified using fMRI include:
  • Fusiform Face Area (FFA)
  • Lateral Occipital Cortex (LOC)

Activity in Fusiform Face Area (FFA)

• FFA is located in the inferior temporal cortex.
• Responds selectively to faces.

Selectivity for Face Shape but not Object Shape in FFA

• fMRI data indicates higher % MR signal change for Faces compared to Objects in the FFA.

Results from Human fMRI Data for Object Shape Selectivity

• Lateral Occipital Cortex (LOC) shows selectivity for object shape.

Results from Human fMRI Data for Object Shape Selectivity

• LOC activations are higher for intact (greyscale or line drawings) than scrambled images.

Brain Regions

• Distinct regions of the fusiform gyrus show selectivity for different categories:
  • Right: Place, Face
  • Left: Place, Face, Shape, Word form

Hierarchical Shape Processing from V1 to LOC / FFA

• Simple cells in V1 (oriented lines).
• Complex cells in V1 (line and motion, color).
• Hypercomplex cells in V1 (line length, end stopping).
• Pattern / feature detectors in V2/V4 (basic features).
• Object shape selective cells in LOC (object shape).
• Face selective cells in FFA (face shape).

Shape Representation in the Brain?

• Neurons recognize/categorize shapes at increasing levels of complexity.
• This doesn’t necessarily explain the representation or encoding of shape at a lower level.

Shape Perception as a Computational Problem

• Shape Perception is a much harder computation when considering the phenomenology.

Bregman's Letters

• Example images from Bregman

Modal and Amodal Completion

• Amodal completion occurs when a shape is perceived as continuing behind an occluding object.
• Modal completion involves perceiving illusory contours.

Regular Shape Preferred

• An illustration shows the brain preferring regular shapes rather than irregular shapes.

Mechanistic (Geometric) Models for Boundary Interpolation and Contour Completion

• Models address how the visual system interpolates boundaries and completes contours.
• Example: T-junctions can be interpreted as two objects or a single object, influencing Ishape (complexity measure) and \position (relative location).

Edge Relatability Criterion

• Edges are relatable if $0 \leq Rcos\theta \leq r$, where R and r are radii of curvature and $\, \theta$ is the angle between the edges.
• Relatable edges facilitate contour completion.

Phenomenology that Geometric Models Can't Explain

• Geometric models do not fully account for certain perceptual phenomena.
• Example provided by U. Tse.

Michael Leyton

• Leyton's work emphasizes the importance of symmetry, causality, and mind in shape perception.

Leyton, 1992: Shape as Dynamic Encoding

• The perception of shape is not static recognition but a dynamic and complex encoding with a complex phenomenology
• Implicit in our perception of shape is a “causal history” of how that shape “came to be”.

Symmetry/Causality Theory of Shape (Leyton, 1992)

• Shape is the perception of a nested hierarchy of symmetry breaking causal actions.

Leyton, 2007: Process Grammar

• Questions whether a shape is perceived as static or dynamic, with internal forces acting along its perimeter.
• Contrasts with the “Recognition theory of shape,” which treats shapes as static features for recognition.

Leyton's suggestion of internal forces

• When we perceive a shape we see these forces.

Causal Processes Defined Over Medial Axes

• Implicit causal actions can be defined over the medial axes of the shape.

Medial Axis Representation

• Introduced by Harry Blum in 1967 as a tool for biological shape recognition.
• A form of symmetry representation.

Leyton, 2007: Implicit Causal Actions

• Implicit causal actions phenomenologically perceived can be defined over the medial axes of the shape and as a causal history originating from actions applied to the simplest (symmetric shape).

Can we develop an integrated model of shape perception?

• Incorporating Perceptual realm, Physical realm, Analytic Description of phenomenology, phenomenology, and Abstract Model Based on phenomenology.