Visual Perception and Cognitive Theories

Foundations of Biological vs. Computer Vision

  • Representation Discrepancy Between Biological and Artificial Systems:

    • For a digital computer, an image is fundamentally an array or pattern of numerical pixel values (e.g., intensity, color coordinates).

    • The computational system must interpret these numbers to extract high-level semantic meaning and spatial structure.

    • Key computational challenges that biological brains resolve effortlessly include:

    • Object Segmentation: Determining where one structure ends and another begins within a visual scene.

    • Perceptual Assignment: Resolving which visual elements belong to which discrete objects.

    • Disambiguating Real Objects from Optical Reflections: Distinguishing an actual entity (such as a pedestrian on a sidewalk) from its specular reflection across glass or polished surfaces.


Historic building reflected in the glass façade of a modern office building
  • Functional Scope of Computer Vision:

    • Computer vision demonstrates near-perfect performance in specialized, bounded tasks such as:

    • Industrial defect detection across manufacturing components.

    • Standardized pattern recognition.

    • Financial and transactional fraud detection.

    • While computer vision algorithms can surpass human visual performance on targeted narrow domains, machine vision remains substantially inferior to biological vision when navigating open-ended, complex, or naturalistic environments.

Questions & Discussion: Evaluating Machine Vision Capabilities

  • Image Ambiguity and Spatial Reasoning Benchmark (as of October 2026):

    • Prompt: "what this picture shows?"

    • Follow-up Inquiry: "is it standing next to, or reflected in, the modern glass office building?"

    • System Response:

    • "Looking more closely at the image, my previous answer was ambiguous.

    • The historic Gothic building appears to be physically standing behind or adjacent to the modern glass façade, rather than simply being a reflection.

    • The clues are:

      • The Gothic building's architectural details remain sharp and consistent across the glass panels.

      • Its position aligns naturally with the street-level entrance and surrounding structures.

      • There is no obvious mirroring or distortion that would strongly indicate a reflection.

    • So the image most likely shows an old historic building incorporated into, or standing immediately behind, a modern glass office development.

    • However, because the photograph is monochrome and the glass is highly reflective, I cannot rule out some reflection effects with complete certainty."

    • Theoretical Significance: Demonstrates that even state-of-the-art vision models misinterpret visual evidence that biological visual systems disambiguate rapidly and accurately, failing to resolve co-planar reflections against occluded geometry.

Invariant Object Recognition

  • Definition and Core Properties:

    • Invariant object recognition refers to the instantaneous, effortless, and accurate categorization and identification of objects despite significant perceptual variations in:

    • Color

    • Size

    • Orientation (both in-plane and in-depth)

    • Illumination

    • Retinal position (eccentricity)

    • Operates reliably within complex visual scenes and across novel, unexpected contexts in fractions of a second.

    • Represents one of the most remarkable processing capabilities of biological vision.

    • Despite decades of cognitive and neuroscientific research, the precise neural mechanisms through which the brain constructs invariant object representations remain poorly understood.

  • Experimental Variations in Invariant Recognition (Karimi-Rouzbahani et al., 2017, Sci Rep):

    • Investigated invariant processing using canonical car models (labeled Car 1, Car 2, Car 3) across multiple systematic transformations:

    • Position Eccentricities: Stimulus presentation at varying visual angles off-center, specifically evaluated at 3∘3^\circ, 7∘7^\circ, and 13∘13^\circ.

    • Size Variations: Stimulus scaling across retinal visual angles, including 6∘6^\circ, 14∘14^\circ, and 17∘17^\circ.

    • In-Plane Rotations: Planar rotations around the line of sight, tested at −40∘-40^\circ, 60∘60^\circ, and 180∘180^\circ.

    • In-Depth Rotations: Three-dimensional perspective rotations off-plane, tested at −40∘-40^\circ, 40∘40^\circ, and 180∘180^\circ.


Variations in car stimuli across default views, position eccentricities, and size variations


In-plane and in-depth angular rotations of car models

Perceptual Framework and Sensory Modalities

  • General Information-Processing Architecture of Perception:

    • Perception is structured as a biological information-processing sequence:

    • An Informational Medium transports physical signals originating from a Distal Object (the environmental object) to the sensory receptors of a person.

    • Upon encounter with receptor surfaces, Proximal Stimulation occurs (energy absorption and transduction).

    • This leads directly to the mental construction of the Perceptual Object.

    • Visual perception serves as the most widely investigated modality within cognitive science.

  • Cross-Modal Breakdown of Perceptual Components:

    • Vision (Sight):

    • Distal Object: Grandma's face.

    • Informational Medium: Reflected light from Grandma's face (visible electromagnetic waves).

    • Proximal Stimulation: Photon absorption within rod and cone photoreceptor cells in the retina (the receptor surface at the rear of the eye).

    • Perceptual Object: Grandma's face.

    • Audition (Sound):

    • Distal Object: A falling tree.

    • Informational Medium: Sound pressure waves generated by the mechanical impact of the tree's fall.

    • Proximal Stimulation: Fluid and sound-wave conduction inducing vibrations along the basilar membrane within the cochlea of the inner ear.

    • Perceptual Object: A falling tree.

    • Olfaction (Smell):

    • Distal Object: Bacon being fried.

    • Informational Medium: Chemical molecules volatilized and released into the air by cooking bacon.

    • Proximal Stimulation: Molecular binding and absorption in the olfactory sensory neurons within the olfactory epithelium of the nasal cavity.

    • Perceptual Object: Bacon.

    • Gustation (Taste):

    • Distal Object: Ice cream.

    • Informational Medium: Chemical molecules of ice cream aerosolized into the air and dissolved into salivary solution.

    • Proximal Stimulation: Molecular contact with gustatory receptors located in taste buds across the tongue and soft palate, integrated with olfactory stimulation.

    • Perceptual Object: Ice cream.

    • Touch:

    • Distal Object: A computer keyboard.

    • Informational Medium: Mechanical pressure and physical vibration generated at the contact boundary between the keyboard surface and the skin.

    • Proximal Stimulation: Mechanical stimulation of mechanoreceptors situated within the dermis (the innermost layer of the skin).

    • Perceptual Object: Computer keys.

Neurophysiology of Vision and Anatomy of the Visual Pathway

  • Gross Anatomy of the Human Eye:

    • Cornea: Outermost transparent optical layer that protects the internal chamber and provides primary optical refraction.

    • Pupil & Iris: The pupil acts as the aperture through which light enters; its diameter is modulated by the musculature of the iris (the pigmented tissue disk).

    • Anterior & Posterior Chambers: Contain aqueous humor, maintaining ocular pressure and supplying nutrients.

    • Lens & Suspensory Ligaments: The crystalline lens alters refractive shape via tension exerted by suspensory ligaments linked to the ciliary body (containing ciliary muscle) to focus light onto the retina.

    • Vitreous Humor: Clear, gelatinous matrix filling the central vitreous chamber to preserve structural integrity.

    • Sclera & Choroid: Sclera comprises the white protective outer fibrous coat; the choroid is a vascular layer between the sclera and retina providing oxygenation and absorbing stray light.

    • Conjunctiva: Mucous membrane covering the front of the eye and lining the inside of the eyelids.

    • Tendon of Rectus Muscle: Extraocular muscle insertions controlling rotational eye movements.

    • Retina: Multi-layered neurosensory sheet at the back of the eye converting photonic energy into neural potentials.

    • Fovea: A specialized, pinhead-sized, thin central depression in the retina packed with cones that provides maximal visual acuity. When an observer gazes directly at a target, the visual axis rotates to project the image onto the fovea.

    • Blind Spot: The anatomical optic disc region completely devoid of photoreceptors where retinal ganglion cell axons converge to exit the globe.


Detailed anatomical diagram of the human eye
  • Ascending Visual Pathway:

    • Phototransduction: Photons reflect from external objects, enter through the pupil, and strike photoreceptors in the retina. Photoreceptors transduce photonic energy into electrical gradients, synapsing through intermediate retinal layers onto retinal ganglion cells.

    • Optic Nerve Formation: Axons of the retinal ganglion cells gather to form the optic nerve bundle.

    • Optic Chiasm Decussation:

    • The optic nerves converge at the optic chiasm at the base of the hypothalamus.

    • In humans, axons representing the nasal hemiretina (inner visual field close to the nose) cross the midline (decussate) to project to the contralateral cerebral hemisphere.

    • Axons originating from the temporal hemiretina (outer visual field toward the temporal cortex) remain uncrossed and project to the ipsilateral hemisphere.

    • Lateral Geniculate Nucleus (LGN):

    • Optic tract axons project primarily to the lateral geniculate nucleus (LGN), a dedicated sensory thalamic nucleus.

    • The LGN relays organized visual signals via optic radiations to the primary visual cortex (striate cortex / V1) in the occipital lobe.

    • Reciprocal corticothalamic feedback: Efferent projections returning from the visual cortex back to the LGN continually modulate and gate thalamic transmission.


Diagram of the human visual pathway from retinas through the optic chiasm and lateral geniculate nucleus to the primary visual cortex

Computational Levels of Perception: Marr’s Tri-Level Hypothesis

  • Theoretical Framework: Formulated by David Marr, stating that any complex information-processing system can be completely specified across three distinct levels of analysis:

    • 1. Computational Level (What & Why?):

    • Defines the primary goal of the computation and the specific constraints governing the task.

    • Addresses what problem the system is designed to solve.

    • Application to Vision: Visual perception is not simply about luminance detection; its computational objective is identifying physical objects, determining their three-dimensional geometry, tracking their positions in space, and guiding appropriate motor actions.

    • 2. Algorithmic Level (How?):

    • Formulates the exact procedural steps, representations, and transformations required to solve the computational problem.

    • Specifies the input representations, intermediate symbolic tokens, and output formats.

    • Application to Face Identification: The computation might be executed by extracting local edge gradients, aggregating edges into geometric boundary descriptions, and comparing these spatial templates against stored exemplar representations in memory.

    • Multiple distinct algorithms can solve identical computational goals.

    • 3. Implementation Level (Where?):

    • Defines the physical realization and substrate architecture of the algorithm.

    • In biological organisms, it specifies the biophysical mechanisms: which neuronal populations fire, which specialized cortical visual regions process each sub-step, and how action potentials and chemical synapses propagate signals.

Theories of Perception: Bottom-Up vs. Top-Down Processing

  • Bottom-Up (Stimulus-Driven) Theories:

    • Posits that perception commences purely from low-level environmental input impinging upon sensory organs, proceeding sequentially to higher-order brain areas without cognitive intervention.

    • a. Gibson’s Theory of Direct Perception (Ecological Perception):

    • Developed by J.J. Gibson; asserts that the sensory array contains rich, unambiguous, structured information (including environmental texture gradients and optical flow) sufficient for perception.

    • Higher-order cognitive deliberation, internal hypothesis generation, and stored mental representations are unnecessary; the ecological environment provides all information directly.

    • b. Template Theories:

    • Suggests the mind stores holistic internal patterns or miniature blueprints (templates).

    • Visual pattern recognition occurs when a sensory input directly matches an existing stored template.

    • Inherent Limitations: Fails to scale computationally. It cannot explain how humans easily identify a character (e.g., the letter "A") across infinite variations in font, scale, slant, and deformation without requiring an infinite library of templates.

    • c. Feature-Matching Theories:

    • Asserts that patterns are recognized by decomposing incoming visual stimuli into elemental building blocks (e.g., horizontal bars, diagonal slants, curves) and matching these constituents against stored feature sets in memory.

    • Inherent Limitations: Lacks precise consensus regarding what constitutes a primitive feature; crucially overlooks the major contribution of context, syntax, and grammatical expectations.


The CHT stimulus showing identical central glyph interpreted as H or A depending on context
  • Top-Down (Constructive / Intelligent) Theories:

    • Known as Constructive Perception or Intelligent Perception.

    • The observer actively constructs a perceptual interpretation of a sensory stimulus using cognitive context, prior knowledge, expectations, and hypothesis testing.

    • Perceptual interpretations are derived from three concurrent streams:

    • Sensory Evidence: The incoming raw physical stimulation.

    • Stored Knowledge: Memory representations accumulated from prior experience.

    • High-Level Inference: Rapid computational hypothesis generation and selection.

    • Perception is bidirectional: it influences and is modulated by real-time experiential context.

  • Integrated Neuroscientific View of Perception:

    • Biological perception is a continuous synthesis of bottom-up sensory extraction and top-down cognitive feedback.

    • Higher-tier processing areas in visual cortex remain tethered to early retinotopic sensory maps.

    • Even as downstream visual areas form abstract, viewpoint-invariant representations, they utilize retinotopic spatial coordinates inherited from primary visual cortex (V1V_1) and the retina.

    • The brain calculates world-centered spatial coordinates and conceptual identities implicitly by fusing high-level object recognition networks with low-level retinotopic signals and ocular motor positional cues.


Integration model of top-down expectations and bottom-up sensory input producing object perception

Gestalt Psychology and Perceptual Organization

  • Historical Background and Central Axiom:

    • Emerged in Germany during the early 20th century.

    • Core theoretical tenet: "The whole is different from the sum of its parts."

    • Biological visual systems automatically structure disparate, fragmented visual features into unified, stable, and coherent configurations.

  • The Law of Prägnanz (Law of Simplicity / Good Figure):

    • The overarching governing law of Gestalt psychology.

    • States that every stimulus configuration is perceived in such a manner that the resulting perceived structure is as simple, regular, and symmetrical as possible.

    • Example: The Olympic rings insignia is spontaneously perceived as five intersecting continuous rings rather than a fractured mosaic of nine irregular geometric shapes.


Olympic rings illustrating the Gestalt law of Prägnanz as five circles versus nine complex sub-shapes
  • Key Gestalt Principles of Grouping:

    • Law of Similarity: Objects that share visual characteristics (e.g., color, shape, size) are perceptually categorized and grouped together. (e.g., an array of alternating dots and squares is organized into vertical columns rather than rows).

    • Law of Good Continuation: Points that, when linked together, establish straight or smooth curvilinear paths are perceived as belonging together; line trajectories are tracked along their smoothest continuous course rather than sharp, abrupt turns.

    • Law of Proximity: Elements situated in close spatial contiguity are perceived as unified entities (e.g., 24 evenly spaced dots split into three spatial clusters are perceived as three groups).

    • Figure–Ground Organization: Visual arrays are organized into a prominent object with defined borders (the figure) standing out against a receding, formless setting (the ground).

    • Closure: The cognitive visual system completes interrupted or incomplete boundaries, mentally interpolating missing structural segments to perceive closed forms.

    • Common Fate: Dynamic visual components moving along a uniform vector and velocity are perceived as belonging to a collective object.


Grid of alternating columns of squares and circles illustrating the law of similarity


Interwoven Celtic knot pattern demonstrating the law of good continuation


Rubin vase optical illusion depicting either a vase or two facing silhouettes

Perceptual Completion, Constancy, and Illusions

  • Perceptual Completion (Filling-in):

    • The mechanism by which the visual brain reconstructs occluded surfaces and unbroken geometric contours despite discontinuous, incomplete retinal signals.

    • Demonstration: Observers looking at people standing behind horizontal bleacher rails see whole individual bodies rather than severed torsos and limbs.


Three men standing on bleachers occluded by wooden railings illustrating perceptual completion
  • Perceptual Constancies:

    • Perceptual constancy describes the preservation of stable mental representations of distal objects despite major variations in the proximal retinal image.

    • Size Constancy: When an observer approaches an object, its subtended retinal angle and physical area on the retina increase dramatically. Nevertheless, the observer perceives the object's physical size as constant.

    • Lightness Constancy: The perceived reflectance (achromatic shade) of light and dark surfaces remains constant across shifting ambient illumination intensities.

    • Serves as empirical validation for constructive perception models over passive bottom-up models.

  • Colour Constancy:

    • The visual capacity to identify the true intrinsic chromatic properties of surfaces across diverse chromatic illumination environments.

    • Example: Viewing objects under blue-tinted lenses or monochromatic illumination does not alter the perceived identity of yellow bananas or white paper.

    • The visual cortex computes relative reflectance by comparing light reflected from an object against the chromatic spectrum reflected by adjacent objects, computationally subtracting the ambient illuminant.


Rubik's cube under yellow and blue illumination illustrating colour constancy
  • The Critical Determinant of Background Context:

    • When contextual backgrounds are masked, isolated patches that appeared distinctly blue under warm light or yellow under cool light are revealed to be physically identical neutral gray swatches.

    • Demonstrates that visual wavelength alone does not specify perceived color; color perception is a relational calculation across local contexts.


Isolated patches from the Rubik's cubes showing that identical physical wavelengths appear as different colours depending on background
  • Retinex Theory and Edge Contrast Illusions:

    • Retinex Theory (Edwin Land):

    • Formulated as a blend of Retina and Cortex.

    • Explains both color and brightness constancy by demonstrating that the cerebral cortex processes and cross-compares spectral distributions across broad, distinct retinal zones to assign absolute surface color.

    • Craik-O’Brien-Cornsweet Illusion:

    • An identical physical luminance across two adjoining surfaces is perceived as two distinctly different shades (e.g., dark gray upper block, light gray or white lower block) purely because of an engineered luminance gradient at their shared central edge.

    • Local edge contrast signals induce the visual cortex to fill in the perceived lightness across the entire interior of the adjacent surfaces.


Craik-O'Brien-Cornsweet illusion demonstrating surface brightness interpolation from edge contrast

Reference Literature

  • Core Textbooks:

    • Sternberg, R. J., & Sternberg, K. (2012). Cognitive Psychology (6th ed.). Wadsworth Cengage Learning. (Chapter 3: Visual Perception).

    • Goldstein, B. E. (2008). Cognitive Psychology: Connecting Mind, Research, and Everyday Experience (2nd ed.). Thomson Wadsworth. (Chapter 3: Perception).

  • Primary Research Citations:

    • Karimi-Rouzbahani, H., Bagheri, N., & Ebrahimpour, R. (2017). Invariant object recognition in biological vision. Scientific Reports.