Notes on Mental Imagery and Propositions

Core Concepts: Knowledge Representation and Mental Imagery

• Knowledge Representation: The form for what you know in your mind about things, ideas, events, and so on, in the outside world.

• Two classical kinds of knowledge structures (classic epistemology):

  • Declarative Knowledge: Knowing "that" (facts that can be stated).

  • Examples: date of birth, name of a friend, what a rabbit looks like.

  • Procedural Knowledge: Knowing "how" (procedures that can be implemented).

  • Examples: tying shoelaces, adding a column of numbers, driving a car.

Empirical Sources on Knowledge Representation

• Two main empirical data sources:

  • Standard Laboratory Experiments: Observe cognitive tasks that require manipulation of mentally represented knowledge; researchers infer how knowledge is represented by performance.

  • Neuropsychological Studies: Examine brain activity during tasks or links between deficits in knowledge representation and brain pathologies.

  • Methods include observing normal brain responses to tasks or linking deficits to brain lesions.

Modes of Representing Knowledge: Pictures, Words, and Propositions

• Knowledge can be stored in multiple formats in the mind:

  • Pictorial or Analog Images: Mental pictures resembling real-world objects.

  • Words: Verbal labels and discrete units of language.

  • Pure Propositions: Abstract, propositional representations (mentalese) that do not resemble perceptual formats.

• Examples discussed:

  • (a) The cat is under the table.

  • (b) The table is above the cat.

  • (c) UNDER(CAT, TABLE) as a propositional form.

• Symbolic representations: The relationship between word and meaning can be arbitrary, e.g., in different languages a cat may be labeled differently (Germany: Katze; France: chat). This illustrates that language is a symbolic code rather than a direct perceptual mapping.

• Imagery: Mental representations of things not currently seen or sensed; can represent experiences not yet encountered; can use any sensory modality.

Dual Code Theory: Images and Symbols

• We use both pictorial (analog) and verbal (symbolic) codes to represent information.

• Analog Codes:

  • Mental images resemble the physical stimuli they represent (e.g., a visual image of an object).

• Symbolic Code:

  • Abstract, arbitrary representation that stands for something without perceptual resemblance.

• Dual-code theory posits that information is organized into two formats for storage, retrieval, and action.

Propositional Theory: Abstract Propositions over Images

• Propositional Theory suggests mental representations are not stored as images or pure words; they are propositions.

• Epiphenomenal Images: We may experience mental representations as images, but those images are secondary effects of more basic cognitive processes.

• Propositional representations can be combined to describe complex relations without preserving perceptual properties.

What is a Proposition?

• A proposition is an assertion that can be true or false.

• Example relationships:

  • “The table is above the cat.” vs. “The cat is beneath the table.”

  • These statements reflect the same spatial relationship; the propositional form encodes that relation independently of how it is heard or seen.

  • Formal representation example: UNDER(CAT, TABLE)

Using Propositions

• Propositions describe various relationships: actions, attributes, spatial positions, class membership, and more.

• Complex knowledge can be modeled by combining multiple propositions.

• Key idea: Propositional representations are neither purely in words nor in images; they are abstract structures that can be instantiated in different formats.

• Example mappings (Table 7.1 style):

  • Actions:

  • Propositional: BITE(MOUSE, CAT)

  • Imaginal: Bite(mouse, cat)

  • Attributes:

  • Propositional: FURRY(MOUSE) or MOUSE ext{ is furry}

  • Imaginal: (furry, mouse)

  • Spatial positions:

  • Propositional: UNDER(CAT, TABLE)

  • Imaginal: Cat under table

  • Class membership:

  • Propositional: ANIMAL(CAT)

  • Imaginal: (cat ∈ animal)

Limitations of Mental Images

• Mental images can be constrained by perceptual properties and task demands.

• Examples illustrating limitations include ability to maintain precision, ambiguity, and transformations that may not be unique or stable.

• Mental imagery can be ambiguous under certain transformations (e.g., rotating perspective can yield multiple valid interpretations).

Limitations of Propositional Theory

• While propositions are flexible, there are challenges in capturing perceptual details and spatial configurations through purely propositional means.

• Reproduced figures with verbal labels illustrate potential gaps between perceptual content and propositional encoding.

Mental Manipulations of Images: Function and Evidence

• Functional Equivalence Hypothesis:

  • Visual imagery is not identical to visual perception but is functionally equivalent to it.

  • Functionally equivalent representations are analogous to the physical percepts they stand for.

  • This hypothesis suggests we rely on images rather than just propositions in some cognitive tasks.

• Principles of Visual Imagery (five core ideas):
  1) Mental transformations of images correspond to transformations of physical objects/percepts.
  2) Spatial relations among elements in a mental image are analogous to real-space relations.
  3) Mental images can generate information not explicitly stored during encoding.
  4) Construction of mental images is analogous to constructing visually perceptible figures.
  5) Visual imagery is functionally equivalent to visual perception in terms of the visual system processes used.

Neuroscience and Functional Equivalence

• Neuroimaging evidence shows that tasks involving imagining an image activate similar brain areas as perceiving actual images, particularly in frontal and parietal regions (Ganis, Thompson, & Kosslyn, 2004).

• Imagery can evoke responses in high-level visual areas and even the primary visual cortex (Ishii et al., 2000; Reddy et al., 2010; Thirion et al., 2006).

Mental Rotations: Transforming Images in the Mind

• Mental Rotation: Rotationally transforming an object’s visual image in the mind.

  • Example: You can rotate a water bottle in your mind as you would physically rotate it in your hands.

• Figure-based demonstrations (Shepard & Metzler, 1971) show how rotation angle affects reaction times and accuracy when judging rotated figures.

• Linear function of rotation degree: greater rotation angles typically require longer processing times; exposure to degraded stimuli and practice effects can modulate performance.

Neuroscience of Mental Rotation

• Animal studies (monkeys) using single-cell recordings in motor cortex show that neurons respond as if the monkey is mentally rotating a stimulus, even when no physical rotation occurs (Georgopoulos et al., 1989).

• The primary motor cortex can be activated during imagined manual rotation, supporting functional equivalence between imagery and perception.

• This convergence strengthens the view that imagery shares neural substrates with perception.

Gender Differences in Mental Rotation

• Studies often report an advantage for males on mental rotation tasks, but findings are not universal.

  • Some studies report gender differences (Maeda & Yoon, 2013; Roberts & Bell, 2000a, 2000b, 2003).

  • Other studies do not find reliable differences (Beste, Heil, & Konrad, 2010; Jäncke & Jordan, 2007; Jansen-Osmann & Heil, 2007).

• Some null results may stem from the use of rotated characters (letters/numbers) rather than real-world objects, which could engage different processes.

Visual Image Properties: Image Scaling and Resolution

• Image Scaling: Representing and using mental images in ways functionally equivalent to percepts; involves resizing and scaling as with digital images.

• Resolution: The level of detail contained in an image.

• The implication is that mental images can be scaled and manipulated similarly to perceptual inputs, preserving functional utility.

Image Scanning: From Perception to Mental Search

• Stephen Kosslyn and colleagues demonstrated that mental images can be scanned similarly to perceptual scans.

• Key idea: Imaginal scanning strategies and responses closely resemble perceptual scanning strategies.

• A methodological approach to test functional equivalence: compare performance during perceptual scanning with performance during imaginal scanning.

Image Scanning: The Imaginary Island (Kosslyn et al.)

• Experimental setup: A map of an imaginary island with landmarks (hut, tree, lake) studied from memory.

• Procedure: After memorization, participants heard the name of a landmark and mentally scanned the image to locate it; they pressed a button when found.

• Purpose: To assess whether mental scanning parallels perceptual scanning in timing and accuracy.

Representational Neglect (Imagery Neglect)

• Representational neglect: A person asked to imagine a scene may ignore half of the imagined scene when describing it.

• Example: Given a kitchen scene, the person may describe one side accurately but omit the other when asked to describe the scene from a different perspective (e.g., from the refrigerator).

Synthesizing Images and Propositions

• The course moves toward integrating imaginal representations with propositional representations to explain cognitive processing.

Mental Imagery, Propositions, and Johnson-Laird’s Mental Models

• Mental imagery and propositions: Core concepts summarized for cognitive science.

• Johnson-Laird’s Mental Models: Mental representations may take three forms:
  1) Propositions
  2) Images
  3) Mental Models

• Mental Models: Knowledge structures that people create to understand and explain experiences; they are guided by implicit theories, beliefs, and perspectives.

• Faulty mental models can lead to thinking errors; experience can help repair them.

Lateralization of Function: Hemispheric Specialization

• Right Hemisphere:

  • Appears to manipulate visuospatial knowledge in a manner similar to perception.

  • Represents knowledge analogously to physical environments.

• Left Hemisphere:

  • Represents and manipulates verbal and symbol-based knowledge.

  • Capable of manipulating imaginal components and generating new information.

Kinds of Images: Visual and Spatial

• Visual Imagery: Images representing colors, shapes, and other visual features.

• Spatial Imagery: Images representing spatial features such as depth, distance, and orientation.

• Evidence suggests images may be stored in different formats depending on the type of image involved; a case (L.H.) illustrates selective impairment.

The Case of L. H. (Temporal Lobe Damage)

• L. H. had lesions in the right temporo-occipital regions and related areas; results indicate differential impairment in imagery domains:

  • Severe deficits in visual imagery tasks (colors, sizes, shapes).

  • Normal performance on spatial imagery tasks (e.g., spatial transformations).

• Despite intact perception (seeing) and ability to copy pictures, L. H. could not recognize or link verbal labels to the pictured objects.

• This dissociation supports the idea that visual and spatial imagery rely on distinct neural substrates.

Spatial Cognition and Cognitive Maps

• Spatial Cognition and Cognitive Maps involve the acquisition, organization, and use of knowledge about objects and actions in 2-D and 3-D spaces.

• They rely on what we have perceived and imagined, forming internal representations of the physical environment and spatial relationships.

• Cognitive maps simulate spatial features of the external world and support navigation and understanding of space.

Cognitive Maps Across Species

• Rats (Tolman): Mental maps of mazes can form even with varying reinforcement schedules; demonstrated through experimental manipulation of reward and learning trials.

• Bees: Demonstrate imaginary/abstract mapping for foraging and communication to other bees.

• Pigeons: Exhibit strong cognitive maps; notable for homing abilities.

• Humans: Use multiple knowledge types for cognitive maps:

  • Landmark knowledge: Information about specific features at a location; can be imaginal or propositional.

  • Route knowledge: Pathways for moving from one location to another; may involve procedural and declarative knowledge.

  • Survey knowledge: Distances and spatial relationships akin to map-like knowledge; can be imaginal or propositional (e.g., numerical distances).

Heuristics in Cognitive Maps

• Heuristics: Informal, intuitive strategies that sometimes yield effective solutions, sometimes not.

• In cognitive maps, propositional knowledge can influence imaginal knowledge (Tversky, 1981).

• Distortions in mental maps show regularization toward approximate geometric regularities:

  • Right-angle bias: Intersections tend to be represented as 90-degree angles more often than they are in reality.

  • Symmetry heuristic: Shapes and boundaries are perceived as more symmetrical than they truly are.

  • Rotation heuristic: Slanted boundaries are distorted toward vertical/horizontal alignments.

  • Alignment heuristic: Slight misalignments lead to better-aligned mental representations.

  • Relative-position heuristic: The relative positions reflect conceptual knowledge about contexts rather than actual spatial configurations.

Synthesis: From Images to Propositions

• The material emphasizes integrating imagery and propositional representations across different cognitive tasks (perception, imagination, navigation, problem solving).

• The interplay between imagery and propositions supports a flexible, multi-representational theory of cognition.

Summary of Key Equations and Notations

• Propositional form of spatial relations: UNDER(CAT, TABLE) or equivalently, “The cat is under the table.”

• General relational mappings (examples):

  • Actions: BITE(MOUSE, CAT)

  • Attributes: FURRY(MOUSE)

  • Spatial: UNDER(CAT, TABLE)

  • Class membership: ANIMAL(CAT)

Connections to Foundational Principles

• Declarative vs Procedural knowledge connects to how we store and retrieve facts versus procedures.

• Dual-code theory aligns with the brain’s ability to encode information in multiple formats (images and symbols).

• Propositional theory offers a compact, flexible representation for complex relationships, while imagery provides perceptual richness and rapid spatial reasoning.

• Mental models and cognitive maps illustrate how knowledge is organized to navigate real-world environments, with cross-species evidence supporting universal cognitive principles.

Practical and Ethical Implications

• Understanding how people represent knowledge can inform education, user interface design, and cognitive rehabilitation (e.g., targeted training to improve visual vs spatial imagery).

• Insights into representational systems can help diagnose and treat perceptual or memory-related disorders (e.g., representational neglect, imagery deficits).

• Recognizing gender differences in certain tasks should be approached with caution to avoid stereotyping; task design (e.g., using object vs character stimuli) can influence outcomes.

Real-World Relevance

• Cognitive maps underpin navigation in everyday life and in professions requiring spatial reasoning (architecture, urban planning, aviation).

• Imagery and propositions support problem solving, planning, and communication across disciplines by providing flexible representations of relationships.

• Neuroscientific evidence linking imagery to perception informs educational strategies that harness mental imagery for learning and memory enhancement.

Notes on Terminology and Cross-References

• Mental imagery: The mental representation of sensory experiences in the absence of direct sensory input.

• Propositions: Abstract, truth-apt statements representing relationships and properties.

• Propositional representations: Abstract structures that encode relationships without relying on perceptual form.

• Imaginal representations: Mental images spanning sensory modalities.

• Mental models: Interpretable, theory-driven constructs that explain experiences; can be revised with experience.

• Cognitive maps: Internal representations of spatial environments used for navigation and reasoning about space.

• Heuristics: Simple, efficient rules that guide judgments about spatial relations; can introduce systematic distortions.

• Lateralization: Distinct roles of left and right hemispheres in handling symbolic versus visuospatial information.

Quick Reference: Key Names and Studies

• Shepard & Metzler (1971): Mental rotation experiments foundational to understanding imagery-based spatial thinking.

• Kosslyn, Ganis, & Thompson (2001): Neuroimaging and mental rotation studies supporting functional equivalence.

• Ishii et al. (2000); Reddy et al. (2010); Thirion et al. (2006): Evidence of high-level visual cortex involvement during imagery tasks.

• Georgopoulos et al. (1989): Monkey motor cortex activity indicating mental rotation processes.

• Thorndyke (1981); Thorndyke & Hayes-Roth (1982): Landmark, route, and survey knowledge in humans.

• Maeda & Yoon (2013); Roberts & Bell (2000a, 2000b, 2003); Beste, Heil, & Konrad (2010); Jäncke & Jordan (2007); Jansen-Osmann & Heil (2007): Gender effects on mental rotation with mixed findings.