The Mind Embodied, Embedded, Extended, and Enacted: Beyond Computationalism

Course Overview and Terminology

• Paradigms in Cognitive Science: The main paradigms discussed include Behaviourism, Computationalism, Connectionism, Ecological Psychology, and Enaction. These are categorized as theoretical systems, perspectives, or approaches.

• Dynamical Systems Theory (DST): It is important to note that DST is NOT a paradigm in psychology. Rather, it is a field of mathematics that provides formal mathematical tools used to investigate cognition. These tools are applied across multiple paradigms, including Computationalism, Connectionism, Ecological Psychology, and Enaction.

• Definition of Embodiment: This term refers to specific claims regarding the role of the physical body in cognitive processes. These claims range from "simple" observations about biological constraints to "radical" hypotheses that challenge the necessity of internal representations.

Review: Week $11$ - Dynamics and the Limits of Computationalism

• Learning Goals for Dynamics:

  • Describe the limits of computationalism as identified by advocates of Dynamic Systems Thinking (DST).

  • Discuss how DST overcomes these limitations.

  • Compare and contrast the definition of "cognition" between computationalism and DST.

  • Describe empirical examples of DST applied to cognition, such as finger tapping (interlimb coordination), infant walking development, the A-not-B error, and adaptive behavior in robots.

• Example: Interlimb Coordination:

  • Empirical Observation: As the speed of movement increases, there is a spontaneous transition from anti-phase (limbs moving in opposite directions) to in-phase (limbs moving in the same direction) coordination.

  • Dynamical Explanation: This transition follows from the intrinsic dynamics of the system. It is modeled as coupled oscillators where stable patterns are represented as attractor states. Changes in these patterns correspond to bifurcations in the potential function.

• Example: Infant Walking:

  • Empirical Observation: Newborns exhibit a stepping reflex that seemingly disappears at $2$ months but walking appears at approximately $1$ year. However, if a $2$-month-old infant is placed waist-deep in water or on a treadmill, the stepping reflex "reappears."

  • Computational Explanation: Neurological maturation of the brain inhibits the primitive reflex.

  • Dynamical Explanation: The reflex does not disappear; rather, the infant's body becomes heavier and their muscles are not strong enough to lift the legs against gravity for a period. Stepping is not centrally controlled by a motor program but is self-organized through the interaction between the body (growth, muscle strength), neurological maturation, and the environment.

• Example: The A-not-B Error:

  • Empirical Observation: The "perseveration" error occurs when infants repeat an action where they were previously successful, even after seeing a reward moved to a different location.

  • Computational Explanation: Infants lack a conceptual understanding of object permanence.

  • Dynamical Explanation: The error results from the intrinsic dynamics of the nervous system coupled with the external task environment and developmental changes in the reaching system. It is a product of neurological maturation and the physical interaction with the environment.

Conceptions of Embodiment: The Simple vs. The Radical Claim

• General Definition of Embodiment: Cognition arises from bodily interactions with the world. It depends on experiences stemming from a body with particular perceptual and motor capabilities. These capabilities are inseparably linked and form the matrix for reasoning, memory, emotion, and language (Thelen et al., $2001$).

• The Role of the Agent's Body: An embodied nervous system utilizes natural biomechanics, the geometry of sensory surfaces, and active control of those surfaces to simplify cognitive problems (Beer, $2003$).

• World vs. Environment:

  • The World: Physicochemical properties that may be described independently of any organism (Physics and Chemistry).

  • The Environment: The specific part of the world that is relevant to an organism. Organisms "bring forth a world" by implying an environment through their structural coupling with it.

• The Animal-Environment (A-E) System: The animal implies an environment and vice versa. They are mutual, reciprocal, and interdependent. Therefore, the A-E system is the proper unit of analysis for cognition. Cognition is necessarily embodied (occurring in a body) and embedded (situated in an environment).

• The Simple Claim:

  • The body and environment constrain cognition. Biomechanical properties (e.g., rotations around joints) and physical properties of the environment dictate possible actions.

  • Example: Passive dynamic walking robots. Honda's Asimo (early models) failed to walk efficiently because it relied on an internal model of the world to plan every movement. In contrast, Cornell University's passive dynamic walker and robots from the Nagoya Institute of Technology use mechanical forces and biomechanics alone to walk or turn, demonstrating that movement arises from body-environment interaction rather than just motor programs.

• The Radical Claim:

  • Cognition does not involve computations over representations. Brains are not the sole cognitive resource; bodies and perceptually guided motions perform the work required to achieve goals, replacing complex internal mental representations. This perspective shifts the definition of what cognition involves entirely.

Where Cognition Happens: A-E Coupling vs. Internal Representations

• Computational Approach:

  • The animal (A) and environment (E) are fundamentally separate.

  • Environment merely provides input ($I$).

  • Primary explanations rely on internal representations.

  • Focuses on the Animal and ignores the Environment.

  • Cognition happens strictly inside the animal.

  • Behavior is controlled by centralized internal processes (motor programs, executive plans).

• Ecological and Enactive Approaches:

  • The animal and environment are fundamentally coupled ($A-E$).

  • Environment is constitutive of cognition.

  • Primary explanations rely on the coupling; no internal representations are required.

  • Cognition happens in the Animal-Environment system.

  • Behavior control is emergent in the A-E system (nonlinear dynamics, self-organization).

The Ecological Approach to Cognition (James and Eleanor Gibson)

• Founders: Established by James Gibson ($1904-1979$) and Eleanor Gibson ($1910-2002$).

• Core Claims:

  • The Information Hypothesis: For every perceivable property of the environment, there is a higher-order variable of information that specifies it ($1:1$ mapping).

  • The Affordance Hypothesis: An affordance is perceivable if there is higher-order information specifying the relation between environmental and animal properties that constitutes it.

  • The Theory of Direct Perception: Perception involves the detection of information available in the environment that specifies animal-relevant properties. Detection is direct; it does not require inference or computational embellishment.

The Concept of Affordances: Opportunities for Behavior

• Definition: Affordances are what the environment offers the animal, provides, or furnishes, for good or ill. The term implies the complementarity of the animal and the environment.

• Examples of Affordances:

  • Nutrition/Harm: Substances afford nutrition or harm depending on the animal's metabolism (e.g., edible objects vs. toxic ones).

  • Grasping/Manipulation: A twig affords grasping for a kingfisher; a stone of specific size and weight affords pounding for a capuchin monkey.

  • Social Interaction: Other individuals afford behaviors such as sex, predation/prey interactions, or social bonding.

  • Surfaces/Support: A surface affords standing, walking, or running if it is nearly horizontal, nearly flat, sufficiently extended, and rigid relative to the animal’s size.

  • Design and Tools: Understanding affordances is crucial for design. Don Norman ($2013$) argues that when external signs (like "Push" or "Pull" on a door) are necessary, it indicates bad design because the affordances should be self-evident. A hand sink that looks like a urinal is an example of poor affordance design.

Environmental Information and the Theory of Direct Perception

• Definition of Information: Lawful patterns in stimulation (light, sound, chemicals) that specify a relevant property. Information is specific to, informs about, and has a $1:1$ mapping with environmental properties.

• Optic Flow: The static image on the retina is ambiguous regarding depth and size ($2D$ vs. $3D$). However, for an active animal, optic flow is unambiguous.

  • Principles: Surfaces far from the animal change slowly in the optic field; surfaces near the animal change quickly. This creates lawful patterns.

  • Example: Walking in a real forest creates an optical pattern showing the verticality of trees and horizontality of the ground. This differs from walking toward a $2D$ picture of a forest.

  • Locomotion Control: Information in optic flow specifies if an animal is resting (no flow) or moving (global changes). Rules for action include: To start moving, make the array flow; to stop, cancel the flow; to approach a target, magnify the patch in the visual field.

Direct vs. Indirect Perception: A Comparative Analysis

• Computational/Indirect Perception (David Marr):

  • Function: Construct an internal representation of the external world.

  • Process: Sense $\rightarrow$ Decide (Cognition) $\rightarrow$ Act.

  • Information Problem: Input is impoverished, tiny, distorted, and upside-down. The brain must use algorithms to infer the $3D$ world.

  • The Ames Room Illusion: Demonstrates how a non-rectangular room looks rectangular when viewed with one eye and no movement, supporting the idea that the brain must infer reality.

• Ecological/Direct Perception (James Gibson):

  • Function: Detect affordances in the environment.

  • Process: Perception and Action are two aspects of a single process (A-E dynamics).

  • Information Problem: Information is unambiguous and specifying. There is no need for internal models because the A-E system is a coupled system. In reality, animals are active and have two eyes, providing much more information than the Ames Room experiment suggests.

Case Study: The Outfielder Problem

• Task: A player perceives a fly ball and must adjust action (running) to catch it.

• Computational Explanation: The nervous system uses the initial angle and velocity to create an internal simulation of the parabolic path, computing the future landing position and programming the run to that spot.

• Ecological Explanation (Optical Acceleration Cancellation):

  • Information: If the ball lands behind the player, the projection of the ball accelerates upwards on the visual plane. If it lands in front, it decelerates.

  • Rule: To intercept the ball, adjust speed so the projection of the ball moves at a constant speed (acceleration = $0$).

  • Evidence: Research shows players (and even dogs catching frisbees) use this prospective control strategy rather than predictive computation.

Neuroscience and Optic Flow (EEG Evidence)

• Study (Agyei et al., $2015$): Longitudinal study of infants at $3-4$ months and $11-12$ months using high-density EEG.

• Research Question: Is there a difference in neural activity when infants explore forward, backward, and random visual motion?

• Results:

  • $3-4$ months: No difference in peak latency of neural activity across different motion types.

  • $11-12$ months: Distinct differences in peak latency; shortest latency for forward optic flow. At this age, infants have crawling experience, which suggests that the ability to differentiate and detect structured optic flow is linked to active movement experience.

Historical Perspectives and Paradigmatic Shifts

• Marr vs. Gibson: David Marr ($1982$) critiqued Gibson for underestimating the complexity of information processing, though he acknowledged Gibson's focus on the real-world function of senses.

• Ulric Neisser's Conversion: After originally publishing "Cognitive Psychology" ($1967$), Neisser moved to Cornell and was influenced by the Gibsons. He eventually concluded that "Gibson is right"—information is in the light, and perception is direct, leading to a major shift in his thinking.

• Analogy: The shift from the computational (Information-processing) model to the Ecological model is likened to the shift from Geocentric to Heliocentric models of the universe. It moves from an egocentric view of behavioral cause to an organism-environment system view.

The Extended Mind and Cognition Hypothesis (Clark and Chalmers)

• Central Question: Where does the mind stop and the rest of the world begin?

• The Parity Principle: If a part of the world functions as a process which, were it done in the head, we would recognize as part of a cognitive process, then that part of the world is part of the cognitive process.

• Three Cases of Problem Solving (Tetris/Rotation):

  1. Case $1$: Mental rotation of a $2D$ shape.

  2. Case $2$: Physical rotation of the image on a screen using a button.

  3. Case $3$: Future neural implant performing rotation as fast as a computer.

• Argument: If Case $1$ and Case $3$ are considered cognitive, then Case $2$ (distributed across agent and computer) should also be considered cognitive. This suggests that the human organism's skin and scalp do not delimit the thinking subject.

Questions & Discussion

• What is cognition and where does it happen?

• Where do you feel yourself located?

• Why is it difficult to think in terms of the A-E system?

  • Education trains us in the dominant paradigm (Information-processing).

  • We are generally not trained in Dynamic Systems Theory.

  • Naïve experience makes us feel like subjects confronting objects (Egocentric view).