Embodied Cognition, Gesture, and Communication

Facial Feedback Hypothesis and Embodied Cognition

• Subjective Mood Assessment: An informal experiment asking individuals to rate their mood on a 1-7 scale (where 1 is low mood) highlights the subjective nature of mood perception.
• Facial Feedback Hypothesis: This concept suggests that your facial expression can alter or improve your mood.
  • It is considered fairly reliable, though not universally confirmed.
  • William James's Theory: A prominent psychologist from the late 1800s and early 1900s, James proposed that physiological reactions precede emotional interpretation.
    • Traditional view: See bear \rightarrow feel afraid \rightarrow tremble.
    • James's view: See bear \rightarrow body automatically trembles \rightarrow interpret trembling as fear ("I'm trembling, I must be afraid.").
  • Physiological Pathway: Sensory information (excluding smell) passes through the thalamus, which consolidates it and triggers the hypothalamus. The hypothalamus regulates the body into sympathetic (e.g., increased heart rate, narrowed pupils, reduced saliva/digestion, sweating) or parasympathetic (e.g., increased saliva/digestion, relaxed, dilated pupils) reactions. This is an automatic, partly endocrinological process, not under conscious control.
  • Mechanism: If sensory information indicates a smile (e.g., muscles contracted), the less conscious brain interprets this as happiness, leading to a more positive mood evaluation.
  • Frowning Evidence: There is less convincing evidence that frowning significantly influences mood negatively.
  • Botox Experiment: A study involving injecting Botox (which tightens skin and forces a smile) into participants' foreheads found that some consistently reported better moods a month later, compared to a baseline assessment. This suggests that the physical act of smiling can induce positive mood changes.
  • Embodied Cognition Link: This aligns with embodied cognition, as it suggests that the interpretation of subconscious physical states (not conscious cortical activity) influences our emotional experience.
  • Sarcasm Caveat: If an individual is genuinely in a bad mood, forcing a smile might lead to an interpretation of sarcasm, potentially worsening their mood, as shown in some studies.

Situation Models and Theory of Mind

• Radvansky's Situation Models: People construct mental models of ongoing situations, known as situation models.
  • Only information relevant to the current situation model is likely to be retained in memory.
  • Example: In a narrative about a jogger, one is more likely to remember the jogger "putting on a sweatshirt" (relevant to the model) than "taking off a sweatshirt" (no longer relevant once removed).
  • "Doorway Effect": Passing through a doorway in a narrative causes people to forget items associated with the room they just left, suggesting a mental update of the situation model.
• ** embodied cognition and neural activation**:
  • Mirror Neurons: Observing someone perform a movement activates corresponding neurons in the observer's brain regions associated with that movement.
  • Language and Action: Hearing words that describe actions (e.g., "kicking," "grabbing") also activates related brain areas.
  • Narrative Immersion: When reading fictitious narratives, individuals update their working memory to align with the developing situation model, effectively putting themselves into the scenario.
• Connection to Theory of Mind: These embodied cognition phenomena are deeply linked to Theory of Mind, our automatic, computationally complex ability to understand others' intentions, knowledge, attention, and goals.

Gesture as an Essential Component of Thought and Communication

• Gesture as Fundamental: Gesture is integral to thinking and language, demonstrated by motor neural reactions to observed actions or associated words, and the existence of motor signatures tied to gestures and actions.
• Tower of Hanoi Task: This classic puzzle involves moving disks between three pegs, adhering to specific rules.
  • Experiment: Participants solved the puzzle either physically or on a computer screen (sliding disks).
  • Finding: When participants first solved the puzzle physically and then again on a computer, they tended to mimic the physical lifting and dropping motions on the computer, even though it was unnecessary. Those who only used the computer interface simply slid the disks.
  • Implication: Gesture preserves "embodied traces of past actions," indicating a deep connection between physical actions and subsequent mental or digital performance.
• Müller-Lyer and Ebbinghaus Illusions: These visual illusions involve misperceiving line lengths or circle sizes.
  • Experiment: Participants were either asked to estimate the size/length or imagine grabbing the objects.
    • Estimation: People were fooled by the illusions.
    • Imagined Grasping: People were not fooled and perceived the objects accurately.
  • Dual Stream Hypothesis: Explains this phenomenon using the brain's visual processing streams:
    • Ventral Stream (The "What" Stream): Associated with object recognition, memory (temporal lobe); is susceptible to visual illusions.
    • Dorsal Stream (The "How" and "Where" Stream): Associated with action, motor control (parietal, frontal lobes, motor cortex); processes visual information accurately for action and is not fooled by these illusions.
  • Implication: Gesture appears to be carried out via the dorsal stream, demonstrating procedural, implicit, unconscious, and automatic processing. It functions more like action than perception.
• Gesture Enhances Performance and Memory (Dual Encoding):
  • Cognitive Benefit: People perform tasks better when they gesture, and they store more memory traces.
  • Dual Encoding Theory: Information encoded in multiple ways (e.g., verbally and physically) is remembered more effectively and retrieved faster.
  • Traffic Light Analogy: A single light that only changes color (one encoding) leads to more accidents and slower responses than a standard traffic light (red on top, yellow in middle, green on bottom), which uses dual encoding (color and position) for better, faster, and more effective reactions.
  • Learning: Gesturing while speaking, or learning from someone who gestures while speaking, improves retention and learning due to dual encoding (visual gesture + auditory speech signal).
  • fMRI Math Study: Children learning math either gestured while speaking or only spoke. A week later, during an fMRI session where they solved math problems without moving, the children who had gestured previously showed motor activity in hand movement areas, even when physically still. This demonstrates that gesture leaves motor traces linked to thinking, verbalizing, and learning.

Unique Insights from Gesture: Patient IW and Language Structure

• Patient IW and Proprioception: Patient IW suffered a destruction of motor-sensory connections, resulting in a loss of proprioception.
  • Proprioception: The sense of the relative position of one's own body parts and strength/effort of movement, often overlooked as a primary sense.
  • Impact: IW could not move or control limbs without visual feedback (e.g., had to look at his arm to raise it) and was unable to move in the dark.
  • Gesture: Interestingly, IW could still automatically gesture while speaking, even in the dark. This demonstrates that gesture is automatic, implicit, and deeply integrated with communication and speech, separate from conscious proprioceptive control.
• Path Encoding vs. Manner Encoding Languages: Languages differ in how they encode path and manner of motion.
  • Path Encoding (e.g., Spanish, Turkish, Romanian): Typically expresses manner and path in two separate clauses (e.g., "She entered the room waltzing").
  • Manner Encoding (e.g., English): Often combines manner and path in a single clause (e.g., "She waltzed into the room").
  • Gesture Patterns: Gesture patterns tend to align with language structure.
    • Manner-encoding language speakers often use one gesture for such combined actions.
    • Path-encoding language speakers often use two separate gestures (one for path, one for manner).
  • Congenitally Blind Individuals: Remarkably, people born blind also gesture, despite never having observed others. Even more surprisingly, their gesture patterns mirror their language structure (e.g., two gestures for path-encoding languages, one for manner-encoding languages). This suggests that the connection between language and gesture is intrinsic and not solely learned through imitation or visual observation.

The Multifaceted Benefits and Functions of Gesture (Golden Meadow's Analysis)

• Gesture Aids Attention: Gesturing (e.g., pointing to a diagram) helps focus attention and leads to better memory, leveraging dual encoding and embodied interaction.
• Gesture Offloads Working Memory: Like external cognitive tools (e.g., writing notes), gesture reduces cognitive load by offloading information from working memory.
  • Experiment: Participants performing a task with gesture better recalled random numbers memorized beforehand, suggesting gesture frees up cognitive resources.
• Gesture Externalizes Thought: Gesture provides a window into internal thoughts, even unconscious biases.
  • Deception Studies: Gestures can reveal true intentions even when spoken words are deceptive (e.g., gesturing differently when lying about an event).
  • Bias Studies: In discussions about co-workers, men's gestures unconsciously displayed lower vertical references for women's skills compared to men's, despite neutral verbal descriptions, revealing hidden biases.
• Gesture Indicates Readiness to Learn: Children who show a mismatch between their words (incorrect) and gestures (correct) during math tasks are often at a pivotal point, ready to learn a new concept.
• Gesture Offers Different Representational Formats: Unlike arbitrary words, gestures can be iconic (e.g., three fingers for "3"). They enrich communication by conveying information not explicitly stated verbally (e.g., gesturing a descending spiral while saying "spun in a circle"). Gesture can also fill gaps in speech when words are hard to retrieve.
• Gesture Functions in Noisy Channels: In loud or difficult listening environments (e.g., during COVID with masks that obscure lip-reading), people naturally gesture more, lean closer, and focus on non-auditory cues. Gesture supplements verbal communication, providing visual information to overcome auditory interference.
• Gesture Nurtures Abstraction: Gesture helps transition from concrete to abstract concepts.
  • Child Language Acquisition and Undergeneralization: Young children often undergeneralize words (e.g., "phone" applies only to their specific phone).
  • "Tiffin" Toy Experiment: Children who gestured squeezing a toy showed improved generalization of the word "tiffin" to other squeezable objects, whereas those who actually squeezed it continued to undergeneralize. This implies that conceptualized gesture promotes abstract thinking by representing an action as a concept rather than a specific act.
• Gesture and Speech as Integrated Systems: Gesture and speech are not separate but are planned simultaneously and cooperatively. Evidence suggests they are integrated during speech production.

What Gestures and Other Cues Reveal

• Gesture's Insights: Gesture is established as an integral part of thought, language, learning, and abstract thinking, leaving motor traces and benefiting communication.
• Windows to Thought: Beyond gesture, other physiological cues offer insights:
  • Pupillometry: Pupil dilation/contraction reveals attention, emotional arousal, and cognitive load.
    • Police DUI Tests: Involve observing eye muscle control (e.g., eyes wiggling) as an indicator of intoxication.
    • Eye Movements: Automatically reflect prediction, attention, and emotional reactions (e.g., looking up for positive words, down for negative, left/right for causation).
  • Language: Verb tense choices and Freudian slips can also reveal underlying thoughts.
  • Animal Posture/Human Power Poses: Body posture communicates power or submission.
    • Power Stances: (e.g., legs apart, hands on hips or crossed on chest) can make individuals feel more confident, less nervous, and project authority.
    • Gambling Study: Individuals adopting power stances tended to feel they wanted more money and took greater risks.
    • Hormonal Shifts: Prolonged power stances can even lead to hormonal changes associated with power versus submission.