2.2.1 COGNITIVE Body Schema and Multisensory Integration

1. Explain the challenges that need to be solved by the brain when integrating information from multiple senses Reference frame, coordinate transformation, eye - /head - /body - centred reference frames 2. Define the concept of ‘body schema’ and explain disorders of body schema Body schema, body image, Alice in Wonderland syndrome, Autotopagnosia, Phantom limbs 3. List the neuroscientific and psychological evidence for multisensory integration in the brain multisensory receptive fields, cross - modal congruency and extinction, temporal- order judgements 4. Discuss why and how tools can be integrated into the body schema in light of the scientific evidence tool use, mirror experiments, peripersonal space

Moravec (1998)

it is comparatively easy to make computers exhibit adult level performance on intellgience tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility

Moravec’s paradox

high level cognitive reasoning → relatively easy

low level reasoning- perception, action → surprisingly hard

computers and task ability

• good at “hard” tasks w higher level reasoning, e.g. chess bots, chat gpt

• bad at low level tasks easy tasks e.g. movement, picking stuff up, perception

Johanasson (2005): match

• investigating Moravec’s paradox

• what does it take to light a match?

• P lights match, normal time- quick and easy

• then anesthetise the fingers → blocks all touch sensation from fingers but does not affect motor control

• other senses like vision still in tact, only thing changed is tactile input

• P struggles to pick up match, dropping it, makes clumsy movements, takes 25 seconds

→ shows importance of tactile input/perception

→ multisensory integration

multisensory integration

touch + vision

→ see and feel that am touching the object

two challenges in mutlisensory integration

• how to transform representations from different senses into a common representational space?

• how to integrate information from different senses into a coherent percept?

localisation of sensory percepts

• vision → eye-centred/retinal

• audition → head centred

• touch → body centred

vision location

location of visual stimulus on the retina

audition location

location of sound source with respect to the ears

touch location

location of tactile stimulus on skin

reference frames

body based

need to convert….

between these reference frames and to external space (world coordinates)

reference frame snake game

• player perspective: coordinates of snake in the world, e.g. upper left corner

• snake perspective: sees world thru its eyes, can turn left or right

• problem:
  sensory input in player perspective but controls in snake perspective

coordinate transformation types

• eye to head

• head to body

eye to head coordinate transformation

need to know orientation of eyes

head to body coordinate transformation

need to know orienation of head

coordinate transformations

• converting between reference frames

• requires us to know the position and orientation of body parts

• need body schema

the body schema

• a representation of the positions of body parts in space, which is updated during body movement

• sensorimotor representation that guides action, does not require awareness

body image

• how you imagine, feel about your body

• conscious, mostly visual representation

very different from the body schema

Haggard & Wolpert, 2005: features of the body schema

1. spatially coded

2. modular

3. updated with movement

4. adaptable

5. supramodal

6. coherent

7. interpersonal

how is body schema coded?

spatially → position of each body part in external space

how does body schema process body parts?

modular → diff body parts processed separately

how does body schema update?

updated with movement → automated and always on

how changeable is body schema?

adaptable → changes when body changes

what modality is body schema?

supramodal → combines input from multiple senses, e.g. proprioception, touch, vision, audition

how is body schema with perception?

coherent → spatial continuity when resolving perceptual conflicts

how is body schema represented?

interpersonal → observed actions represented within the same body schema

two types of bodily illusions

1. pinocchio illusion

2. rubber hand illusion

Pinocchio illusion

Kilteni et al., (2015)

blindfolded:

1) vibrating the biceps makes arm feel extended

2) if arm is extended, but we are grasping our nose → nose must be really long

Rubber Hand illusion

Kilteni et al., (2015); Botvinick & Cohen, 1998

setup:

• Ps real hand hidden from view

• experimental strokes real and rubber hand simultaneously

• P sees and FEELs rubber hand being strokes

• P starts to feel as if rubber hand is real hand

vision + touch = multisensory integration

body schema + perception: study, does body posture affect perception?

Heed & Azañón, Front Psychol, 2014

Heed & Azañón, Front Psychol, 2014: procedure

• does body posture affect perception?

• temporal order judgement task

• setup: stimulate both hands in random order

• task: indicate which hand was stimulated first

• conditions: arms uncrossed/crossed

Heed & Azañón, Front Psychol, 2014: findings

• Ps mix up which hand was stimulated when arms are crossed

• solving the task does not need input from body schema

• → body schema affecting basic perception

how does the body schema develop?

at 6mo, the body schema starts to interefere with tactile orienting (Begum Ali et al., 2015)

→ correct tactile orienting responses same at four months regardless of if feet crossed or not

→ if uncrossed, same at 6mo as 4mo. if crossed at 6mo, significantly less correct response on tactile orientating task

Maravita et al., Curr Biol, 2003: cross modal integration and body schema

• Ps recieve tactile stimulus and respond with the location (up or down)

• visual distractor (up or down) is presented nearby on same hand or other hand

• congruent and incongruent

• congruency effect = RT for incongruent - RT for congruent

• congruent rt less

• a congruency effect exists for both hands → evidence for cross-modal integration and visual interference

• but, effect is greater for same (congruent) hand

Maravita et al., Curr Biol, 2003: is the greater interefere based on hand location (body schema) or visual field?

arm crossing - changing the mapping of the body schema

• tactile stimulus on same side of body

• visual stimulus on different side of body

→ effect of visual distractor moves with the hand during arm-crossing

→ body schema (not visual representation of external space) mediates integration between vision and tactile stimulation

• cross modal interactions are mediated by body schema

disorders of body schema

• alice in wonderland syndrome

• autotopagnosia

• phantom limbs

alice in wonderland syndrome

• distortion in size perception

• body parts might appear smaller (microsomatognosia) or larger (macrosomatognosia) than their true size

• can effect whole body

• associated with childhood and migraines

autotopagnosia

• inability to locate body parts

• loss of spatial unity of body

• patients can name body parts but relative order lost

• finger agnosia: fused percept of fingers

phantom limbs

• after limb loss, presence of limb still felt

• can include agency over limb (movement)

• often associated w pain

• phantom limb can change size of time, e.g. shrink, telescoping

peripersonal space

• the space immediately surrounded our bodies

• objects within peripersonal space can be grasped and manipulated immediately

• extent not fixed- can contract and expand

Serino, 2019

neurons incorporating tools

• expansion of peripersonal space during tool use reflected in neural responses

• tools are incorporated into the body schema during use

Marvita et al., 2003

tool use: extending the body

• tools incorp’d into body schema during use

• cross modal congruency effects apply during tool use:

→ no crossing of body parts, only tools crossed

→ same delay effects as seen during arm crossing

• tools become part of body schema

Marvita et al., 2003