Experiential Influences on Multimodal Perception of Emotion

To examine the impact of two types of learning experiences on children’s perception of multimodal emotional cues.
- Types of experiences:
- Familiarity with the emotional expresser (familiar vs. unfamiliar adults).
- Salience of particular emotional expressions (comparison between abused and non-abused children).
Central hypothesis: Children learn to decode emotional signals from multiple cues early in life.

Age Range: 7 to 12 years
Total Participants: 63 children
- Abused Group: 33 children from substantiated cases of maltreatment.
- Control Group: 30 non-abused children.
Demographics: Efforts were made to recruit similar family demographics for both groups.

Tested the familiarity's impact by presenting emotions via familiar and unfamiliar adults.
Investigated the salience of anger expressions among abused compared to non-abused children.

Children were presented with conflicting facial and vocal emotions in various scenarios:
- Two types of mothers (familiar/stranger).
- Emotional content: happiness, sadness, anger, respectively.
Procedure: Each child completed a series of trials in a sound-controlled room with visual and auditory stimuli presented synchronously.
Tasks to evaluate how well children processed multimodal cues.

Emotional messages are conveyed through:
- Linguistic cues: The semantic content of spoken language.
- Paralinguistic cues: Facial expressions, vocal tone, physical gestures, body posture.
Congruent cues facilitate efficient processing of emotional signals, while conflicting cues challenge integration.
Child development research shows that infants can accurately perceive emotional cues from as young as 6 months, indicating early sensitivity to multimodal signals.

Emotional learning begins within family contexts as infants learn to interpret their caregivers' expressive behaviors.
Familiarity enhances a child's ability to process emotional signals.

Children show a preference for auditory cues when expressed by their mothers, but reliance shifts based on familiarity.
Developmental shifts in modality preferences:
- Infants favor auditory over visual input in recognizing emotions; adults show the opposite trend.

Overall Findings: No consistent preference for auditory or visual modalities across all scenarios.
Emotional Salience: Children recognized happy facial expressions more readily than happy vocal cues.
Familiarity Effects: When emotions were conveyed by familiar adults, children preferred processing audio signals; visual signals were preferred with unfamiliar adults.

Children processed emotional expressions differently based on the familiarity of the individual expressing emotion:
- Familiar Faces: Preference for facial processing.
- Unfamiliar Faces: Preference for auditory processing.
Physically abused children demonstrated increased sensitivity to anger, especially with their abusive mothers, indicating a contextual and survival-oriented perceptual advantage.

Statistical Measurements: ANOVA with maltreatment group as a between-subjects factor and emotion, familiarity, and modality as within-subjects factors.
Abused children identified anger significantly more than non-abused children in scenarios involving mothers expressing anger vocally.

Familiarity with emotional expressions affects comprehension and recognition rates of emotions among children.
The heightened attention toward vocal anger in abused children reflects an adaptation to an abusive upbringing.
This study emphasizes the need to incorporate both auditory and visual modalities in the study of emotional perception, moving beyond predominant analyses focusing solely on facial expressions.

Explore deliberate control of attention toward differing modalities in children.
Investigate auditory expressions in various contexts, including abusive backgrounds, to enhance understanding of emotional learning environments.

Key studies mentioned include:
- [Bachorowski, J. (1999)] - Vocal expression and perception of emotion.
- [Ekman, P. (1993)] - Facial expression and emotion recognition.
- [Robinson, C. W., & Sloutsky, V. M. (2004)] - Changing modality preferences in children.

Thanks to collaborators and financial support from the National Institute of Mental Health and University of Wisconsin Graduate Fellowship.
Special thanks to participating children and their families.