PURCOM 4, Multimodality

Multimodality

use of several modes in communicating a person

Modes Involved:

• Lingual

• Visual

• Aural

• Gesture

• Speaking

Matthiesen and Gomez’s notion on Multimodality

Matthiesen (2007) and Gomez (2015) show a connection between gestures and meaning in early and later language use.

Non-verbal Elements on Speech

• Tone of Voice

• Pitch

• Volume

• Rhythm

• Intonation

Paralinguistic

• non-verbal elements of speech such as voice, pitch, rhythm, intonation

Multimodality approaches the following beyond language, Bezemer and Jewitt (2014)

• Communication

• Representation

• Interaction

Semiotic Resources in Studying Communication, (Camiciottoli and Gomez, 2015)

• Linguistic

• Visual

• Aural

• Spatial

Linguistic

Vocabulary, generic structure of grammar

Visual

color, vectors, and viewpoint in still-moving objects

Aural

pitch, volume, rhythm of music, and sound effects.

Gestural

features movement,

Spatial

proximity, direction, the layout and organization of objects in space

Kress and Van Leeuwen’s insight on Design and Multimodality

Design can reinforce itself by using multiple modes in complementary roles of hierarchical order

________________ notes that the most significant change has been the transition from paper to digital screens

Kres (2010)

The ________ is the carrier of the meaning

image

Halliday’s insight on multimodality, 1989

Text creation involves more than just spoken and written forms, encompassing various other modes, known as multimodal text.

Anstey and Bull’s insight on multimodal text, 2010

two or more combination of semiotics system are called multimodal text

Who claimed that multimodal texts require practice to interpret and analyze and required in-depth analysis to detect signals and avoid biases in communication

Batemann, 2008

Kress and Van Leeuwen’s Various Factors on Multimodal Texts, (1996)

• Representational Meaning

• Interactive Meaning

• Compositional

Representational Meaning

• content

• social context

• symbolism

• reading paths

• reactions

Interactive Meaning

• actions

• gaze

• framing

• modality

• background

• color

• saturation

• power

• status

Compositional Meaning

• salience

• positioning

• layout

• typeface

• language techniques

• lighting