done ✅Multimodal Discourse Analysis — Page-by-Page Notes
Page 1
Topic: Multimodal Discourse Analysis as introduced in Paltridge’s Discourse Analysis: An Introduction (2nd edition).
Companion website for the book is available online: http://linguistics.paltridge2e.continuumbooks.com
Access: type the URL to receive a unique password for access to online resources for the book.
Troubleshooting: If problems accessing resources, contact Bloomsbury at info@continuumbooks.com.
Page 2
Bloomsbury Discourse Series overview
Series Editor: Professor Ken Hyland, University of Hong Kong
Core idea: Discourse is central to humanities and social sciences because language mediates and shapes interactions with social, political, and cultural formations.
Aim of the series: Capture fast-developing interest in discourse; provide an essential bookshelf for students, teachers, and researchers in applied linguistics, ELT, and English language.
Each book covers a core topic in discourse studies to give readers an in-depth, structured introduction to how language operates in real life.
Other titles in the series (examples):
The Discourse of Online Consumer Reviews — Camilla Vásquez
News Discourse — Monika Bednarek and Helen Caple
The Discourse of Text Messaging — Caroline Tagg
The Discourse of Twitter and Social Media — Michele Zappavigna
Workplace Discourse — Almut Koester
The Discourse of Blogs and Wikis — Greg Myers
Professional Discourse — Britt-Louise Gunnarsson
Academic Discourse — Ken Hyland
School Discourse — Frances Christie
Historical Discourse — Caroline Coffin
Using Corpora in Discourse Analysis — Paul Baker
Metadiscourse — Ken Hyland
Page 3
Publication details (front matter, copyright, and cataloging information)
Bloomsbury Academic, an imprint of Bloomsbury Publishing Plc, locations: London & New York; addresses listed.
First published 2012; © Brian Paltridge, 2012. All rights reserved.
Legal notice: No part of the publication may be reproduced or transmitted without permission.
Paltridge asserted authorship rights under the Copyright, Designs and Patents Act, 1988.
British Library Cataloguing-in-Publication Data and Library of Congress data included; ISBNs provided for various formats (alk paper, pbk alk paper, eBook pdf/epub).
Typeset by Newgen Imaging Systems Pvt Ltd, Chennai, India; Printed and bound in India.
Page 4
Introduction to Multimodal Discourse Analysis
Multimodal discourse analysis studies texts that combine multiple semiotic resources beyond words (e.g., pictures, film, video, sound).
Example: 11 September events were communicated not only through verbal reports but also through televised images; images contributed to creating proximity and involvement with events (Machin 2007; Kress 2010).
Multimodality affects both print and television genres: e.g., Cosmopolitan magazine uses images alongside text to construct meaning about women.
Cosmopolitan: global brand with 44 editions worldwide; local editors even visit New York to align with a consistent focus and format across editions (Machin & Thornborrow 2003; Machin & van Leeuwen 2007).
The magazine sells not only products but values (independence, power, fun) through multimodal presentation: background, setting, color, lighting, and portrayal of women in relation to men and power.
Women in Cosmopolitan are depicted as fun and fearless with a take-charge attitude; power is linked to bodily presentation rather than professional competence (Machin & Thornborrow 2003).
Multimodal presentation includes clothing, lipstick, hair, places women shop and go to for lunch; all contribute to a reading of the magazine’s messages.
Central claim: Cosmopolitan aligns readers with the magazine’s brand values, expressed through multimodal means and cross-promoted products (e.g., Cosmo cafés, fashion, entertainment).
8.1 Background to multimodal discourse analysis
Theory rooted in Halliday’s social semiotic approach: language as one semiotic resource among many (gesture, images, music) used to communicate meaning within sociocultural contexts.
Multimodal discourse analysis seeks to describe socially situated semiotic resources used for communication.
Texts draw on multiple modes (pictures, film, video, images, sound) in combination with words to make meaning; analysis examines how semiotic tools (color, framing, focus, element positioning) contribute to meaning.
Halliday (2009a) identifies three types of social meaning realized simultaneously in language use:
Ideational meaning: what the text is about
Interpersonal meaning: relations between participants
Textual meaning: how the message is organized
In multimodal texts, these meanings are realized visually (ideational/representational, interpersonal/modal, textual/compositional) in images and their arrangement (de Silva Joyce & Gaudin 2007).
Foundational references to: Kress & van Leeuwen (Reading Images, 2006); Kress (Multimodality, 2010); van Leeuwen (Introducing Social Semiotics, 2005); O’Halloran (Multimodal Discourse Analysis, 2004); Painter, Martin & Unsworth (Reading Visual Narratives, 2012).
Jewitt (2009a) outlines four key assumptions about multimodal analysis: (1) language is one of several equally meaningful modes; (2) other modes realize different meanings; (3) people configure modes to make meaning; (4) multimodal meanings are social and shaped by genre conventions.
Page 5
8.2 Examples of multimodal discourse analysis (continued)
People typically use multiple modes simultaneously (images, gesture, gaze, posture) along with language (Machin 2007; Jewitt 2009a).
Each mode has affordances (Gibson 1977) within a context: what it can represent or express.
Meaning potential of a mode is shaped by its past uses, recurrent meanings, and social conventions (van Leeuwen 2005a; Jewitt 2009a).
Images can have their own grammar: mood (eye contact) and perspective (angle) influence interpretation:
Eye contact can indicate demand vs. offer; perspective (horizontal vs. high/low angle) suggests involvement or hierarchy.
Distance (close, medium, long), lighting, color, focus, and the degree to which an image reflects reality affect interpretation.
Time Magazine (1 June 2009) cover example: Michelle Obama image with text “The Meaning of Michelle.”
Analysis highlights eye contact, full-page central placement, lighting, focus, horizontal orientation, direct gaze, and salience as a way to connect with readers.
Page composition conveys information value and salience; the image dominates the page, framed within a vertical triptych layout; interplay with the caption and title creates intermodal complementarity (Painter & Martin, forthcoming).
Social-semiotic theory (Kress, 2010) asks: what meaning is being made, how it is made, what resources are used, in what social environment, and whose interest/agency is at work?
The Time cover’s foregrounded image carries the major informational load; its design communicates a message about Michelle Obama that aligns with the article’s content and broader political context (Obama era).
Online vs print: The online Time page features a different, less dramatic image of Michelle Obama, with similar communicative aims; readers navigate to the full article by scrolling or printing.
8.3 Genre, speech acts and multimodality
Van Leeuwen (2005a, 2005b) connects speech acts and genre to multimodality, focusing on the ‘how’ rather than the ‘what’ of multimodal communication.
Illocutionary act: the intended action of the speech act (e.g., persuade, command, promise).
Perlocutionary effect: the effect on readers/listeners (e.g., convincing to buy, change attitude).
Texts rely on multiple modalities (linguistic and visual resources) to achieve perlocutionary effects.
Genres provide templates for doing things: service encounters have conventionalized stages (e.g., purchasing) that may combine language and actions; other genres (e.g., ATM use) combine verbal and visual instructions.
In some contexts (e.g., IKEA stores), customers perform tasks themselves with minimal or no interaction with staff, illustrating how genres organize multimodal interaction.
Page 6
8.2 Examples of multimodal discourse analysis (continued)
Across genres, multimodal texts involve images, gaze, posture, etc., integrated with language to produce meaning.
The concept of affordances and how context shapes what a mode can convey.
The role of perspective (eye contact, shot distance) in signaling social relations (inclusion, distance, power).
Page 7
8.2 Examples of multimodal discourse analysis (continued)
Further discussion of image grammar and viewer relations: mood, perspective, distance, lighting, color, focus, and reality versus stylization.
The Michelle Obama Time cover example is analyzed in terms of eye contact, front-on angle, extreme close-up, central placement, and salience to create a strong interpersonal connection with readers.
8.2 Examples (case study): The Time cover and its layout elements are used to illustrate how image and text work together to convey meaning and persuade readers.
8.3 Genre, speech acts and multimodality (continued)
Reiterates the link between illocutionary/perlocutionary aims and the multimodal resources used to achieve them.
Uses the service encounter as an example of how genre templates shape multimodal text design.
Page 8
8.3 Genre, speech acts and multimodality (continued)
Additional points on how genres function as templates for performing social actions in culturally specific ways.
The IKEA example illustrates mixed verbal and visual guidance within a multimodal genre to facilitate self-service purchase.
8.4 Carrying out multimodal discourse analysis
Analytical process mirrors other discourse analysis but with attention to multiple modes.
Data collection and logging steps: collect data, log (summarize), add contextual notes and first impressions for later depth.
When data is multimodal (print, web, video), analysts document how each mode contributes to meaning.
After initial reduction, select key extracts for in-depth analysis; then verify whether findings generalize within the dataset.
Baldry & Thibault (2005) offer detailed guidelines for transcribing and analyzing multimodal data across media (ads, cartoons, web pages, film/TV genres).
Key concept: clusters (groupings of items) and cluster analysis focus on connections between clusters rather than isolated items; relationships create meaning in context.
In film/TV, attention to shots, phases, transitions, and the relation between soundtrack and images helps situate meaning within context of situation and culture.
Ethnographic data and contextual data help link multimodal analysis to production/consumption contexts (Prior 2013; Street, Pahl & Rowsell 2009; Molle & Prior 2008).
8.5 Limitations of multimodal discourse analysis
Time-intensive; interpretable and subjective nature can lead to varying readings.
Analyses often focus on final products rather than production processes or readers’ receptions.
Some studies under-emphasize language in favor of other modes; linking to broader social theories helps offset limitations (Machin & van Leeuwen 2007).
Ethnographic data can strengthen analyses by connecting multimodal findings to social contexts.
8.6 Summary
Overview of multimodal discourse analysis: concept, theory, and empirical examples.
Relationships among genre, speech acts, and multimodality.
Introduction of a genre-and-multimodality framework and its layered analytical description.
Practical guidance for conducting multimodal analyses, along with limitations and directions for further research.
Page 9
8.7 Discussion questions
(1) Front page of a newspaper: How does image use, color, font, and text size guide your response?
(2) Magazine cover: What strategies give the cover images salience?
(3) Memorable film scene: How do color, sound, costuming, and setting influence impact?
8.8 Data analysis projects
(1) Collect magazine covers with photos of people; analyze using Kress (2010) questions: meanings, resources, social environment, and agency.
(2) Collect image-nuclear news stories from online newspapers; analyze text-image-caption relations and intertextual allusions (Caple, 2009).
(3) Collect web pages from a site you visit; analyze with Bateman’s Genre and Multimodality framework.
(4) Record a short TV program; analyze using Iedema’s framework for film/TV genres.
Page 10
8.9 Exercises (continued) and 8.3–8.4 integration
Exercise 1: Image-nuclear news story analysis of a Mao Zedong hand sculpture image and its text-caption relationship; identify text components and their relation to the image.
Exercise 2: Book cover analysis: layout, horizontal vs vertical triptych, relationship between text and image, information hierarchy, font choices, and placement.
Exercise 3: Film trailer analysis (Bridget Jones’s Diary): analyze Orientation section using Maier’s framework; determine modes indicating Orientation.
8.10 Directions for further reading
Bezemer, J. & Jewitt, C. (2010): Multimodal analysis key issues (methodology and data collection/analysis examples).
Jewitt, C. (ed.) (2009): Routledge Handbook of Multimodal Analysis; Chapter 1 provides background; Chapter 2 outlines approaches (social semiotic vs. systemic functional).
Additional recommended reads: Kress & van Leeuwen (Reading Images, 2006); Machin (Introduction to Multimodal Analysis, 2007); van Leeuwen (Introducing Social Semiotics, 2005).
The companion website provides extended references and resources.
Page 11
8.4 Carrying out multimodal discourse analysis (online continuation)
Emphasis on integrating print and online media genres (image-nuclear stories, newsbites) and their interrelations.
Online content, home pages, and section pages provide navigational cues and social interactivity that influence interpretation.
Caple (2009) and Knox (2007, 2010) illustrate how online newspaper genres differ from print (image-nuclear stories vs. newsbites).
Page 12
8.4 Carrying out multimodal discourse analysis (film/TV frameworks)
Iedema (2001) provides a hierarchical framework for analyzing film/TV genres: frame, shot, scene, sequence, generic stage, work as a whole.
Table 8.2 (summarized):
Frame: a salient still from a shot
Shot: unedited camera movement within a shot
Scene: a sequence of shots within a time-space
Sequence: camera moves across time-spaces with subtopics/characters
Generic stage: beginnings, middles, endings; genre-specific stages
Work as a whole: overall classification and relation to social/cultural orders
Obama 2008 victory speech example: single frame analysis on the Grant Park stage; orientation, social distance, horizontal perspective, and central framing; sequence of shots from crowd to podium; identification of expository vs. rhetorical genre features.
Contextual notes: contextual factors (time of day, audience, broadcast context) influence interpretation (Baldry & Thibault 2005; O’Halloran 2004/2011).
8.5 Limitations (continued)
Further caveats about generalizing from multimodal data; complexity of context; need for triangulation with ethnographic data.
Page 13
8.3–8.4 Case studies: film trailers and multimodality frameworks
Maier (2011) on film trailers: trailers contain implicit promotional stages followed by explicit promotional stages; voiceover, music, and editing contribute to the overall effect.
The structure typically includes diegetic information (story-related content) and non-diegetic information (promotional content such as voiceover, evaluations, and recommendations).
The modes of voice, music, images, and text work together to achieve the trailer’s purpose.
8.4 Carrying out multimodal discourse analysis (closing notes)
The process includes repeated viewing (sound and vision together, then separately) to isolate each mode’s contribution.
For print or web data, analyze how each mode (words, font, layout, images) contributes to meaning and how these are contextualized in notes.
Page 14
8.4 Carrying out multimodal discourse analysis (continued)
Table 8.3: Stages of comedy film trailers and their functions (Maier 2011, p.147)
Implicit promotional (diegetic information): Prologue, Appetiser, Orientation, Complication, Evaluation
Explicit promotional (non-diegetic and diegetic information): Promotional identification, Promotional recapitulation, Promotional interpretation, Promotional recommendation, Promotional information (e.g., website address, release dates)
The staged progression contributes to how audiences interpret and engage with trailers.
The overall aim is to show how multiple modalities (sound, vision, text) function together to promote the film and shape audience expectations.
Page 15
8.4 Carrying out multimodal discourse analysis (data handling)
After data collection and logging, repeatedly view the data to understand contributions from each mode.
When analyzing any multimodal data, note how features such as color, layout, typography, and image composition influence interpretation.
Use contextual notes to interpret how the modes relate to production/consumption contexts.
Baldwin and Thibault (2005) emphasize clustering as a way to connect items; this helps to understand meaning in context rather than focusing on isolated pieces.
Ethnographic data, such as interviews and observations, can illuminate how texts are produced and received.
Page 16
8.5 Limitations (summary)
Time, interpretation, and reader/viewer variation remain central concerns.
Multimodal analysis can underemphasize language; linking to social theories (global media discourse) helps address this.
Ethnographic approaches can provide deeper context for multimodal texts and their production/consumption.
8.6 Summary (reiterated)
Multimodal discourse analysis investigates how texts use more than one mode to communicate meaning.
It includes theoretical backgrounds, exemplars, and a practical framework for analysis.
Connections among genre, speech acts, and multimodality are highlighted.
A genre-and-multimodality framework is proposed to describe layers of meaning in multimodal texts, along with practical steps and limitations.
Page 17
8.7 Discussion questions (reiterated)
(1) News front page composition: image, color, font, and size affect interpretation.
(2) Magazine cover salience: image strategies to draw attention.
(3) Memorable film scene: color, sound, costume, setting influence impact.
8.8 Data analysis projects (reiterated)
(1) Magazine covers: meanings, resources, social environment, and agency using Kress (2010) questions.
(2) Image-nuclear news stories: analysis of headings, images, captions; intertextual allusions (Caple 2009).
(3) Web pages: apply Bateman’s framework.
(4) Short TV program: apply Iedema’s framework.
Page 18
8.9 Exercises (continued) and references to figures
Exercise 1: Image-nuclear story (Mao Zedong hand sculpture) with text and its relation.
Exercise 2: Book cover analysis (old edition figure 8.2): layout type, words-image relationship, information value, and fonts.
Exercise 3: Bridget Jones’s Diary trailer: analyze Orientation section using Maier’s framework; identify modes used for Orientation.
8.10 Directions for further reading (detailed references)
Bezemer, J. & Jewitt, C. (2010): Multimodal analysis: key issues (in Research Methods in Linguistics, L. Litosseliti). A practical and accessible overview with classroom interaction example.
Jewitt, C. (ed.) (2009): The Routledge Handbook of Multimodal Analysis. Chapter 1: background; Chapter 2: approaches (social semiotic vs. systemic functional).
Other core texts for extended readings: Kress & van Leeuwen (Reading Images, 2006); Machin (Introduction to Multimodal Analysis, 2007); van Leeuwen (Introducing Social Semiotics, 2005).
The companion website provides an extended list of references and readings.
Page 19
Figure references
Figure 8.1: Image-nuclear news story example (Caple’s discussion referenced).
Figure 8.2: Cover of the first edition of Discourse Analysis (Paltridge, 2006).
Page 20
Final bibliographic notes and additional references
Summary of major texts used throughout the chapter for multimodal discourse analysis and related frameworks.
Emphasis on the integration of textual, visual, and contextual analysis to understand how meaning is produced and interpreted in modern media.