done ✅Multimodal Discourse Analysis — Page-by-Page Notes

Page 1

  • Topic: Multimodal Discourse Analysis as introduced in Paltridge’s Discourse Analysis: An Introduction (2nd edition).

  • Companion website for the book is available online: http://linguistics.paltridge2e.continuumbooks.com

  • Access: type the URL to receive a unique password for access to online resources for the book.

  • Troubleshooting: If problems accessing resources, contact Bloomsbury at info@continuumbooks.com.

Page 2

  • Bloomsbury Discourse Series overview

    • Series Editor: Professor Ken Hyland, University of Hong Kong

    • Core idea: Discourse is central to humanities and social sciences because language mediates and shapes interactions with social, political, and cultural formations.

    • Aim of the series: Capture fast-developing interest in discourse; provide an essential bookshelf for students, teachers, and researchers in applied linguistics, ELT, and English language.

    • Each book covers a core topic in discourse studies to give readers an in-depth, structured introduction to how language operates in real life.

  • Other titles in the series (examples):

    • The Discourse of Online Consumer Reviews — Camilla Vásquez

    • News Discourse — Monika Bednarek and Helen Caple

    • The Discourse of Text Messaging — Caroline Tagg

    • The Discourse of Twitter and Social Media — Michele Zappavigna

    • Workplace Discourse — Almut Koester

    • The Discourse of Blogs and Wikis — Greg Myers

    • Professional Discourse — Britt-Louise Gunnarsson

    • Academic Discourse — Ken Hyland

    • School Discourse — Frances Christie

    • Historical Discourse — Caroline Coffin

    • Using Corpora in Discourse Analysis — Paul Baker

    • Metadiscourse — Ken Hyland

Page 3

  • Publication details (front matter, copyright, and cataloging information)

    • Bloomsbury Academic, an imprint of Bloomsbury Publishing Plc, locations: London & New York; addresses listed.

    • First published 2012; © Brian Paltridge, 2012. All rights reserved.

    • Legal notice: No part of the publication may be reproduced or transmitted without permission.

    • Paltridge asserted authorship rights under the Copyright, Designs and Patents Act, 1988.

    • British Library Cataloguing-in-Publication Data and Library of Congress data included; ISBNs provided for various formats (alk paper, pbk alk paper, eBook pdf/epub).

    • Typeset by Newgen Imaging Systems Pvt Ltd, Chennai, India; Printed and bound in India.

Page 4

  • Introduction to Multimodal Discourse Analysis

    • Multimodal discourse analysis studies texts that combine multiple semiotic resources beyond words (e.g., pictures, film, video, sound).

    • Example: 11 September events were communicated not only through verbal reports but also through televised images; images contributed to creating proximity and involvement with events (Machin 2007; Kress 2010).

    • Multimodality affects both print and television genres: e.g., Cosmopolitan magazine uses images alongside text to construct meaning about women.

    • Cosmopolitan: global brand with 44 editions worldwide; local editors even visit New York to align with a consistent focus and format across editions (Machin & Thornborrow 2003; Machin & van Leeuwen 2007).

    • The magazine sells not only products but values (independence, power, fun) through multimodal presentation: background, setting, color, lighting, and portrayal of women in relation to men and power.

    • Women in Cosmopolitan are depicted as fun and fearless with a take-charge attitude; power is linked to bodily presentation rather than professional competence (Machin & Thornborrow 2003).

    • Multimodal presentation includes clothing, lipstick, hair, places women shop and go to for lunch; all contribute to a reading of the magazine’s messages.

    • Central claim: Cosmopolitan aligns readers with the magazine’s brand values, expressed through multimodal means and cross-promoted products (e.g., Cosmo cafés, fashion, entertainment).

  • 8.1 Background to multimodal discourse analysis

    • Theory rooted in Halliday’s social semiotic approach: language as one semiotic resource among many (gesture, images, music) used to communicate meaning within sociocultural contexts.

    • Multimodal discourse analysis seeks to describe socially situated semiotic resources used for communication.

    • Texts draw on multiple modes (pictures, film, video, images, sound) in combination with words to make meaning; analysis examines how semiotic tools (color, framing, focus, element positioning) contribute to meaning.

    • Halliday (2009a) identifies three types of social meaning realized simultaneously in language use:

    • Ideational meaning: what the text is about

    • Interpersonal meaning: relations between participants

    • Textual meaning: how the message is organized

    • In multimodal texts, these meanings are realized visually (ideational/representational, interpersonal/modal, textual/compositional) in images and their arrangement (de Silva Joyce & Gaudin 2007).

    • Foundational references to: Kress & van Leeuwen (Reading Images, 2006); Kress (Multimodality, 2010); van Leeuwen (Introducing Social Semiotics, 2005); O’Halloran (Multimodal Discourse Analysis, 2004); Painter, Martin & Unsworth (Reading Visual Narratives, 2012).

    • Jewitt (2009a) outlines four key assumptions about multimodal analysis: (1) language is one of several equally meaningful modes; (2) other modes realize different meanings; (3) people configure modes to make meaning; (4) multimodal meanings are social and shaped by genre conventions.

Page 5

  • 8.2 Examples of multimodal discourse analysis (continued)

    • People typically use multiple modes simultaneously (images, gesture, gaze, posture) along with language (Machin 2007; Jewitt 2009a).

    • Each mode has affordances (Gibson 1977) within a context: what it can represent or express.

    • Meaning potential of a mode is shaped by its past uses, recurrent meanings, and social conventions (van Leeuwen 2005a; Jewitt 2009a).

    • Images can have their own grammar: mood (eye contact) and perspective (angle) influence interpretation:

    • Eye contact can indicate demand vs. offer; perspective (horizontal vs. high/low angle) suggests involvement or hierarchy.

    • Distance (close, medium, long), lighting, color, focus, and the degree to which an image reflects reality affect interpretation.

    • Time Magazine (1 June 2009) cover example: Michelle Obama image with text “The Meaning of Michelle.”

    • Analysis highlights eye contact, full-page central placement, lighting, focus, horizontal orientation, direct gaze, and salience as a way to connect with readers.

    • Page composition conveys information value and salience; the image dominates the page, framed within a vertical triptych layout; interplay with the caption and title creates intermodal complementarity (Painter & Martin, forthcoming).

    • Social-semiotic theory (Kress, 2010) asks: what meaning is being made, how it is made, what resources are used, in what social environment, and whose interest/agency is at work?

    • The Time cover’s foregrounded image carries the major informational load; its design communicates a message about Michelle Obama that aligns with the article’s content and broader political context (Obama era).

    • Online vs print: The online Time page features a different, less dramatic image of Michelle Obama, with similar communicative aims; readers navigate to the full article by scrolling or printing.

  • 8.3 Genre, speech acts and multimodality

    • Van Leeuwen (2005a, 2005b) connects speech acts and genre to multimodality, focusing on the ‘how’ rather than the ‘what’ of multimodal communication.

    • Illocutionary act: the intended action of the speech act (e.g., persuade, command, promise).

    • Perlocutionary effect: the effect on readers/listeners (e.g., convincing to buy, change attitude).

    • Texts rely on multiple modalities (linguistic and visual resources) to achieve perlocutionary effects.

    • Genres provide templates for doing things: service encounters have conventionalized stages (e.g., purchasing) that may combine language and actions; other genres (e.g., ATM use) combine verbal and visual instructions.

    • In some contexts (e.g., IKEA stores), customers perform tasks themselves with minimal or no interaction with staff, illustrating how genres organize multimodal interaction.

Page 6

  • 8.2 Examples of multimodal discourse analysis (continued)

    • Across genres, multimodal texts involve images, gaze, posture, etc., integrated with language to produce meaning.

    • The concept of affordances and how context shapes what a mode can convey.

    • The role of perspective (eye contact, shot distance) in signaling social relations (inclusion, distance, power).

Page 7

  • 8.2 Examples of multimodal discourse analysis (continued)

    • Further discussion of image grammar and viewer relations: mood, perspective, distance, lighting, color, focus, and reality versus stylization.

    • The Michelle Obama Time cover example is analyzed in terms of eye contact, front-on angle, extreme close-up, central placement, and salience to create a strong interpersonal connection with readers.

  • 8.2 Examples (case study): The Time cover and its layout elements are used to illustrate how image and text work together to convey meaning and persuade readers.

  • 8.3 Genre, speech acts and multimodality (continued)

    • Reiterates the link between illocutionary/perlocutionary aims and the multimodal resources used to achieve them.

    • Uses the service encounter as an example of how genre templates shape multimodal text design.

Page 8

  • 8.3 Genre, speech acts and multimodality (continued)

    • Additional points on how genres function as templates for performing social actions in culturally specific ways.

    • The IKEA example illustrates mixed verbal and visual guidance within a multimodal genre to facilitate self-service purchase.

  • 8.4 Carrying out multimodal discourse analysis

    • Analytical process mirrors other discourse analysis but with attention to multiple modes.

    • Data collection and logging steps: collect data, log (summarize), add contextual notes and first impressions for later depth.

    • When data is multimodal (print, web, video), analysts document how each mode contributes to meaning.

    • After initial reduction, select key extracts for in-depth analysis; then verify whether findings generalize within the dataset.

    • Baldry & Thibault (2005) offer detailed guidelines for transcribing and analyzing multimodal data across media (ads, cartoons, web pages, film/TV genres).

    • Key concept: clusters (groupings of items) and cluster analysis focus on connections between clusters rather than isolated items; relationships create meaning in context.

    • In film/TV, attention to shots, phases, transitions, and the relation between soundtrack and images helps situate meaning within context of situation and culture.

    • Ethnographic data and contextual data help link multimodal analysis to production/consumption contexts (Prior 2013; Street, Pahl & Rowsell 2009; Molle & Prior 2008).

  • 8.5 Limitations of multimodal discourse analysis

    • Time-intensive; interpretable and subjective nature can lead to varying readings.

    • Analyses often focus on final products rather than production processes or readers’ receptions.

    • Some studies under-emphasize language in favor of other modes; linking to broader social theories helps offset limitations (Machin & van Leeuwen 2007).

    • Ethnographic data can strengthen analyses by connecting multimodal findings to social contexts.

  • 8.6 Summary

    • Overview of multimodal discourse analysis: concept, theory, and empirical examples.

    • Relationships among genre, speech acts, and multimodality.

    • Introduction of a genre-and-multimodality framework and its layered analytical description.

    • Practical guidance for conducting multimodal analyses, along with limitations and directions for further research.

Page 9

  • 8.7 Discussion questions

    • (1) Front page of a newspaper: How does image use, color, font, and text size guide your response?

    • (2) Magazine cover: What strategies give the cover images salience?

    • (3) Memorable film scene: How do color, sound, costuming, and setting influence impact?

  • 8.8 Data analysis projects

    • (1) Collect magazine covers with photos of people; analyze using Kress (2010) questions: meanings, resources, social environment, and agency.

    • (2) Collect image-nuclear news stories from online newspapers; analyze text-image-caption relations and intertextual allusions (Caple, 2009).

    • (3) Collect web pages from a site you visit; analyze with Bateman’s Genre and Multimodality framework.

    • (4) Record a short TV program; analyze using Iedema’s framework for film/TV genres.

Page 10

  • 8.9 Exercises (continued) and 8.3–8.4 integration

    • Exercise 1: Image-nuclear news story analysis of a Mao Zedong hand sculpture image and its text-caption relationship; identify text components and their relation to the image.

    • Exercise 2: Book cover analysis: layout, horizontal vs vertical triptych, relationship between text and image, information hierarchy, font choices, and placement.

    • Exercise 3: Film trailer analysis (Bridget Jones’s Diary): analyze Orientation section using Maier’s framework; determine modes indicating Orientation.

  • 8.10 Directions for further reading

    • Bezemer, J. & Jewitt, C. (2010): Multimodal analysis key issues (methodology and data collection/analysis examples).

    • Jewitt, C. (ed.) (2009): Routledge Handbook of Multimodal Analysis; Chapter 1 provides background; Chapter 2 outlines approaches (social semiotic vs. systemic functional).

    • Additional recommended reads: Kress & van Leeuwen (Reading Images, 2006); Machin (Introduction to Multimodal Analysis, 2007); van Leeuwen (Introducing Social Semiotics, 2005).

    • The companion website provides extended references and resources.

Page 11

  • 8.4 Carrying out multimodal discourse analysis (online continuation)

    • Emphasis on integrating print and online media genres (image-nuclear stories, newsbites) and their interrelations.

    • Online content, home pages, and section pages provide navigational cues and social interactivity that influence interpretation.

    • Caple (2009) and Knox (2007, 2010) illustrate how online newspaper genres differ from print (image-nuclear stories vs. newsbites).

Page 12

  • 8.4 Carrying out multimodal discourse analysis (film/TV frameworks)

    • Iedema (2001) provides a hierarchical framework for analyzing film/TV genres: frame, shot, scene, sequence, generic stage, work as a whole.

    • Table 8.2 (summarized):

    • Frame: a salient still from a shot

    • Shot: unedited camera movement within a shot

    • Scene: a sequence of shots within a time-space

    • Sequence: camera moves across time-spaces with subtopics/characters

    • Generic stage: beginnings, middles, endings; genre-specific stages

    • Work as a whole: overall classification and relation to social/cultural orders

    • Obama 2008 victory speech example: single frame analysis on the Grant Park stage; orientation, social distance, horizontal perspective, and central framing; sequence of shots from crowd to podium; identification of expository vs. rhetorical genre features.

    • Contextual notes: contextual factors (time of day, audience, broadcast context) influence interpretation (Baldry & Thibault 2005; O’Halloran 2004/2011).

  • 8.5 Limitations (continued)

    • Further caveats about generalizing from multimodal data; complexity of context; need for triangulation with ethnographic data.

Page 13

  • 8.3–8.4 Case studies: film trailers and multimodality frameworks

    • Maier (2011) on film trailers: trailers contain implicit promotional stages followed by explicit promotional stages; voiceover, music, and editing contribute to the overall effect.

    • The structure typically includes diegetic information (story-related content) and non-diegetic information (promotional content such as voiceover, evaluations, and recommendations).

    • The modes of voice, music, images, and text work together to achieve the trailer’s purpose.

  • 8.4 Carrying out multimodal discourse analysis (closing notes)

    • The process includes repeated viewing (sound and vision together, then separately) to isolate each mode’s contribution.

    • For print or web data, analyze how each mode (words, font, layout, images) contributes to meaning and how these are contextualized in notes.

Page 14

  • 8.4 Carrying out multimodal discourse analysis (continued)

    • Table 8.3: Stages of comedy film trailers and their functions (Maier 2011, p.147)

    • Implicit promotional (diegetic information): Prologue, Appetiser, Orientation, Complication, Evaluation

    • Explicit promotional (non-diegetic and diegetic information): Promotional identification, Promotional recapitulation, Promotional interpretation, Promotional recommendation, Promotional information (e.g., website address, release dates)

    • The staged progression contributes to how audiences interpret and engage with trailers.

    • The overall aim is to show how multiple modalities (sound, vision, text) function together to promote the film and shape audience expectations.

Page 15

  • 8.4 Carrying out multimodal discourse analysis (data handling)

    • After data collection and logging, repeatedly view the data to understand contributions from each mode.

    • When analyzing any multimodal data, note how features such as color, layout, typography, and image composition influence interpretation.

    • Use contextual notes to interpret how the modes relate to production/consumption contexts.

    • Baldwin and Thibault (2005) emphasize clustering as a way to connect items; this helps to understand meaning in context rather than focusing on isolated pieces.

    • Ethnographic data, such as interviews and observations, can illuminate how texts are produced and received.

Page 16

  • 8.5 Limitations (summary)

    • Time, interpretation, and reader/viewer variation remain central concerns.

    • Multimodal analysis can underemphasize language; linking to social theories (global media discourse) helps address this.

    • Ethnographic approaches can provide deeper context for multimodal texts and their production/consumption.

  • 8.6 Summary (reiterated)

    • Multimodal discourse analysis investigates how texts use more than one mode to communicate meaning.

    • It includes theoretical backgrounds, exemplars, and a practical framework for analysis.

    • Connections among genre, speech acts, and multimodality are highlighted.

    • A genre-and-multimodality framework is proposed to describe layers of meaning in multimodal texts, along with practical steps and limitations.

Page 17

  • 8.7 Discussion questions (reiterated)

    • (1) News front page composition: image, color, font, and size affect interpretation.

    • (2) Magazine cover salience: image strategies to draw attention.

    • (3) Memorable film scene: color, sound, costume, setting influence impact.

  • 8.8 Data analysis projects (reiterated)

    • (1) Magazine covers: meanings, resources, social environment, and agency using Kress (2010) questions.

    • (2) Image-nuclear news stories: analysis of headings, images, captions; intertextual allusions (Caple 2009).

    • (3) Web pages: apply Bateman’s framework.

    • (4) Short TV program: apply Iedema’s framework.

Page 18

  • 8.9 Exercises (continued) and references to figures

    • Exercise 1: Image-nuclear story (Mao Zedong hand sculpture) with text and its relation.

    • Exercise 2: Book cover analysis (old edition figure 8.2): layout type, words-image relationship, information value, and fonts.

    • Exercise 3: Bridget Jones’s Diary trailer: analyze Orientation section using Maier’s framework; identify modes used for Orientation.

  • 8.10 Directions for further reading (detailed references)

    • Bezemer, J. & Jewitt, C. (2010): Multimodal analysis: key issues (in Research Methods in Linguistics, L. Litosseliti). A practical and accessible overview with classroom interaction example.

    • Jewitt, C. (ed.) (2009): The Routledge Handbook of Multimodal Analysis. Chapter 1: background; Chapter 2: approaches (social semiotic vs. systemic functional).

    • Other core texts for extended readings: Kress & van Leeuwen (Reading Images, 2006); Machin (Introduction to Multimodal Analysis, 2007); van Leeuwen (Introducing Social Semiotics, 2005).

    • The companion website provides an extended list of references and readings.

Page 19

  • Figure references

    • Figure 8.1: Image-nuclear news story example (Caple’s discussion referenced).

    • Figure 8.2: Cover of the first edition of Discourse Analysis (Paltridge, 2006).

Page 20

  • Final bibliographic notes and additional references

    • Summary of major texts used throughout the chapter for multimodal discourse analysis and related frameworks.

    • Emphasis on the integration of textual, visual, and contextual analysis to understand how meaning is produced and interpreted in modern media.