ENGLISH LINGUISTICS II

Flashcards about Multimodal Text and Discourse

Multimodality (Discourse Studies)

The combination of words with other modalities such as pictures, film, video, images, and sound.

Linguistic Multimodality

Textual combination of different modes and their integration in terms of structure, discourse semantics, and rhetorical function within contexts of social interaction.

Multimodal Discourse Analysis

how texts draw on modes of communication such as pictures, video, and sound in combination with words to make meaning.

Questions of MDA

What meaning is being made in a text?

How is meaning being made?

What resources have been drawn on to make the meaning in the text?

In what social environment is the meaning being made?

Whose interest and agency is at work in the making of the meaning?

Modes and Multimodal Texts

• printed-text

• audio-text

• audiovisual text

• electronic text

printed-text(Modes and Multimodal Texts)

• language (writing)

• picture (still)

• typography

audio-text(Modes and Multimodal Texts)

• language(speech)

• music

• noise

audiovisual text (Modes and Multimodal Texts)

• language(speech/writing)

• picture(static/moving)

• music

• noise

• typography(static/dynamic)

Electronic text (Modes and Multimodal Texts)

• language,typography

• picture

• music

• noise

• (navagation,Interaction)

Mode Equality

Modes form part of a communicative ensemble(ансамбль); each mode has equal potential to contribute to the meaning.

Different Modal Reaches/Potentials

Each mode comes equipped with semiotic/pragmatic strengths and weaknesses.

Strategic Mode Combination

In multimodal text production, designers select various modes with an eye to their reach/potential.

Modal Reach

Distinct potentials and limitations of a mode.

semantics (LANGUAGE versus IMAGE)

LANGUAGE: clear reference/predica2on + SA IMAGE: foreground/background + salience

semiotics (LANGUAGE versus IMAGE)

LANGUAGE: linear, abstract signs + grammar IMAGE: concrete signs + spa2al grammar

cognition (LANGUAGE versus IMAGE)

LANGUAGE: slow, removed from percep2on IMAGE: fast, high impact on memory

function (LANGUAGE versus IMAGE)

LANGUAGE: narrate/explain … IMAGE: show/present physical detail

Multimodal Interpretation of Genre

Any multimodal artifact realizing a genre's underlying functional, logical, structural, and stylistic regime.

Uncovering multimodal genre

Describing how textual structures and their stages are orchestrated by various modes and their combinations.

TWO PRINCIPLED WAYS OF REALIZING STAGES

① 1 STAGE = 1 MODE (e.g. print)

IMAGE: show news actor/place LANGUAGE: tell news event

② 1 STAGE = 2+ MODES (e.g. film)

LANGUAGE: tell news event IMAGE: tell news event

The GeM model

Defines the layers of description for multimodal documents.

the GeM-Model structures LNLCRG

① LAYOUT STRUCTURE

② NAVIGATION STRUCTURE
③ LINGUISTIC STRUCTURE

④ CONTENT STRUCTURE
⑤ RHETORICAL STRUCTURE

⑤ GENRE STRUCTURE

Layout Structure (GeM-Model)

Positioning/Composition: top-down structure, paragraphing, framing/boxes

Navigation Structure (GeM-Model)

Interactive Elements: magnifying tool, hyperlinks/tags, like/share-buttons, scroll-function

Linguistic Structure (GeM-Model)

Verbal Elements: name image elements, explain material/theme, interpret context/impact

Content Structure (GeM-Model)

Topics: painter, art-period, content + making + interpretation of painting

Rhetorical Structure (GeM-Model)

Relational Propositions: ELABORATION + BACKGROUND + CIRCUMSTANCE (nucleus = image)

Genre Structure (GeM-Model)

Stages: Painting – Annotation + Navigation – Commentary – Description – Tags + Hyperlinks

Core idea of multimodality

Interplay between modes; intersemiotic relations.

Mode connectedness 5

1. Grammatical cohesion

2. lexical cohesion

3. coherence- relational propositions

4. coherence - frames/scripts

5. coherence -rhetorical figures

Grammatical Cohesion(Mode connectedness)

is the use of grammatical elements to connect sentences and clauses within a text, helping to maintain coherence and clarity in writing. This includes devices like pronouns, conjunctions, and other referencing tools that link ideas logically.

Lexical Cohesion(Mode connectedness)

The way words in a text relate to each other through their meanings, thereby enhancing the unity and coherence of the text.

Relational Propositions(Mode connectedness)

elaboration+ circumstances+ background

Frames & Scripts(Mode connectedness)

help individuals interpret and organize information based on prior knowledge and experiences. They provide a framework for understanding specific situations, guiding how language is processed and meaning is constructed.

Rhetorical Figures(Mode connectedness)

Rhetorical figures related to the organization and connection of ideas in discourse, enhancing the persuasive impact of a text.

Typical Genre Features of INNSs

① large, aesthetically pleasing image

② heading + caption intro – both directly relating to image

③ verbal-visual play between image, headline, caption

④ extended caption – describing image + providing context

⑤ evaluative stance – expressing editorial/ideological angles

EXPERIENTIAL ORIENTATION (Rhetorical Structure of INNS)

Refers to the perspective or viewpoint through which experiences are articulated in discourse, often influencing the way information is presented and understood in rhetorical contexts.

CONTEXTUAL EXTENSION(Rhetorical Structure of INNS)

A technique in rhetorical analysis that involves extending the meaning of a text based on its context, particularly how structures influence interpretation.