Multimodal Linguistics I

What is a sign at the most basic level?

A sign is something that stands for something else. It has the capacity to represent or communicate something beyond itself. This happens either through a causal link (natural signs) or through an arbitrary, conventional agreement (symbolic signs).

Explain the three components of a sign in semiotics.

Signifier: The form of the sign

Signified: The meaning or concept it represents

Interpretant: The person who perceives and interprets the relationship between signifier and signified

What is the difference between natural (causal) and conventional (symbolic) signs?

Natural Signs: Have a direct, physical cause-and-effect relationship with what they signify (e.g., smoke is a natural sign of fire)

Conventional Signs: Get their meaning from social agreement and convention, not from a natural link (e.g., the word "cat" for the animal)

What three important features characterize sign relationships?

Causality: A perceived causal relationship (e.g., fire causes smoke)

Temporality: A temporal relationship (e.g., fire and then smoke)

Intentionality: Comes from the interpretant; natural signs have no intent, humans create the meaning

What is semiotics? Provide a comprehensive definition.

Semiotics (also known as semiology) is the scientific study of signs, their organization into sign systems (e.g., language), and the ways individuals interpret and use these sign systems. It is the study of how meaning is created and communicated through signs.

What is the relationship between linguistics and semiotics?

Although semiotics is often treated as a subfield of linguistics, linguistics is actually a subfield of semiotics. Both deal with meaning, but from different perspectives and with different levels of granularity.

Why is the language model only partially applicable to other semiotic modes (e.g., pictures)?

Signs: Pictures use iconic, non-arbitrary, motivated signs, but it's unclear if there's a pictorial "system"

Structure: Syntax is misleading since pictures are non-linear

Depiction: Pictures have strong referential power but struggle with abstract concepts

Behavior & Medium: Conventions are less clear than in language

What makes humans particularly good at working with signs?

Humans are very good at seeing patterns and making meaning. They can create conventional signs whenever they need them, and these can be elaborated, expanded, and changed over time and across different cultures and contexts.

What is Social Semiotics and how does it differ from "pure" theory?

Social Semiotics is not "pure" theory and not a self-contained field. It only comes into its own when applied to specific instances and specific problems. It is a form of enquiry that does not offer ready-made answers but offers ideas for formulating questions and ways of searching for answers.

Name the four main characteristics of Social Semiotics.

Dynamic, interactional view (not static)

Practices in context (not fixed rules)

Critical stance (observation → reflection → critique)

Method more than fixed theory

What do we understand by multimodality in communication?

Multimodality refers to the fact that language in actual use is often combined with other symbolic systems of expression (e.g., pictures, music, noise). Our daily interactions integrate spoken language with complex sign systems such as gestures, body movements, or tone of voice.

What purpose do sign systems serve in society?

Sign systems help us function as a society. Once we have sign systems, we can use them to build structures or texts that help us transmit knowledge and information across time and space.

What functions do natural signs fulfill for humans?

Natural signs help us avoid danger, understand our environment, and anticipate. They don't intend to do this - we build this relationship ourselves. In general, signs direct attention to what they represent, helping people understand the world, predict outcomes, and communicate ideas.

How can signs be organized into complex artifacts?

Natural Signs: Can be turned into more conventional signs, expanded, and added to other signs to make more conventional sign systems

Conventional Signs: Can be elaborated and expanded as needed and changed over time and across cultures and contexts

How interdisciplinary is semiotics and what is its origin?

Semiotics is a very interdisciplinary field, although it has grown out of linguistics and is still predominantly influenced and shaped by linguistics (for better or worse).

Explain the glasses metaphor in relation to semiotics and linguistics.

Take a pair of glasses - it's the same object, but as we turn and rotate it and look at it from different angles, it will look vastly different. It is the same with meaning. Both semiotics and linguistics look at meaning, but they are doing so from different perspectives and different levels of granularity.

What is the main difference between structural and general semiotics?

Structural semiotics is an analytical method focusing on internal relationships and rules within sign systems

General semiotics is the broader field studying all signs and sign systems, including how signs are used in different contexts.

Who was Ferdinand de Saussure and why is he important?

Saussure (1857-1913) was a Swiss linguist considered one of the founders of modern linguistics and semiotics. His ideas laid the foundation for structuralism in the 20th century.

What are the two components of Saussure's sign model?

The signifier (the form of the sign - the physical/material aspect) and the signified (the meaning or concept it represents).

These two components are inseparable and together constitute the sign

What are Saussure's four key dichotomies?

1) Langage vs. Langue vs. Parole (ability vs. system vs. use)

2) Signifier vs. Signified (form vs. meaning)

3) Syntagmatic vs. Paradigmatic (horizontal combination vs. vertical association)

4) Synchronic vs. Diachronic (state at one point vs. historical development)

What does the syntagmatic dimension refer to?

The syntagmatic dimension refers to the horizontal combination of signs - how signs are combined in sequence to create meaning (like words in a sentence or elements in a visual composition)

What does the paradigmatic dimension refer to?

The paradigmatic dimension refers to the vertical association of signs - the set of alternative signs that could be substituted at any given position

Who was Charles Sanders Peirce?

Peirce (1839-1914) was an American scientist and philosopher known as "the father of pragmatism." He is considered the founder of modern semiotics and developed the triadic sign model.

What are the three components of Peirce's triadic sign model?

1) Sign form/sign vehicle (the material form)

2) Reference object (the object in external reality that is signified)

3) Interpretant (both the sign user and the sign content in the user's mind)

What does the triadic model add compared to Saussure's dyadic model?

The triadic model shows signs as dynamic and changeable rather than static. It reveals that signs are made and used by humans, and the relationship between form and content is culturally regulated and changeable.

How does Peirce's model demonstrate the dynamic nature of signs?

Signs can be reinterpreted in appropriate contexts (example: coffee bean → bean shape → eye shape → closed eyes → sleep). The association of form and content is created by the sign user in context.

What is an icon?

An icon is a sign that has a relationship of similarity or resemblance to what it represents (e.g., a photograph, realistic drawing, or pictogram that looks like what it depicts)

What is an index?

An index is a sign that has a causal or physical connection to what it represents (e.g., smoke indicating fire, footprints, an arrow pointing direction)

What is a symbol?

A symbol is a sign based on arbitrary, conventional agreement without logical references or similarity to the designated object (e.g., words, flags, traffic light colors)

Can a single sign contain all three types (icon, index, symbol)?

Yes, most signs involve all three modes. For example, a danger pictogram may be iconic (picture resembles danger), indexical (arrow shows direction), and symbolic (colors, shapes have conventional meanings)

On what four levels can differences between icon, index, and symbol be established?

1) Relationship between sign and referent (similarity, causality, arbitrariness)

2) Connection between signifier/signified (motivated vs. non-motivated)

3) Nature of relationship (conventional vs. intuitively clear)

4) Effort in decoding (learning vs. intuitive)

Who was Karl Bühler and what did he contribute?

Bühler (1879-1963) was a German psychologist and linguist who brought a cognitive approach to semiotics. He developed the "organon model" of communication, focusing on how signs function for users.

What are the three functions of signs in Bühler's organon model?

1) Representational/Symbol function (conveying facts about reality)

2) Appeal/Signal function (influencing receiver to act)

3) Expressive/Symptom function (revealing sender's inner state)

How does Bühler's model differ from previous sign models?

Unlike purely structural models, Bühler's model functions as both a sign model and a communication model. It shows how signs connect sender and receiver to convey messages.

Do the three sign functions occur in isolation?

No, these functions usually appear together with one function dominating in any given communicative situation.

Give examples of the representational function in graphic signs.

Coats of arms, monograms, musicographic signs, meteorological signs, cartographic signs, company logos

Give examples of the appeal function in graphic signs.

Public signs, traffic signs, washing instruction symbols that give commands (do not wash, do not dry, do not iron)

Give examples of the expressive function in graphic signs.

Typographic signs, watermarks, handwriting that expresses the writer's age or emotional state - graphic signs are particularly suited for expressive function due to variability in execution

What are the four types of concepts in relation to sign vehicles?

1) Pictography (symbols represent visible objects)

2) Ideography (symbols represent complex concepts, independent of language)

3) Logography (symbols represent a word)

4) Phonography (symbols represent a sound)

What are Legi, Sin, and Quali signs?

Different levels of sign reading.

Legi signs have conventionalized meanings.

Sin signs allow for individual modifications (motif, perspective). Quali signs involve changes in materiality (color, frames).

Signs can be read on multiple levels.

What is an example of synaesthesia in signs?

Braille writing possesses both visual and haptic-tactile qualities. These qualities allow written characters to acquire object status and function as representations, engaging multiple senses.

What does it mean that language is inherently multimodal?

Language in actual use is combined with other symbolic systems (pictures, music, gestures, body movements, tone of voice) - our interactions integrate multiple sign systems bc everything makes meaning

What is the relationship between linguistic and pictorial signs?

The boundary is fluid. Marks can be read as letters or pictorial signs. Graphic signs are either arbitrary (writing) or iconically motivated (images), and individual symbols can be combined to form complex signs.

What should you remember about the relationship between linguistics and semiotics?

Linguistics influenced semiotics significantly, but semiotics recognizes that language is one of many sign systems. Language is neither the sole nor dominant sign system

Why is it important to study semiotics?

Semiotics provides tools for analyzing how meaning is created across all human activities and cultural practices, revealing underlying structures of communication beyond just language

What is the current state of semiotics as a discipline?

Semiotics is still developing - described as "more a project than an established science." It lacks comprehensive theoretical foundation but is sustained by a shared point of view about studying sign systems.

What are the three main theoretical approaches to signs?

1) Structural semiotics (Saussure) - internal relationships within sign systems

2) General semiotics (Peirce) - all sign systems and their use in context

3) Communicative semiotics (Bühler) - sign functions in communication

What is the famous quote by Watzlawick about communication?

"One cannot not communicate" - even silence and 'non-behaviour' have the character of a message

What are the four main aspects that communication involves according to general consensus?

1) Information processing

2) Coded flows of information

3) Exchange of signs

4) Meaning-making (semantics)

What is the difference between unilateral semiosis and bilateral communication?

Unilateral semiosis is the reception and processing of signals by a living being (a pre-form of communication)

Bilateral communication is bilateral interaction between at least two organisms involving procedures by which one mind may affect another

Communication is congruent and intentional, what does that mean?

Congruence: compatibility between the senders message and the receivers interpretation

Intention: communication as goal-directed activity

What are the key ingredients/components necessary for communication? (6)

Signs/Messages

Sign Repertoire (shared)

Perceiving/Thinking Individuals

Contact between Individuals (Medium)

Interaction-Intentionality-Congruence

Meta-communicative Ability

What does Saussure's model of the 'speech circuit' (1916) offer?

It offers communication as a circular process, an act of mental de- and encoding and an interplay of hearing and speaking

What does Shannon & Weaver's communication model (1949) offer?

It offers communication as a linear process (chain) with a telegraphic model, focuses on sending/transporting information, no interaction factored in and represents a technical idea of communication

What are the six basic elements of Jakobson's communication model?

1) Sender/producer of signs

2) Recipient/addressee

3) Content (referent)

4) Message

5) Contact/medium

6) Code (shared sign repertoire)

What are the six communicative functions of signs according to Jakobson?

Referential (information about world)

Expressive (internal mental state)

Conative (change addressee's behavior)

Phatic (maintain contact)

Metalingual (comment on communication itself)

Poetic (emphasize forms and structures)

What are the the six sign functions of Jakobson's communication model?

EXPRESSIVE: characterize the producer e.g. handwriting

CONTATIVE: address the recipient e.g. care instruction icons

PHATIC: establish and maintain the channel e.g. signs in public spaces

META-COMMUNICATIVE: signs reflect the rules of the code e.g. coat of arms

POETIC: create recurring and salient strucutres in messages e.g. advertising

REFERENTIAL: convey content about a real world e.g. cartographic signs

What is Significs and who founded it?

Significs is a philosophical and linguistic movement focusing on the "science of meaning" and involving critical analysis of language to improve communication

Founded by Lady Victoria Welby in the 1890s

What are the core principles of Significs? (4)

Study of meaning (sign, sense, meaning, significance)

Language reform to reduce ambiguity

Promotion of critical thinking

A holistic approach encompassing all forms of communication

What is the modern revelance of Significs?

Relevant to the study of fake news and digital misinformation

What is Social Semiotics?

The study of how meaning is created and interpreted through various social and cultural practices, viewing texts as multimodal and exploring the social dimensions of meaning-making

What are the key aspects of Social Semiotics? (6)

Meaning is social (not inherent)

Multimodality (multiple resources combined)

Functions as a social theory

Emphasizes agency

Investigates connection to power

Uses analytical tools like commutation

What is Cognitive Semiotics?

An interdisciplinary field that studies how meaning is made by the mind, combining theories and methods from cognitive science and semiotics to investigate the relationship between mind, meaning, and signs

What are the key aspects of cognitive semiotics? (6)

Integration of fields

Focus on meaning-making

Multidisciplinary approach

Methodological diversity (mix of methods)

Understand human signification

Bridging disciplines (humanities and science)

What is a code in semiotics and what does it include (3)?

A sign system used to communicate

Includes a set of signs, the meaning relations among them and the rules for combining them to create and interpret messages

What are the three categories to differ between the various codes?

Digital vs. Analogue (precise distinctions vs. continous variations to convey meaning)

Simple vs. Complex (Few basic elements vs. countless elements)

Conventional vs. Natural (human-made vs. nature-made)

What are the three main categories of codes identified by semioticians?

Social codes (language, bodily codes, commodity codes, behavioral codes), Textual codes (scientific, aesthetic, genre, mass-media codes) and Interpretive codes (perceptual, ideological)

What is Semiosis?

The process of sign interpretation - signs trigger a continuous process of interpretation and meaning-making in perception. It involves the dynamic relationship between sign form, idea, and object

Which two respects is Semiosis relevant to?

Historically, the emergence of signs and Current, for current sign practice

When is Semiosis often used?

Advertising

What are secondary signs and connotation?

Secondary signs occur when a sign meaning becomes a sign again with new meaning

Connotation refers to how a sign opens up a "courtyard" of meanings (associations) beyond its literal meaning

What are the three levels of sign interpretation mentioned?

1) Literal (Saussure's connection between form and content)

2) Connotative (Morris's secondary sign meanings)

3) Mythical (Barthes' idea that all sign meanings constitute a myth/ideology)

What is multimodality and what does it involve?

The phenomenon where different types of signs occur in multiple complex combinations in texts/artefacts

It involves how different semiotic modes (language, image, music, sound) combine and recombine in context

What are the key semiotic properties of graphic signs? (5)

They have strong conventionalization of meaning

Use metonymies for quick access to concepts

Combine simple images (pictograms) and indices (arrows)

Have highly restricted syntax

Are extremely location- and context-dependent