LECTURE 3: Language and the Brain - Detailed Notes

Language and the Brain

Overview

• What makes language special in neuroscience?

  • Language as an object of investigation: levels of analysis and interpretation

  • Linguistics vs. cognitive neuroscience

• Language processing: brain structures and functions

  • Can language be localized? Is the brain modular?

  • Is function determined by location or the other way around?

• An example from the cognitive neuroscience of language

Language as a Human-Specific Function

• The language system is unique in the universe.

• Its evolutionary origin and acquisition in childhood are among the greatest scientific riddles.

Language as an Anatomically Identified Brain Function

• One of the first anatomically identified, non-sensorimotor brain functions

• Neural specialization and specificity: Which area(s) subserve language?

• Plasticity and modularity: Are there systems dedicated to specific functions in general, and can these be localized?

Language as an Abstract Brain Function

• One of the most abstract (non-sensorimotor) brain functions.

• Can language be linked to or derived from perception, speech sounds, or perceptual systems?

• Are language and thought identical? (Saphir-Whorf hypothesis)

The Uniqueness of Human Language

• von Humboldt (1836): Language is infinite. Infinite employment of finite means (freedom of thought).

• de Saussure (1916): Language is arbitrary. The relationship between the signifier and signified is undetermined:

  • "table" – "asztal" (it is not possible to map it on the world with no teaching/learning)

• Chomsky (1957): Language is extremely complex computationally

  • Its combinatorial explosion makes it impossible to learn it through examples.

  • Speech is not sequential but a hierarchical structure.

  • Language is grammar.

Levels of Analysis of a Sentence

A sentence can be analyzed at multiple levels:

• (a) Phonological structure: Includes prosodic structure, syllabic structure, and segmental structure.

• (b) Syntactic structure: Describes the grammatical relationships between words.

• (c) Semantic/conceptual structure: Represents the meaning of the sentence.

• (d) Spatial structure: Concerns spatial relations described in the sentence.

Language as a Matter of Linguistic Analysis

• Language in a narrow and broad sense:

  • Animals do not talk.

  • Films have no language.

  • Computer languages are not language-like.

• Levels of analysis:

  • Phonology: Speech sounds and their rules.

  • Morphology: Word forms, inflection (morphosyntax: compounding).

  • Syntax: Grammar (both as rules for language and thought).

  • Semantics: Linguistic meaning (words and sentences).

  • Pragmatics: Language in use, social attribution of intended meaning.

Language as a Matter of Neuroscience

• Neurology and neuropsychology:

  • Brain injury: case studies, rehabilitation.

  • Localization-based reverse engineering.

• Cognition: Language as a model of information processing.

  • Computational, algorithmic, implementation levels (Marr, 1982).

  • Language system - software - hardware

• Cognitive neuroscience: Human language in an ape brain

  • Matching neural and language functions: language processing as neural activity

  • Exploring the brain via the workings of language

Outstanding Questions

• Serial or parallel processing?

• Modularity? Neural organization?

• Top-down, bottom-up, interaction?

• Separation of (Grammatical) rules and units (words)?

• Regular vs. irregular inflection: organization of the mental lexicon?

• Sentences as meaningful constructions vs. structures requiring syntactic analysis (surface vs. deep structure)?

• Language =

  • Syntax (grammar): Formal Grammar: Chomsky, Bever, Pinker, Bloom

  • Semantics (meaning): Constructivists: Lakoff, Turner, Fauconnier, Talmy

  • Pragmatics (usage): Grice, Sperber, Tomasello

Broca's Patient

• Broca's (1861) patient: Tan tan tan tan

Aphasia Models

• Wernicke, 1874

• Geschwind, 1972

• Includes Broca's area, Wernicke's area, arcuate fasciculus, angular gyrus.

Aphasia Typology (Desai & Riccardi, 2021)

A table to classify aphasia:

• Columns: Fluent, Comprehends, Repeats

• Types of Aphasia:

  • Global

  • Mixed transcortical

  • Broca's

  • Transcortical motor

  • Wernicke's

  • Transcortical sensory

  • Conduction

  • Anomic

Anatomy of Language: A Review of 100 fMRI Studies (Price, 2010)

• Different brain regions are activated by different language tasks.

• fMRI studies show various areas associated with prelexical processing, word processing, semantic/syntactic ambiguity, sentence comprehension, word retrieval and articulation.

• Components of Language:

  • Execution

  • Selection

  • Volition

  • Action

  • Hierarchical sequencing

  • Articulatory planning

  • Auditory-motor

  • Audio-visual sentences

  • Amodal semantic combinations

Towards a New Neurobiology of Language (Poeppel et al., 2012)

• Includes Broca's area, Wernicke's area, primary auditory area.

• Articulatory network

• Dorsal stream

• Combinatorial network

• Ventral stream

• Sensorimotor interface

• Phonological network

• Conceptual network

• Lexical interface

Language Processing (Turker et al., 2023)

• Includes Main Effect of Language, Overlap across Subdomains.

• Subdomain Processing (Syntax, Semantics).

• Domain Overlap (Phonology, Prosody).

Language Tracts and Gender Differences (Catani & Mesulam, 2008)

• Fronto-parietal fibers

• Anterior segment

• Callosal Fibres

• Geschwind's territory

• Arcuate

• Broca's territory

• Long segment

• Wernicke's territory

• Posterior segment

• Uncinate

• Includes Group1 extreme left lateralization (-60%)

• Includes Group2 mild left lateralization (-20%)

• Includes Group3 bilateral, symmetrical (-20%)

Bilingualism (Kim et al., 1997)

• Late Bilinguals

• Early Bilinguals

Structural Correlates of Regular Inflection (Marslen-Wilson & Tyler, 2007)

• Regular inflection – Broca’s area, BUT – a lower statistical threshold includes the arcuate fasciculus and Wernicke’s area?

• Irregular inflection – grammar, but the rule does not help

  • meaning is specified but modified

  • non-language areas?

Broca is not Homogenous

• Topological Approach

• Hodological Approach

Functions of Broca

• Unification of phonology syntax and semantics (Hagoort, 2005)

  • BA 47 & BA 45: semantics

  • BA 45 & BA 44: syntax

  • BA 44 & BA 6: phonology

• Sequential processing during reading (Sahin et al., 2009)

  • 200 ms: lexical

  • 320 ms: grammatical

  • 400 ms: phonological

• Morphological processing

  • complexity, inflection (e.g., Bozic et al., 2007; Marslen-Wilson & Tyler, 2007)

  • Morpho-syntactic compounding (e.g., Koester & Schiller, 2011)

• Working Memory (e.g., Owen et al., 2005)

Broca’s Meta-Analysis: 485 Neuroimaging Studies (1991-2007) (Liakakis et al., 2011)

• Cluster 1: Semantic & phonological tasks (BA 45/46)

  • verbal fluency

  • semantic retrieval

  • word repetition, synonym production

  • lexical decision

  • inner speech

  • semantic categorization

• Cluster 2: Working memory (BA 44)

  • n-back

  • associative learning

  • recall of topographic routes

  • verbal memory

  • arithmetic operations

  • forward & backward digit recall

• Cluster 3: Empathy (BA 47)

  • in/congruent motor expressions of emotions

  • laughing & seeing others laugh

  • feeling guilt

  • evaluating emotional face expressions

Alternative Hypotheses for Broca (Fink et al., 2005)

• Broca’s area is a mosaic of many different functional regions, only some of which are directly related to language

• Broca’s area computes a highly abstract function, such as selection from a pool of competing alternative representations, which subsumes aspects of language processing and many other cognitive systems, including local visual search

• Particular neuroanatomical regions, including Broca’s area, change their functions consequent upon the simultaneous activation of other regions that are effectively connected to a given region

• Modules could be functional and/or anatomical, but no 1:1 mapping between anatomy and function

Functional EEG Markers of Language

• (E)LAN: Syntactic violations

  • Grammatical agreement or word category expectations (Hagoort, 2005)

  • “The woman sees the man because with the binoculars”

• N400: Semantic processing

  • Predictions of semantic memory systems (Kutas & Federmeier, 2011)

• P600: Syntactic integration

  • Violations, ambiguities, complexity

  • Garden path sentences

  • Constructing sentential structure

  • Word category violations evoke all effects ELAN N400 P600

Semantic Memory Retrieval: N400 (Kutas & Hillyard, 1980; Kutas & Federmeier, 2011)

• Graphs used to demonstrate semantic congruity, semantic category, word repetition, word frequency, word position

N400-like MEG (Halgren et al. 2002)

• Terminal Word

• Incongruous

• Congruous

• subject 1

• 300-500ms

Both Sides Get the Point (Federmeier et al., 2005)

• Farmers grow…..

  • crops

  • worms
    *Difference worms-crops

Lateralization of Language (Van Lancker Sidtis, 2006)

• LEFT HEMISPHERE

  • Linguistic

    • Phonetics

      • Segmental

      • Prosodic (timing; tones)

    • Phonology

    • Morphology

    • Syntax

    • Semantics

• RIGHT HEMISPHERE

  • Contextual

    • Familiar proper nouns

    • Emotional terms

    • Pragmatics

    • Conversation

    • Topic, theme

    • Inference

    • Formulaic language

    • Prosodic (pitch)

    • ortholinguistics

    • paralinguistics

    • emotional meanings

    • attitudinal meanings

    • intonation in phrases

    • voice recognition

Figurative Language: Right Hemisphere Processing?

• Language disruption of right hemisphere damaged patients

• Social language: ambiguities, irony, jokes, indirect requests, etc.

  • Bottini et al. 1994 Winner & Gardner, 1977

What is a Metaphor?

• “Odysseus is a lion…”

• Who is the lion?

• Why do we say things figuratively instead of always speaking literally?

• How our brain is capable of differentiating and correctly interpreting?

The Neural Processing of Metaphor

• Right hemisphere language

  • Language specialization beyond the left hemisphere?

  • Contradictory results (replication problems)

• Serial processing: identifying a violation?

  • Literal meaning needs to be rejected

  • We process language based on literal “truth-value”

• Are metaphors captivating because they are based on bodily experience?

  • Is language comprehension based on sensorimotor processing

  • Are there no language centers? (Gallese & Lakoff, 2005)

Novel Metaphors

• Idiomatic, conventional metaphors have a lexical entry

• Novel metaphors (“brilliant elegance”): semantic combinatorics

  • Right hemisphere?

  • Slower serial processing?

  • Embodied sensorimotor simulations?

Coarse Semantic Coding Theory (Beeman, 1998; Jung-Beeman, 2005)

• Shows processing in both hemispheres with diagrams.

Disentangling Familiarity & Figurativeness Forgács, B., Lukács, Á., & Pléh, Cs. (2014)

• Is it metaphoricity or novelty that drives right hemisphere effects?

• Serial processing of figurative meaning or just slower processing of unusual expressions?

• Divided visual field paradigm + eye-tracking

Forgács, B., Bohrn, I., Baudewig, J., Hofmann, M. J., Pléh, Cs., & Jacobs, A. M. (2012).

• Event-related fMRI study

Conventionals > Novels

• Left Hemisphere

• Right Hemisphere

t(39)

p<0.000010

Metaphorical Language

• RH processing: no evidence

  • No specialized neural systems either for metaphors: even novel metaphors are processed by in the LH

  • No RH processing of novel language

• Serial processing: no evidence

  • Novel metaphors are processed just as quickly as literal expressions matched in novelty

  • There still might be metaphor-specific computations (fMRI and EEG data)

• Embodiment?: no evidence

  • Literal language evokes sensorimotor feature processing, but metaphors do not

  • The early automatic activation of literal meaning (and corresponding sensorimotor areas) are not contested

  • When concrete words are used in figurative sense, they evoke an abstractness effect

  • What is left of embodiment as a model of cognition if it cannot account for abstract thought?

Summary

• From Broca and Wernicke to neural networks

• Language processing as a neural function

• From syntax to meaning

• Inflection

• Sentence construction

• From the left hemisphere to the right

• From literal language to figurative meaning