LECTURE 3: Language and the Brain - Detailed Notes

Language and the Brain

Overview

  • What makes language special in neuroscience?

    • Language as an object of investigation: levels of analysis and interpretation

    • Linguistics vs. cognitive neuroscience

  • Language processing: brain structures and functions

    • Can language be localized? Is the brain modular?

    • Is function determined by location or the other way around?

  • An example from the cognitive neuroscience of language

Language as a Human-Specific Function

  • The language system is unique in the universe.

  • Its evolutionary origin and acquisition in childhood are among the greatest scientific riddles.

Language as an Anatomically Identified Brain Function

  • One of the first anatomically identified, non-sensorimotor brain functions

  • Neural specialization and specificity: Which area(s) subserve language?

  • Plasticity and modularity: Are there systems dedicated to specific functions in general, and can these be localized?

Language as an Abstract Brain Function

  • One of the most abstract (non-sensorimotor) brain functions.

  • Can language be linked to or derived from perception, speech sounds, or perceptual systems?

  • Are language and thought identical? (Saphir-Whorf hypothesis)

The Uniqueness of Human Language

  • von Humboldt (1836): Language is infinite. Infinite employment of finite means (freedom of thought).

  • de Saussure (1916): Language is arbitrary. The relationship between the signifier and signified is undetermined:

    • "table" – "asztal" (it is not possible to map it on the world with no teaching/learning)

  • Chomsky (1957): Language is extremely complex computationally

    • Its combinatorial explosion makes it impossible to learn it through examples.

    • Speech is not sequential but a hierarchical structure.

    • Language is grammar.

Levels of Analysis of a Sentence

A sentence can be analyzed at multiple levels:

  • (a) Phonological structure: Includes prosodic structure, syllabic structure, and segmental structure.

  • (b) Syntactic structure: Describes the grammatical relationships between words.

  • (c) Semantic/conceptual structure: Represents the meaning of the sentence.

  • (d) Spatial structure: Concerns spatial relations described in the sentence.

Language as a Matter of Linguistic Analysis

  • Language in a narrow and broad sense:

    • Animals do not talk.

    • Films have no language.

    • Computer languages are not language-like.

  • Levels of analysis:

    • Phonology: Speech sounds and their rules.

    • Morphology: Word forms, inflection (morphosyntax: compounding).

    • Syntax: Grammar (both as rules for language and thought).

    • Semantics: Linguistic meaning (words and sentences).

    • Pragmatics: Language in use, social attribution of intended meaning.

Language as a Matter of Neuroscience

  • Neurology and neuropsychology:

    • Brain injury: case studies, rehabilitation.

    • Localization-based reverse engineering.

  • Cognition: Language as a model of information processing.

    • Computational, algorithmic, implementation levels (Marr, 1982).

    • Language system - software - hardware

  • Cognitive neuroscience: Human language in an ape brain

    • Matching neural and language functions: language processing as neural activity

    • Exploring the brain via the workings of language

Outstanding Questions

  • Serial or parallel processing?

  • Modularity? Neural organization?

  • Top-down, bottom-up, interaction?

  • Separation of (Grammatical) rules and units (words)?

  • Regular vs. irregular inflection: organization of the mental lexicon?

  • Sentences as meaningful constructions vs. structures requiring syntactic analysis (surface vs. deep structure)?

  • Language =

    • Syntax (grammar): Formal Grammar: Chomsky, Bever, Pinker, Bloom

    • Semantics (meaning): Constructivists: Lakoff, Turner, Fauconnier, Talmy

    • Pragmatics (usage): Grice, Sperber, Tomasello

Broca's Patient

  • Broca's (1861) patient: Tan tan tan tan

Aphasia Models

  • Wernicke, 1874

  • Geschwind, 1972

  • Includes Broca's area, Wernicke's area, arcuate fasciculus, angular gyrus.

Aphasia Typology (Desai & Riccardi, 2021)

A table to classify aphasia:

  • Columns: Fluent, Comprehends, Repeats

  • Types of Aphasia:

    • Global

    • Mixed transcortical

    • Broca's

    • Transcortical motor

    • Wernicke's

    • Transcortical sensory

    • Conduction

    • Anomic

Anatomy of Language: A Review of 100 fMRI Studies (Price, 2010)

  • Different brain regions are activated by different language tasks.

  • fMRI studies show various areas associated with prelexical processing, word processing, semantic/syntactic ambiguity, sentence comprehension, word retrieval and articulation.

  • Components of Language:

    • Execution

    • Selection

    • Volition

    • Action

    • Hierarchical sequencing

    • Articulatory planning

    • Auditory-motor

    • Audio-visual sentences

    • Amodal semantic combinations

Towards a New Neurobiology of Language (Poeppel et al., 2012)

  • Includes Broca's area, Wernicke's area, primary auditory area.

  • Articulatory network

  • Dorsal stream

  • Combinatorial network

  • Ventral stream

  • Sensorimotor interface

  • Phonological network

  • Conceptual network

  • Lexical interface

Language Processing (Turker et al., 2023)

  • Includes Main Effect of Language, Overlap across Subdomains.

  • Subdomain Processing (Syntax, Semantics).

  • Domain Overlap (Phonology, Prosody).

Language Tracts and Gender Differences (Catani & Mesulam, 2008)

  • Fronto-parietal fibers

  • Anterior segment

  • Callosal Fibres

  • Geschwind's territory

  • Arcuate

  • Broca's territory

  • Long segment

  • Wernicke's territory

  • Posterior segment

  • Uncinate

  • Includes Group1 extreme left lateralization (-60%)

  • Includes Group2 mild left lateralization (-20%)

  • Includes Group3 bilateral, symmetrical (-20%)

Bilingualism (Kim et al., 1997)

  • Late Bilinguals

  • Early Bilinguals

Structural Correlates of Regular Inflection (Marslen-Wilson & Tyler, 2007)

  • Regular inflection – Broca’s area, BUT – a lower statistical threshold includes the arcuate fasciculus and Wernicke’s area?

  • Irregular inflection – grammar, but the rule does not help

    • meaning is specified but modified

    • non-language areas?

Broca is not Homogenous

  • Topological Approach

  • Hodological Approach

Functions of Broca

  • Unification of phonology syntax and semantics (Hagoort, 2005)

    • BA 47 & BA 45: semantics

    • BA 45 & BA 44: syntax

    • BA 44 & BA 6: phonology

  • Sequential processing during reading (Sahin et al., 2009)

    • 200 ms: lexical

    • 320 ms: grammatical

    • 400 ms: phonological

  • Morphological processing

    • complexity, inflection (e.g., Bozic et al., 2007; Marslen-Wilson & Tyler, 2007)

    • Morpho-syntactic compounding (e.g., Koester & Schiller, 2011)

  • Working Memory (e.g., Owen et al., 2005)

Broca’s Meta-Analysis: 485 Neuroimaging Studies (1991-2007) (Liakakis et al., 2011)

  • Cluster 1: Semantic & phonological tasks (BA 45/46)

    • verbal fluency

    • semantic retrieval

    • word repetition, synonym production

    • lexical decision

    • inner speech

    • semantic categorization

  • Cluster 2: Working memory (BA 44)

    • n-back

    • associative learning

    • recall of topographic routes

    • verbal memory

    • arithmetic operations

    • forward & backward digit recall

  • Cluster 3: Empathy (BA 47)

    • in/congruent motor expressions of emotions

    • laughing & seeing others laugh

    • feeling guilt

    • evaluating emotional face expressions

Alternative Hypotheses for Broca (Fink et al., 2005)

  • Broca’s area is a mosaic of many different functional regions, only some of which are directly related to language

  • Broca’s area computes a highly abstract function, such as selection from a pool of competing alternative representations, which subsumes aspects of language processing and many other cognitive systems, including local visual search

  • Particular neuroanatomical regions, including Broca’s area, change their functions consequent upon the simultaneous activation of other regions that are effectively connected to a given region

  • Modules could be functional and/or anatomical, but no 1:1 mapping between anatomy and function

Functional EEG Markers of Language

  • (E)LAN: Syntactic violations

    • Grammatical agreement or word category expectations (Hagoort, 2005)

    • “The woman sees the man because with the binoculars”

  • N400: Semantic processing

    • Predictions of semantic memory systems (Kutas & Federmeier, 2011)

  • P600: Syntactic integration

    • Violations, ambiguities, complexity

    • Garden path sentences

    • Constructing sentential structure

    • Word category violations evoke all effects ELAN N400 P600

Semantic Memory Retrieval: N400 (Kutas & Hillyard, 1980; Kutas & Federmeier, 2011)

  • Graphs used to demonstrate semantic congruity, semantic category, word repetition, word frequency, word position

N400-like MEG (Halgren et al. 2002)

  • Terminal Word

  • Incongruous

  • Congruous

  • subject 1

  • 300-500ms

Both Sides Get the Point (Federmeier et al., 2005)

  • Farmers grow…..

    • crops

    • worms
      *Difference worms-crops

Lateralization of Language (Van Lancker Sidtis, 2006)

  • LEFT HEMISPHERE

    • Linguistic

      • Phonetics

        • Segmental

        • Prosodic (timing; tones)

      • Phonology

      • Morphology

      • Syntax

      • Semantics

  • RIGHT HEMISPHERE

    • Contextual

      • Familiar proper nouns

      • Emotional terms

      • Pragmatics

      • Conversation

      • Topic, theme

      • Inference

      • Formulaic language

      • Prosodic (pitch)

      • ortholinguistics

      • paralinguistics

      • emotional meanings

      • attitudinal meanings

      • intonation in phrases

      • voice recognition

Figurative Language: Right Hemisphere Processing?

  • Language disruption of right hemisphere damaged patients

  • Social language: ambiguities, irony, jokes, indirect requests, etc.

    • Bottini et al. 1994 Winner & Gardner, 1977

What is a Metaphor?

  • “Odysseus is a lion…”

  • Who is the lion?

  • Why do we say things figuratively instead of always speaking literally?

  • How our brain is capable of differentiating and correctly interpreting?

The Neural Processing of Metaphor

  • Right hemisphere language

    • Language specialization beyond the left hemisphere?

    • Contradictory results (replication problems)

  • Serial processing: identifying a violation?

    • Literal meaning needs to be rejected

    • We process language based on literal “truth-value”

  • Are metaphors captivating because they are based on bodily experience?

    • Is language comprehension based on sensorimotor processing

    • Are there no language centers? (Gallese & Lakoff, 2005)

Novel Metaphors

  • Idiomatic, conventional metaphors have a lexical entry

  • Novel metaphors (“brilliant elegance”): semantic combinatorics

    • Right hemisphere?

    • Slower serial processing?

    • Embodied sensorimotor simulations?

Coarse Semantic Coding Theory (Beeman, 1998; Jung-Beeman, 2005)

  • Shows processing in both hemispheres with diagrams.

Disentangling Familiarity & Figurativeness Forgács, B., Lukács, Á., & Pléh, Cs. (2014)

  • Is it metaphoricity or novelty that drives right hemisphere effects?

  • Serial processing of figurative meaning or just slower processing of unusual expressions?

  • Divided visual field paradigm + eye-tracking

Forgács, B., Bohrn, I., Baudewig, J., Hofmann, M. J., Pléh, Cs., & Jacobs, A. M. (2012).

  • Event-related fMRI study

Conventionals > Novels

  • Left Hemisphere

  • Right Hemisphere

t(39)

p<0.000010

Metaphorical Language

  • RH processing: no evidence

    • No specialized neural systems either for metaphors: even novel metaphors are processed by in the LH

    • No RH processing of novel language

  • Serial processing: no evidence

    • Novel metaphors are processed just as quickly as literal expressions matched in novelty

    • There still might be metaphor-specific computations (fMRI and EEG data)

  • Embodiment?: no evidence

    • Literal language evokes sensorimotor feature processing, but metaphors do not

    • The early automatic activation of literal meaning (and corresponding sensorimotor areas) are not contested

    • When concrete words are used in figurative sense, they evoke an abstractness effect

    • What is left of embodiment as a model of cognition if it cannot account for abstract thought?

Summary

  • From Broca and Wernicke to neural networks

  • Language processing as a neural function

  • From syntax to meaning

  • Inflection

  • Sentence construction

  • From the left hemisphere to the right

  • From literal language to figurative meaning