What makes language special in neuroscience?
Language as an object of investigation: levels of analysis and interpretation
Linguistics vs. cognitive neuroscience
Language processing: brain structures and functions
Can language be localized? Is the brain modular?
Is function determined by location or the other way around?
An example from the cognitive neuroscience of language
The language system is unique in the universe.
Its evolutionary origin and acquisition in childhood are among the greatest scientific riddles.
One of the first anatomically identified, non-sensorimotor brain functions
Neural specialization and specificity: Which area(s) subserve language?
Plasticity and modularity: Are there systems dedicated to specific functions in general, and can these be localized?
One of the most abstract (non-sensorimotor) brain functions.
Can language be linked to or derived from perception, speech sounds, or perceptual systems?
Are language and thought identical? (Saphir-Whorf hypothesis)
von Humboldt (1836): Language is infinite. Infinite employment of finite means (freedom of thought).
de Saussure (1916): Language is arbitrary. The relationship between the signifier and signified is undetermined:
"table" – "asztal" (it is not possible to map it on the world with no teaching/learning)
Chomsky (1957): Language is extremely complex computationally
Its combinatorial explosion makes it impossible to learn it through examples.
Speech is not sequential but a hierarchical structure.
Language is grammar.
A sentence can be analyzed at multiple levels:
(a) Phonological structure: Includes prosodic structure, syllabic structure, and segmental structure.
(b) Syntactic structure: Describes the grammatical relationships between words.
(c) Semantic/conceptual structure: Represents the meaning of the sentence.
(d) Spatial structure: Concerns spatial relations described in the sentence.
Language in a narrow and broad sense:
Animals do not talk.
Films have no language.
Computer languages are not language-like.
Levels of analysis:
Phonology: Speech sounds and their rules.
Morphology: Word forms, inflection (morphosyntax: compounding).
Syntax: Grammar (both as rules for language and thought).
Semantics: Linguistic meaning (words and sentences).
Pragmatics: Language in use, social attribution of intended meaning.
Neurology and neuropsychology:
Brain injury: case studies, rehabilitation.
Localization-based reverse engineering.
Cognition: Language as a model of information processing.
Computational, algorithmic, implementation levels (Marr, 1982).
Language system - software - hardware
Cognitive neuroscience: Human language in an ape brain
Matching neural and language functions: language processing as neural activity
Exploring the brain via the workings of language
Serial or parallel processing?
Modularity? Neural organization?
Top-down, bottom-up, interaction?
Separation of (Grammatical) rules and units (words)?
Regular vs. irregular inflection: organization of the mental lexicon?
Sentences as meaningful constructions vs. structures requiring syntactic analysis (surface vs. deep structure)?
Language =
Syntax (grammar): Formal Grammar: Chomsky, Bever, Pinker, Bloom
Semantics (meaning): Constructivists: Lakoff, Turner, Fauconnier, Talmy
Pragmatics (usage): Grice, Sperber, Tomasello
Broca's (1861) patient: Tan tan tan tan
Wernicke, 1874
Geschwind, 1972
Includes Broca's area, Wernicke's area, arcuate fasciculus, angular gyrus.
A table to classify aphasia:
Columns: Fluent, Comprehends, Repeats
Types of Aphasia:
Global
Mixed transcortical
Broca's
Transcortical motor
Wernicke's
Transcortical sensory
Conduction
Anomic
Different brain regions are activated by different language tasks.
fMRI studies show various areas associated with prelexical processing, word processing, semantic/syntactic ambiguity, sentence comprehension, word retrieval and articulation.
Components of Language:
Execution
Selection
Volition
Action
Hierarchical sequencing
Articulatory planning
Auditory-motor
Audio-visual sentences
Amodal semantic combinations
Includes Broca's area, Wernicke's area, primary auditory area.
Articulatory network
Dorsal stream
Combinatorial network
Ventral stream
Sensorimotor interface
Phonological network
Conceptual network
Lexical interface
Includes Main Effect of Language, Overlap across Subdomains.
Subdomain Processing (Syntax, Semantics).
Domain Overlap (Phonology, Prosody).
Fronto-parietal fibers
Anterior segment
Callosal Fibres
Geschwind's territory
Arcuate
Broca's territory
Long segment
Wernicke's territory
Posterior segment
Uncinate
Includes Group1 extreme left lateralization (-60%)
Includes Group2 mild left lateralization (-20%)
Includes Group3 bilateral, symmetrical (-20%)
Late Bilinguals
Early Bilinguals
Regular inflection – Broca’s area, BUT – a lower statistical threshold includes the arcuate fasciculus and Wernicke’s area?
Irregular inflection – grammar, but the rule does not help
meaning is specified but modified
non-language areas?
Topological Approach
Hodological Approach
Unification of phonology syntax and semantics (Hagoort, 2005)
BA 47 & BA 45: semantics
BA 45 & BA 44: syntax
BA 44 & BA 6: phonology
Sequential processing during reading (Sahin et al., 2009)
200 ms: lexical
320 ms: grammatical
400 ms: phonological
Morphological processing
complexity, inflection (e.g., Bozic et al., 2007; Marslen-Wilson & Tyler, 2007)
Morpho-syntactic compounding (e.g., Koester & Schiller, 2011)
Working Memory (e.g., Owen et al., 2005)
Cluster 1: Semantic & phonological tasks (BA 45/46)
verbal fluency
semantic retrieval
word repetition, synonym production
lexical decision
inner speech
semantic categorization
Cluster 2: Working memory (BA 44)
n-back
associative learning
recall of topographic routes
verbal memory
arithmetic operations
forward & backward digit recall
Cluster 3: Empathy (BA 47)
in/congruent motor expressions of emotions
laughing & seeing others laugh
feeling guilt
evaluating emotional face expressions
Broca’s area is a mosaic of many different functional regions, only some of which are directly related to language
Broca’s area computes a highly abstract function, such as selection from a pool of competing alternative representations, which subsumes aspects of language processing and many other cognitive systems, including local visual search
Particular neuroanatomical regions, including Broca’s area, change their functions consequent upon the simultaneous activation of other regions that are effectively connected to a given region
Modules could be functional and/or anatomical, but no 1:1 mapping between anatomy and function
(E)LAN: Syntactic violations
Grammatical agreement or word category expectations (Hagoort, 2005)
“The woman sees the man because with the binoculars”
N400: Semantic processing
Predictions of semantic memory systems (Kutas & Federmeier, 2011)
P600: Syntactic integration
Violations, ambiguities, complexity
Garden path sentences
Constructing sentential structure
Word category violations evoke all effects ELAN N400 P600
Graphs used to demonstrate semantic congruity, semantic category, word repetition, word frequency, word position
Terminal Word
Incongruous
Congruous
subject 1
300-500ms
Farmers grow…..
crops
worms
*Difference worms-crops
LEFT HEMISPHERE
Linguistic
Phonetics
Segmental
Prosodic (timing; tones)
Phonology
Morphology
Syntax
Semantics
RIGHT HEMISPHERE
Contextual
Familiar proper nouns
Emotional terms
Pragmatics
Conversation
Topic, theme
Inference
Formulaic language
Prosodic (pitch)
ortholinguistics
paralinguistics
emotional meanings
attitudinal meanings
intonation in phrases
voice recognition
Language disruption of right hemisphere damaged patients
Social language: ambiguities, irony, jokes, indirect requests, etc.
Bottini et al. 1994 Winner & Gardner, 1977
“Odysseus is a lion…”
Who is the lion?
Why do we say things figuratively instead of always speaking literally?
How our brain is capable of differentiating and correctly interpreting?
Right hemisphere language
Language specialization beyond the left hemisphere?
Contradictory results (replication problems)
Serial processing: identifying a violation?
Literal meaning needs to be rejected
We process language based on literal “truth-value”
Are metaphors captivating because they are based on bodily experience?
Is language comprehension based on sensorimotor processing
Are there no language centers? (Gallese & Lakoff, 2005)
Idiomatic, conventional metaphors have a lexical entry
Novel metaphors (“brilliant elegance”): semantic combinatorics
Right hemisphere?
Slower serial processing?
Embodied sensorimotor simulations?
Shows processing in both hemispheres with diagrams.
Is it metaphoricity or novelty that drives right hemisphere effects?
Serial processing of figurative meaning or just slower processing of unusual expressions?
Divided visual field paradigm + eye-tracking
Event-related fMRI study
Left Hemisphere
Right Hemisphere
t(39)
p<0.000010
RH processing: no evidence
No specialized neural systems either for metaphors: even novel metaphors are processed by in the LH
No RH processing of novel language
Serial processing: no evidence
Novel metaphors are processed just as quickly as literal expressions matched in novelty
There still might be metaphor-specific computations (fMRI and EEG data)
Embodiment?: no evidence
Literal language evokes sensorimotor feature processing, but metaphors do not
The early automatic activation of literal meaning (and corresponding sensorimotor areas) are not contested
When concrete words are used in figurative sense, they evoke an abstractness effect
What is left of embodiment as a model of cognition if it cannot account for abstract thought?
From Broca and Wernicke to neural networks
Language processing as a neural function
From syntax to meaning
Inflection
Sentence construction
From the left hemisphere to the right
From literal language to figurative meaning