Notes on Sentence Processing: Two-Stage Models and Constraint-Based Parsing
Overview
Focus of the lecture: understanding sentence processing in psychology, linguistics, and related fields through models of parsing and interpretation.
Central goal: explain how the mind assigns structure to sentences (syntax) and integrates meaning (semantics), and why parsing can be difficult or biased.
Two broad model families discussed:
Serial/two-stage approaches (syntax first, then semantics) or initial syntactic parsing followed by revision if needed.
Constraint-based approaches where syntax, semantics, and lexical properties (e.g., verb information, frequency) influence parsing from the start.
The talk ties to broader themes in language processing, including serial search vs. interactive activation, and aims to answer why sentences are parsed the way they are, and why ambiguity is common.
Key concepts and definitions
Parsing: the assignment of a syntactic structure to a sentence.
Syntactic processing vs semantic processing: the two components that may operate serially or interactively.
Terminology for sentence structure:
NP = noun phrase
VP = verb phrase
PP = prepositional phrase
S = sentence (often constructed from hierarchical phrases)
Incremental syntactic assignment: structures are built as each word becomes available.
Attachment strategies:
Minimal attachment: choose the syntactic structure with the fewest nodes/nonterminal commitments.
Late closure: prefer attaching new material to the phrase currently being processed.
Garden-path effect / guarded path effect: when initial parsing leads to an incorrect interpretation that must be revised upon encountering disambiguating information.
Complementizer (e.g., that): a word that introduces a clause and can reduce ambiguity by creating an independent structure.
Moving from ambiguity to disambiguation often requires a secondary analysis or reanalysis of structure.
Working memory (WM): a cognitive system with limited capacity that stores and processes information to integrate current input with prior context; important for complex processing, including sentence comprehension.
Constraint-based parsing: a framework where multiple sources of information (lexical, syntactic, semantic, frequency, verb semantics) constrain the parse in parallel rather than strictly sequentially.
Verb- and semantics-driven constraints: the idea that the properties of verbs (subcategorization frames, argument structure) influence how sentences are parsed.
Semantic hierarchy in verb semantics: some roles (e.g., experience vs. stimulus) are more prominent in interpretation and influence processing ease.
Reduced relative clauses and garden-path sentences: classic paradigms to probe how people resolve syntactic ambiguity under memory and knowledge constraints.
Theoretical models of sentence processing
Two autonomous stages (as discussed in the lecture):
Stage 1: syntactic parsing – assign structure to the sentence (syntactic framework).
Stage 2: semantic integration – add semantic information to the syntactic frame.
Note: even though two stages are described, there is debate about interaction; some models allow parallel/synchronous influence of semantics on syntax.
Serial (data-driven) parsing model (garden-path style):
Information is processed in a sentence based on one initial analysis.
If that analysis fails, a secondary analysis is initiated to achieve successful processing.
This is a classic garden-path scenario where early interpretations must be revised.
Interactive/constraint-based models:
Syntax and semantics are used together from early on; lexical information (especially verbs) guides parsing decisions.
Verbs are especially important because they constrain potential argument structure and role assignments.
Constraints include frequency of usage, plausibility, and world knowledge.
Attachment strategies and example scenarios
Minimal attachment vs late closure (attachment strategies):
Minimal attachment: prefer the simplest syntactic tree consistent with the encountered input (fewer nonterminal nodes).
Late closure: attach incoming material to the phrase currently being processed, which can lead to garden-path effects if the attachment is incorrect.
Example 1: "I found a wallet in her backpack" (attachment choice)
One interpretation attaches "in her backpack" to wallet (NP), yielding a simpler NP structure.
Alternative interpretation attaches "in her backpack" to the VP "found" (PP as object of the sentence), which can be more complex.
Example 2: Verbal phrase attachment with distance expression
Base sentence: "Jay always jogs two kilometers".
Leg closure tendency: readers tend to attach "two kilometers" as an object to the verb (VP), interpreting it as a direct object attachment in naïve parsing.
If the context implies a different structure (e.g., the distance is not a direct object but a modifier of the action described in the clause that follows), a reanalysis may be needed (gives rise to garden-path effects).
Example 3: Complementizer usage to reduce ambiguity
Ambiguous: "The criminal confessed he is seeing harm to many people" (ambiguous because it could attach to a VP or form a higher clause).
Less ambiguous: with a complementizer: "The criminal confessed that he's seeing harm to many people" which creates an explicit clause boundary and reduces garden-path risk.
Guarded path effect (garden-path effect): the reading times tend to be slower in the ambiguity region when the initial parsing is incorrect and must be revised.
Constraint-based parsing and verb semantics
Key idea: parsing decisions are influenced not only by syntactic structure but also by lexical information and semantics, especially verb subcategorization frames.
Classic examples discussed:
The reporter saw either giving you a compensator or not giving you a compensator, a friend was not succeeding. (Ambiguity depends on how the verb governs the following clause.)
The reporter saw her friend was not succeeding. Show ambiguity under certain verb contexts depending on the presence of a complementizer.
The difference between attachment strategies can be modulated by verb type:
For example, with the verb see vs say, readers show different attachment preferences, reflecting a clause-bias vs object attachment bias influenced by verb type.
Frequency and experience-based constraints:
The idea that how often a structure is encountered (frequency) and how likely a given verb context is to occur can influence the ease of attachment and the preferred interpretation.
The results do not always align with purely minimal attachment or leg-closure predictions; they align more with constraint-based models.
The role of verb semantics in structures: verbs carry information about the agent/patient roles and how an event is related, which constrains parsing beyond the syntactic rules alone.
The semantic hierarchy of verb semantics and processing ease
A distinction is drawn between verb types that encode different argument structures:
Surprising or amusing verbs: typically involve a stimulus (agent/experiencer) and a patient (the experience) in a particular alignment.
Fear or cherish-type verbs: the experience is the agent or the experiencer with the stimulus as the object.
Examples and interpretation cues:
Active vs passive and the argument relation (who is the stimulus vs who experiences) influence ease of processing.
In English, the experience tends to be a prominent semantic role in subject position; this can make sentences with experiencer as subject easier to process than those where the experience is hidden or the stimulus is the subject.
Experimental approach to semantic roles:
Present sentences with variations in who is the experiencer vs the stimulus and measure reading times for active and passive constructions.
Hypotheses derived from semantic hierarchy predict that sentences with higher-prominence semantic roles in the subject position are easier to process than those where the prominent role is not in subject position.
Example results (reading-time patterns):
Active example: "The joke amused the host" vs passive: "The host was amused by the joke".
Theory predicts the passive (where the experiencer is in a postverbal position) may have different processing demands; reading-time data show patterns consistent with the semantic-priority view for certain verbs.
Working memory and individual differences in sentence processing
Working memory (WM) definition and role:
WM is a cognitive system for temporarily storing and processing information to support ongoing reasoning and tasks, including language processing.
It involves a competition for finite resources between storage and processing, which can affect real-time sentence processing.
Measures and tasks:
Reading-span or similar tasks to gauge WM capacity.
Word-by-word reading tasks with self-paced progression to measure processing difficulty word-by-word (e.g., pressing a spacebar to advance through words and measure reading times per word).
Relationship between WM and parsing:
Studies explored whether higher WM capacity helps resolve temporary syntactic ambiguity (garden-path sentences).
Predictions: high WM should help when there is a temporary garden-path; low WM would show greater difficulty.
Key empirical takeaway from discussed work:
The evidence for a tight, universal link between WM and syntactic parsing is mixed.
Some studies find little support for a strong WM effect on syntactic parsing; high WM does not consistently predict better parsing of garden-path structures.
This has led to nuanced views: WM may influence other aspects of reading (e.g., long-range dependencies, recall) but not necessarily the core syntactic parsing step.
Neuropsychological evidence and dissociations:
Case studies of patients with brain damage (e.g., aphasia) show dissociations between production and comprehension abilities.
Caramazza & Zurif (classic study) showed that some patients with agrammatic aphasia have intact semantic comprehension but impaired syntactic processing, suggesting that syntax and semantics can be dissociated.
Other studies show production can be impaired while comprehension remains relatively intact, or vice versa, challenging the idea that syntax is entirely shared between production and comprehension.
Implications of case studies:
The brain may contain separate or partially overlapping systems for language and for general working memory.
Different brain regions (e.g., Broca’s area) contribute to different aspects of language, and damage can lead to varied profiles across production and comprehension.
Neuropsychological evidence on production vs comprehension
Caramazza & Zurif (classic study) and subsequent work:
Used sentence-picture matching tasks to test comprehension in aphasic patients.
Found that some patients with impaired production (agrammatic speech) could still comprehend semantically reversible sentences, suggesting preserved semantics and partial syntactic processing.
Reduced relative clauses and syntactic processing in patients:
Some patients show good comprehension for semantically non-reversible sentences but struggle with syntactic reversals when word-order changes affect interpretation.
Double dissociation examples and counterarguments:
Some cases show production intact but comprehension impaired, while others show the opposite pattern.
These findings challenge the claim that syntax is entirely shared between production and comprehension.
Practical takeaway:
Language structure and meaning processing may rely on overlapping but not perfectly shared neural and cognitive resources.
Data from aphasia studies highlight the complexity and variability of how syntax and semantics are implemented in the brain.
Additional notes from the lecture:
A video clip about Broca’s area was shown to illustrate how brain damage can impact language processing.
Measurement and experimental details discussed
Reading-time methodology in the semantic-attachment studies:
Participants read sentences word-by-word, typically advancing with a space bar.
Reading time for each word is measured from the onset of the word to the space bar press that advances to the next word.
In some cases, participants indicate if a sentence is unacceptable or not, providing data on processing difficulty at specific regions (e.g., ambiguity regions).
Example data points mentioned:
In one verb-based comparison, reading times for the verb in one context (1b) were faster than in another context (1a).
For a second sentence variant, 2a was faster than 2b (e.g., where embodiment of semantics or clause structure changed ease of processing).
Specific numbers cited: 434 ms vs 459 ms for related conditions, illustrating small but reliable timing differences that reflect parsing ease or difficulty.
Interpretation of reading-time data:
Faster reading times in ambiguous regions suggest easier parsing in that condition, often due to more plausible initial attachments guided by semantics or lexical constraints.
Slower reading times in ambiguous regions suggest garden-path effects, where initial interpretation is revised later.
Connections to broader theories and prior work
Serial search vs interactive activation models (referenced as prior weeks):
Serial search aligns with the garden-path perspective: parse step-by-step, revise as needed.
Interactive activation models align with constraint-based parsing: multiple cues (syntactic, semantic, lexical) activate competing structures in parallel.
Foundational principles:
The role of verbs in constraining structure is a central theme across models, underscoring how lexical information can shape parsing before full syntactic structure is determined.
The idea that not all parsing decisions can be explained by simple minimal attachment or late-closure alone, and that frequency and experience modulate expectations.
Real-world relevance and implications:
Understanding sentence processing helps explain why language learners struggle with ambiguity and how different languages (with different word orders and verb semantics) might yield different parsing profiles.
Clinical insights from aphasia research inform language rehabilitation and highlight potential separation between syntax and semantics in the brain.
Summary of takeaways for exam preparation
Expect questions on the distinction between two-stage (syntax-first vs constraint-based) accounts of parsing and how they predict different outcomes in garden-path scenarios.
Be able to explain attachment strategies (minimal attachment, late closure) and how they can lead to garden-path effects depending on sentence structure and context.
Understand the role of complementizers in removing ambiguity and how complementizers influence syntactic tree-building.
Recognize that verb semantics and lexical constraints play a crucial role in constraint-based parsing, including frequency effects and the predictive value of verb subcategorization frames.
Be able to discuss the semantic hierarchy concept (experience vs stimulus) and how it can influence processing ease in different syntactic configurations.
Recall that working memory has a nuanced relationship with parsing; it does not uniformly predict garden-path effects, though it relates to broader reading skills and handling of complex structures.
Appreciate the evidence from neuropsychology (production vs comprehension dissociations, agrammatic aphasia) and what it implies about separable vs shared syntactic processing systems in the brain.
Be prepared to discuss how reading-time data are collected and interpreted, including word-by-word measures and region-by-region comparison in ambiguity regions.
Notation and LaTeX references
General structural notation:
NP, VP, PP, S denote phrase and sentence structures.
Attachment strategies discussed in terms of node-minimization and phrase scope.
Example formulas (conceptual):
Stage 1:
Stage 2:
Reading-time data presentation: measured in milliseconds, e.g., for different conditions.
Important terms to review
Parsing, syntactic structure, NP, VP, PP, S, complementizer, garden-path, guard path effect, minimal attachment, late closure, working memory, constraint-based parsing, frequency effects, verb subcategorization, semantic roles (experiencer, stimulus), reduced relative clauses, production vs comprehension dissociations, agrammatic aphasia.
If you’d like, I can convert these notes into a printable study sheet or generate a condensed formula-driven cheat sheet for quick review before the exam.