Claude's LM Exam 2

Rescorla-Wagner model: what is it called in terms of surprise (type of model)?

A US Reduction model. The US becomes less surprising as the CS predicts it better, so the ability of the US to produce learning is gradually reduced.

R-W model: main assumption about associative learning tasks

Associative learning tasks = predictive learning tasks. CS = predictor, US = outcome. Each CS-US pairing increases the CS's ability to predict the US.

R-W model: (a) What produces learning?

Pairing the CS (predictor) with the US (outcome) when the US is surprising. Each pairing increases the association strength and the CS's ability to predict the US.

R-W model: (b) What causes learning to stop?

As prediction improves, the US becomes less surprising and the amount learned per trial shrinks. When the US is 100% predicted (not surprising), learning stops (V = lambda).

R-W model: (c) How do theoretical increments in associative strength relate to the observed learning curve?

Early trials = large increments (US very surprising); later trials = smaller increments; at asymptote = no more learning. This matches the empirical negatively accelerating learning curve (rapid early learning, then leveling off).

Learning curve shape (R-W / role of surprise)

Negatively accelerating function: big gains early, smaller gains later, leveling off at an asymptote (maximum possible learning).

Role of surprise in learning (lecture assumptions)

Surprise = what you get is not what you expect. It is necessary for learning, determines how much is learned on any episode, decreases as total knowledge increases, and when knowledge = maximum, learning stops.

R-W: What does V stand for?

V = current strength of the CS-US association (what you predict).

R-W: What does lambda (λ) stand for?

Lambda = maximum possible strength of the CS-US association (what actually happens; the US).

R-W: What does k stand for?

k = learning rate parameter, reflecting the salience of the CS/US. It determines how fast learning occurs.

R-W: How is surprise of the US on a trial calculated?

Surprise = (λ - V) = what happens minus what you predicted.

R-W formula: change in associative strength on a trial (ΔV)

ΔV = k(λ - Vprev)

R-W formula: updated associative strength after the current trial (V current)

V_current = ΔV + Vprev

R-W computation: what is V_prev on Trial 1 of acquisition?

V_prev = 0 on Trial 1.

R-W computation: worked example with k = 0.4 and λ = 1 (Trial 1 and 2)

Trial 1: ΔV = 0.4(1 - 0) = 0.4, V = 0.4. Trial 2: ΔV = 0.4(1 - 0.4) = 0.24, V = 0.64. Increments shrink each trial.

R-W: How do you identify the last trial on which learning occurs in a computational example?

The last trial where ΔV > 0. Learning stops when V = λ, so ΔV = 0 (practically, when V stops changing).

R-W: What happens if k is bigger?

The curve rises faster/steeper at first, but it levels off at the same place (same asymptote, set by lambda).

R-W: What happens if lambda is bigger?

The learning rate (shape/speed) is the same, but the curve levels off at a higher asymptote (different maximum).

R-W: Main job of the model

To predict theoretical trial-by-trial changes in associative strength (V).

R-W: What does the model assume about the strength of the CR?

That the strength of the CR is proportional to the strength of learning (V).

R-W: trial-by-trial predictions summary

Early = large increments in learning; Later = small increments; At maximum = learning stops.

R-W: What core phenomena of classical conditioning does it correctly predict?

Acquisition (and how it proceeds), extinction (and how it proceeds), blocking, overshadowing, and overexpectation.

Why do people like the R-W model?

It is elegant: it explains a lot with very little (simple equation, big impact; one of the most exported models outside psychology).

Extinction according to the R-W lecture notes

Extinction is treated as unlearning (V decreases toward 0 when the US is omitted). Note: after extinction is complete, spontaneous recovery still occurs.

Empirical vs. theoretical: what is the difference?

Empirical effect = what was actually MEASURED (e.g., one group showed greater conditioned changes in heart rate/startle). Theoretical conclusion = what we INFER (e.g., that group learned/had more fear).

'Paired with' vs. 'associated with'

'Paired with' is a PROCEDURE. 'Associated with' is an ASSUMPTION about learning that has taken place.

Define blocking

When two CSs are paired with one US, prior learning about one CS (CS1) prevents learning about the second CS (CS2). CS1 already predicts the US, so CS2 provides no new information.

Blocking design (Exp. group)

Phase 1: CS1 → US. Phase 2: [CS1 + CS2] → US. Test: CS2 alone → little or no CR (weak cr). Control group skips Phase 1 and shows a strong CR to CS2.

Define overshadowing

When two CSs are presented together (compound) with one US, the more salient/noticeable CS acquires a stronger association with the US than the less salient CS.

Overshadowing example (scapegoat effect)

Chemotherapy patients eat a strong-flavored candy (salient) plus their normal food before treatment. The candy becomes the 'scapegoat' for nausea, so they develop less aversion to the normal food (Flavor 2).

Overshadowing: adaptive vs. maladaptive examples

Adaptive: blue jay learns to avoid the salient bad-tasting butterfly. Maladaptive: chemo patients associate nausea with a salient candy instead of treatment. Scapegoating technique = overshadowing.

Why does blocking occur according to R-W? (model language)

CS1 has already been conditioned (V approaches λ), so in Phase 2 the US is no longer surprising (λ - V ≈ 0). CS1 has 'used up' all the available associative strength, leaving none for CS2. Model assumption: lambda is conserved.

Why does blocking occur? (plain English)

CS1 already predicts the US, so there is nothing new to learn about the US when CS2 is added. CS2 is a redundant predictor, so it never gets associated with the US.

Why does overshadowing occur according to R-W?

The two CSs compete for the limited associative strength (λ). The more salient CS (higher k) gains strength faster and takes most of it, leaving less for the less salient CS.

Why is blocking significant?

It violates the assumption that learning is automatic. Contiguity, frequency, and intensity (laws of associative learning) are not enough: learning depends on whether the US is surprising.

Blocking in humans: minefield/gun experiment (causal learning)

Grp 1 pretrained Minefield → explosion, then Minefield + Gun → explosion. Grp 2 had no pretraining. Grp 1 rated the Gun as less likely to cause the explosion (learning about the gun was blocked). Suggests classical conditioning may underlie causal learning.

Blocking in humans: chocolate/licorice allergy example

Pretraining: chocolate → allergic reaction. Then chocolate + licorice → allergic reaction. People judge licorice 'probably not' the cause; without pretraining, both seem possible. Blocking affects learning about CAUSES.

What is stimulus selection?

The process whereby the nervous system selects some stimuli for processing and not others.

Stimulus selection and perception: what is the link?

What we perceive is only a small part of everything happening around us because of stimulus selection. Classical conditioning shows which stimuli get selected for learning.

Are blocking and overshadowing examples of stimulus selection? Why?

Yes. In both, two CSs are paired with a US but one CS is selected for learning and the other is not; the CSs compete for learning about the US. Blocking selects the previously learned CS; overshadowing selects the more salient CS.

What do blocking/overshadowing suggest about the role of classical conditioning in perception?

Associative learning is predictive learning, and learning is NOT automatic. We do not learn about every stimulus we encounter; it depends on how well a stimulus predicts the US and how salient it is. CC may underlie causal learning in people (cause → effect).

What is a CS Reduction model?

A model in which learning depends on how much the CS provides predictive/useful information about the US (processing of the CS is what changes).

CS Reduction vs. US Reduction models

CS reduction: attention to/processing of the CS changes depending on how informative it is. US reduction (R-W): learning depends on how surprising the US is; a fully predicted US produces little or no learning.

Which model better explains the CS Preexposure Effect?

A CS Reduction model. Preexposure teaches the organism that the CS is uninformative (predicts nothing), so it is harder for the CS to acquire predictive value later.

Thorndike's Law of Effect

Behavior followed by a satisfying outcome → more likely to be repeated. Behavior followed by an annoying outcome → less likely to be repeated. Consequences of behavior are critical determinants of learning.

John Watson's message

The environment predicts ALL behavior; experience with the environment determines who we are.

B.F. Skinner: key ideas

Consequences of behavior drive the structure of behavior; response-reinforcer contingency is key; invented the Skinner box and free-operant trials; radical behaviorist who argued behavior is lawful and can be predicted/modified ('free will is fiction').

Skinner's three-term contingency (A-B-C)

A = Antecedent of behavior, B = Behavior, C = Consequence that follows behavior. Stimulus → response → outcome.

Operant vs. classical conditioning contingency

Operant: the outcome is dependent on/caused by the response. Classical: the outcome is not caused by the behavior.

Language of the response-reinforcer contingency

SD : R → SR. SD = discriminative stimulus, R = response, SR = reinforcing stimulus. SΔ (S-delta) = discriminative stimulus signaling extinction (no reward). SP = punishment.

What do discriminative stimuli do?

They convey which rule (response-reinforcer contingency) is currently in effect. Example: Tone: lever press → food (SD); Light: lever press → no food (SΔ).

Discrete-trial procedures

Distinct, separated trials; the operant response is performed one at a time in a trial; the EXPERIMENTER determines the number of trials (e.g., T-maze, 'What's this word?').

Free-operant procedures

Continuous responding with no clear trial boundaries; the SUBJECT determines the number of responses (e.g., rat pressing a lever). Skinner's innovation; allows continuous observation of behavior in real time and better establishes cause-effect between contingency and behavior.

Emitted vs. elicited behavior (operant vs. respondent)

Classical (respondent) responses are elicited and narrowly defined (eyeblink, GSR, heart rate). Operant responses are emitted and defined as a CLASS of responses: anything that produces the outcome.

Shaping by successive approximation

Teaching a new behavior by breaking it into steps and gradually raising the threshold of what behavior is rewarded (e.g., approach lever → touch → press). It takes advantage of and captures the inherent variability in naturally occurring behavior.

Two consequences of shaping

1) Increases the probability of a response. 2) Creates a NEW response. Skinner claimed shaping may underlie complex human behavior.

What is response chaining?

Linking multiple individual learned behaviors together in a specific sequence, each behavior serving as a cue for the next. Uses shaping; produced via instrumental conditioning.

Forward chaining

Teach the FIRST step first, then add each next step in order until the whole chain is learned. Problem: for long chains, the first link is far from the reward so early links tend to drift.

Backward chaining

Teach the LAST step first, then add previous steps until the chain is learned. Benefits: the last behavior is already reinforced and can serve as a secondary reinforcer for earlier steps; easier/more manageable; reinforcement comes earlier in training.

Task analysis (in chaining)

Identifying the discrete components/parts of the entire behavior sequence.

Project Pigeon

Skinner trained pigeons to peck at a screen to guide missiles to their targets, showing shaping and response chaining for complex behavior (later honored with an Ig Nobel).

Positive reinforcement

Presenting a DESIRABLE event after a behavior → behavior INCREASES. Example: getting candy for good grades.

Negative reinforcement

REMOVING an UNDESIRABLE event after a behavior → behavior INCREASES. Example: you don't have to do the dishes if you get good grades.

Positive punishment

PRESENTING an UNDESIRABLE event after a behavior → behavior DECREASES. Example: doing dishes for a month (or scolding).

Negative punishment

REMOVING a DESIRABLE event after a behavior → behavior DECREASES. Example: no candy (or no TV privileges) for a month.

How to state any instrumental contingency on the exam

Specify BOTH parts: whether a DESIRABLE vs. UNDESIRABLE event is PRESENTED vs. REMOVED, plus the effect on behavior (increase vs. decrease). In this system, 'positive' = presented and 'negative' = removed; reinforcement increases, punishment decreases behavior.

Reinforcement vs. reinforcer

Reinforcement is the PROCESS; a reinforcer is the EVENT/stimulus (food, candy bar, money).

Primary vs. secondary reinforcers

Primary: naturally reinforcing (e.g., food). Secondary: stimuli in the environment that have become associated with positive things. Generalized secondary reinforcer: works in many circumstances (e.g., money).

Extrinsic vs. intrinsic reinforcers

Extrinsic: can be separated from the behavior (cook who doesn't care about food, works for pay). Intrinsic: cannot be separated from the behavior being reinforced (cook who loves food).

Variables that affect reinforcer effectiveness

Contingency, Quantity, Quality, Contiguity, and Prior experience.

Reinforcer variable: Contingency

Is the reinforcer actually contingent on (caused by) the response?

Reinforcer variable: Contiguity

The closer in time the reward follows the behavior, the more effective it is.

Why does a delay between response and reinforcer weaken learning?

It becomes less clear which behavior is being reinforced, and other behaviors may occur in the meantime (interference), weakening the response-reinforcer association.

How can you compensate for a delay between response and reward?

Use a secondary reinforcer (e.g., money bridges work → food) or a marker (e.g., a clicker or coach marking the target behavior).

Adventitious reinforcement effect (Skinner)

If a behavior is immediately followed by a reinforcer, it is strengthened regardless of what the behavior is or whether it actually caused the reinforcer (erroneous self-shaping).

How does adventitious reinforcement support the importance of temporal contiguity?

Behavior is strengthened with NO real contingency, only close timing. So close temporal contiguity between response and reinforcer is a key factor in instrumental conditioning.

Adventitious reinforcement: other implications

Supports noncontingent reinforcement as the basis of human superstition; may contribute to psychopathology by strengthening irrational beliefs (paranoia, unrealistic self-appraisal in eating disorders, exaggerated threat in anxiety disorders).

Positive behavioral contrast

Small reward in Phase 1 (S) → LARGE reward in Phase 2 (S-L). Terminal running speed is FASTER than a group that got Large all along (L-L). S-S stays the same. Due to prior experience, not just reward magnitude.

Negative behavioral contrast

LARGE reward in Phase 1 → SMALL reward in Phase 2 (L-S). Running speed is SLOWER than a group that got Small all along (S-S). L-L stays the same. Due to prior experience, not just reward magnitude.

Behavioral contrast: what does it show?

Prior experience with a reward (not just its specific magnitude) determines how strongly it reinforces. Exam alert: always specify 'positive' vs. 'negative' behavioral contrast.

Multiple schedule

A linear series of different reinforcement schedules that repeats; r+ is earned while each schedule is in effect. An experimental model of everyday contingencies. Response rate reflects the conditioned reinforcing value of each schedule.

Contrast in multiple schedules

Negative contrast: a schedule becomes less attractive when the rate of r+ in the other schedule increases.

Positive contrast: a schedule becomes more attractive when the rate of r+ in the other schedule decreases.

Schedule of reinforcement

The rule for earning reward; the response-reinforcer contingency. The most important determinant of behavior (per Skinner). Different schedules should produce different patterns of behavior.

Continuous reinforcement (CRF)

Every response is reinforced. Not common in real life but important when first learning. The opposite is extinction.

Intermittent (partial) reinforcement

Behavior is reinforced only some of the times it occurs. More common in real life. Ratio = response-based; Interval = time-based.

What can a cumulative recorder show?

Total responses over time, response rate (slope: steep vs. shallow), patterns of behavior, and consistency (straight vs. curved/variable line).

Fixed Ratio (FR) schedule

Reinforcer after a fixed number of responses. FR30: every 30th response. Pattern: steady responding with a post-reinforcement pause (break-and-run). Example: factory worker paid per 30 items (response = assembling an item, reinforcer = pay).

Variable Ratio (VR) schedule

Reinforcer after an AVERAGE number of responses. VR30: on average every 30th response (30/20/40). Pattern: HIGH, steady rates, NO pauses. Example: slot machine (response = pull, reinforcer = payout).

Fixed Interval (FI) schedule

Reinforcer for the first response after a fixed amount of time. FI-30s: first response after 30 s. Pattern: FI scallop; low responding after reward, gradually accelerating, fastest near the end of the interval. Example: checking for mail delivered at the same time daily; self-administered morphine with a 60-min lockout.

Variable Interval (VI) schedule

Reinforcer for the first response after an AVERAGE amount of time. VI-30s: avg 30 s (30/20/40). Pattern: steady, moderate rate with no pauses. Example: checking your phone for texts (response = check, reinforcer = new message).

FI scallop: describe it

A scalloped cumulative-record pattern: low responding right after reinforcement, then responding gradually speeds up, reaching the highest rate near the end of the interval when reward becomes available.

Post-reinforcement pause (PRP)

The pause after each reinforcer on a fixed ratio schedule (break-and-run pattern). Implication: it is easier to continue working once you begin; just do a little.

Why no pauses on VR and VI schedules?

You can't predict how many more responses (VR) or how much more time (VI) before the reward, so responding is steady and consistent.

Ratio vs. interval: which produces a higher response rate? Why?

Ratio > interval. Ratio schedules are feedback functions where faster responding earns more reward (more responding = more reward). On interval schedules, response rate is irrelevant to getting r+ any sooner, so there's no benefit to responding faster.

As the response requirement goes up on a ratio schedule (no ratio strain), what happens to response rate? Why?

Response rate does not decrease; running rate stays high or increases (faster responding gets the reinforcer sooner), while the post-reinforcement pause lengthens because the next reward is farther away. Willingness to work goes up (to a certain point).

Ratio strain

Responding decreases/breaks down when the ratio requirement gets too high (schedule too lean). There is a limit to what is reasonable to expect; past it, response rate drops.

DRH (Differential Reinforcement of High Rates)

Reinforces a HIGH rate of behavior. Example: reward for completing 20 problems in 10 minutes.

DRL (Differential Reinforcement of Low Rates)

Reinforces a LOW rate of behavior. Example: reward for interrupting fewer times during class.

DRO (Differential Reinforcement of Other Behavior)

Reinforces the ABSENCE of a target behavior for a set time. Example: reward for 10 minutes without interrupting.

Skinner's view of thinking

Thinking is covert behavior, but it is still behavior and subject to reinforcement contingencies; not a special process inaccessible to science.