9 & 10 - instrum cond

Instrumental behavior initially controlled by

Consequences (outcomes)

Basic instrumental relation

S → R → O (stimulus → response → outcome)

Law of Effect

Behavior strengthened by reward, weakened without it

Reinforcer

Outcome that affects behavior

Reinforcer properties

Quality and quantity

Orbitofrontal cortex (OFC)

Encodes reward value and subjective preference

OFC function

Economic decision-making and reward evaluation

OFC neurons

Fire in anticipation of reward

Dopamine neurons

Encode reward prediction and value

Dopamine firing

Increases for better or unexpected rewards

Positive contrast

Small → large reward → ↑ responding

Negative contrast

Large → small reward → ↓ responding

Contrast effect example

Rats run faster after reward increase, slower after decrease

Temporal relation

Time between response and reinforcement

Temporal contiguity

Reinforcement occurs immediately after response

Best timing for reinforcement

Immediate reinforcement

Effect of delayed reinforcement

Weakens learning

Problem with delay

Other behaviors interfere (third variables)

Third variable issue

Reinforcer may seem unrelated to response

Neural basis of delay

OFC, ventral striatum, dopamine

Reward anticipation neurons

Fire while waiting for reward

Dopamine and delay

Stronger response to immediate rewards

Solution to delay

Secondary (conditioned) reinforcement

Secondary reinforcement example

“Good boy” or clicker training

Purpose of secondary reinforcement

Marks correct behavior immediately

Response-reinforcer contingency

Degree response causes outcome

High contingency

Strong learning

Learned helplessness

Learning that outcomes are uncontrollable

Helplessness experiment result

Group Y stopped trying

Effect of learned helplessness

Reduced effort and motivation

Activity deficit

Stopping behavior due to perceived lack of control

Instrumental conditioning

Behavior shaped by consequences (cause → effect)

Operant conditioning

Another name for instrumental conditioning

Instrumental vs classical

Instrumental = voluntary; Classical = automatic

Elicited behaviors

Habituation, sensitization, classical conditioning

Thorndike

Key theorist of instrumental conditioning

Law of Effect

Reward ↑ behavior; no reward ↓ behavior

S-R learning

Stimulus–response association

Reinforcement

Increases behavior

Positive contingency

Behavior → stimulus appears

Negative contingency

Behavior → stimulus removed

Appetitive stimulus

Pleasant outcome

Aversive stimulus

Unpleasant outcome

Discrete trial

Separate trials with start/end

Discrete task examples

Maze, runway, T-maze

Free operant

Continuous responding

Operant behavior defined by

Effect on environment

Operant examples

Lever press, nose poke

Main dependent variable

Response rate

Other dependent variables

Total responses, latency

Magazine training

Food cup = reward

Shaping

Reinforce small steps toward behavior

Positive reinforcement

Add reward → ↑ behavior

Negative reinforcement

Remove aversive → ↑ behavior

Punishment

↓ behavior

Omission training

No reward if behavior → ↓ behavior

Delay discounting

Delay reduces reward value

Effort discounting

Effort reduces reward value

OFC lesion effect

Prefers immediate rewards

Instrumental conditioning core

Behavior → outcome

Reinforcement core

Increases behavior

Punishment core

Decreases behavior

Shaping core

Build behavior step-by-step