Schedules of Reinforcement

Schedules of Reinforcement

Schedules of Reinforcement

• Reinforcement schedule: Rules for when a behavior is reinforced.

• Skinner & Ferster (1957): Studied different reinforcement schedules.

• Main finding: Different schedules produce different behavior patterns.

Schedule Effects

• Schedule effects: Changes in behavior caused by reinforcement schedules.

• Different schedules affect:

  • Response rate: How often a behavior occurs.

  • Response pattern: How behavior is spread over time.

• Example: Piecework pay may increase productivity compared to hourly pay.

How Learning Is Measured

How Learning Is Measured

• Acquisition: Learning a new behavior.

• Response rate: Changes in how frequently behavior occurs.

• Response pattern: Changes in when behavior occurs.

• Key idea: Learning doesn't always involve a new behavior.

Types of Reinforcement Schedules

Types of Reinforcement Schedules

• Simple schedules: One set of reinforcement rules.

• Compound schedules: Combine two or more simple schedules.

Continuous Reinforcement (CRF)

Continuous Reinforcement (CRF)

• CRF: Reinforcement after EVERY response.

• Purpose: Fastest way to teach new behaviors.

• FR 1 = CRF: Every response is reinforced.

• Example: Dog gets a treat every time it sits.

Fixed Ratio (FR)

Fixed Ratio (FR)

• FR: Reinforcement after a SET number of responses.

• FR 5: Reward after every 5 responses.

• Response rate: High and steady between rewards.

• Post-reinforcement pause: Brief break after reinforcement.

• Higher FR requirement → Longer pauses.

• Run rate: Response rate between pauses.

• Example: Paid for every 10 items produced.

Variable Ratio (VR)

Variable Ratio (VR)

• VR: Reinforcement after a CHANGING number of responses.

• VR 5: Average of 5 responses per reward.

• Response rate: High and steady.

• Pauses: Few or short.

• VR usually produces more responses than FR.

• Examples: Gambling, slot machines, sales commissions.

Fixed Interval (FI)

Fixed Interval (FI)

• FI: First response AFTER a SET time gets reinforced.

• FI 30: First response after 30 seconds gets rewarded.

• Response pattern: Slow after reward → faster near next reward.

• Scalloped curve: Response rate gradually increases.

• Post-reinforcement pauses: Common.

• Example: Checking the oven near the end of baking.

Variable Interval (VI)

Variable Interval (VI)

• VI: First response after a CHANGING time interval gets reinforced.

• VI 30: Average interval of 30 seconds.

• Response rate: Moderate, steady.

• Pauses: Few.

• Examples: Waiting for a bus, hunting prey.

Extinction

Extinction

• Extinction: Previously reinforced behavior stops receiving reinforcement.

• Effect: Behavior gradually decreases.

• Extinction burst: Temporary increase in responding when reinforcement first stops.

• Skinner (1938): Rats gradually stopped pressing levers when food was removed.

Factors Affecting Extinction

Factors Affecting Extinction

• More prior reinforcement → Slower extinction.

• More effort required → Faster extinction.

• Longer break between extinction sessions → Greater spontaneous recovery.

• Extinction does not always permanently erase learned behavior.

Fixed & Variable Duration

Fixed & Variable Duration

• Fixed Duration (FD): Reinforcement after continuous behavior for a SET time.

  • Example: Practicing piano for 30 minutes.

• Variable Duration (VD): Reinforcement after continuous behavior for CHANGING amounts of time.

  • Example: Practicing piano for varying lengths of time.

Noncontingent Reinforcement (NCR)

Noncontingent Reinforcement (NCR)

• NCR: Reinforcement occurs regardless of behavior.

• Fixed Time (FT): Reward after a SET time, regardless of behavior.

  • Example: Birthday gifts every year.

• Variable Time (VT): Reward after CHANGING time intervals, regardless of behavior.

  • Example: Flowers blooming at unpredictable times.

Key difference:

• FI / VI: Require a response for reinforcement.

• FT / VT: Do NOT require a response.

Progressive Ratio (PR)

Progressive Ratio (PR)

• PR: Number of responses required increases after reinforcement.

• Arithmetic progression: Add a fixed number (2, 4, 6, 8).

• Geometric progression: Multiply by a fixed number (2, 4, 8, 16).

• Break point: Point where responding stops or drops sharply.

Stretching the Ratio

Stretching the Ratio

• Stretching the ratio: Gradually increasing work required for reinforcement.

• Purpose: Build persistence while giving fewer rewards.

• Example: FR 1 → FR 3 → FR 5 → FR 10.

• Ratio strain: Behavior weakens or stops when requirements increase too quickly.

• Key rule: Increase requirements gradually.

Multiple vs. Mixed Schedules

Multiple vs. Mixed Schedules

• Multiple: Schedules alternate WITH signals.

• Mixed: Schedules alternate WITHOUT signals.

• Signals: Lights, sounds, or colors showing which schedule is active.

• Example: Red light = FI 10; yellow light = VR 10.

Chain vs. Tandem Schedules

Chain vs. Tandem Schedules

• Chain: Complete schedules in order; reward after the LAST schedule.

• Tandem: Same as chain, but WITHOUT signals between schedules.

• Key difference: Chain = signals; tandem = no signals.

• Conditioned reinforcers: Signals in chain schedules can become reinforcing.

Cooperative Schedules

Cooperative Schedules

• Cooperative: Reinforcement depends on the behavior of 2 or more individuals.

• Example: Two pigeons must complete a combined 20 pecks for food.

• Problem: Unequal effort; some individuals do more work than others.

• Solution: Require each individual to contribute.

Concurrent Schedules

Concurrent Schedules

• Concurrent: Two or more schedules available at the SAME time.

• Key feature: Individual can CHOOSE between schedules.

• Example: Choose between VR 50 and VR 20.

• Preference: Often choose the schedule requiring less work.

Partial Reinforcement Effect (PRE)

Partial Reinforcement Effect (PRE)

• PRE: Intermittently reinforced behaviors resist extinction longer than continuously reinforced behaviors.

• Intermittent reinforcement: Only some responses are rewarded.

• CRF: Every response is rewarded.

• Key finding: Less frequent reinforcement → Greater resistance to extinction.

• Mowrer & Jones (1945): Demonstrated PRE in rats.

Discrimination Hypothesis

Discrimination Hypothesis

• Idea: Extinction is harder to recognize after intermittent reinforcement.

• CRF: No reward clearly signals extinction.

• Intermittent: No reward is normal, making extinction harder to detect.

• Weakness: Research showed PRE even when extinction was easy to recognize.

Frustration Hypothesis (Amsel)

Frustration Hypothesis (Amsel)

• Idea: Lack of reinforcement causes frustration.

• Intermittent training: Animals learn to keep responding despite frustration.

• Extinction: Frustration becomes a cue to continue responding.

• Result: Greater resistance to extinction.

Sequential Hypothesis (Capaldi)

Sequential Hypothesis (Capaldi)

• Idea: Behavior depends on the sequence of rewarded and unrewarded responses.

• Intermittent: Unrewarded responses become signals to keep responding.

• CRF: Missing a reward does NOT signal continued responding.

• Result: Intermittently reinforced behaviors persist longer.

Frustration vs. Sequential Hypotheses

Frustration vs. Sequential Hypotheses

• Frustration: Emotional response to missing rewards.

• Sequential: Memory of previous unrewarded responses.

• Similarity: Both explain how nonreinforcement becomes a cue to keep responding.

Response Unit Hypothesis (Mowrer & Jones)

Response Unit Hypothesis (Mowrer & Jones)

• Idea: PRE may be an illusion caused by how responses are counted.

• Response unit: Number of responses required for reinforcement.

• FR 1: 1 response = 1 unit.

• FR 3: 3 responses = 1 unit.

• Finding: When counting response units instead of individual responses, PRE can disappear.

Matching Law

Matching Law

• Matching law: Behavior is divided based on available reinforcement.

• Richard Herrnstein (1961): Developed the matching law.

• Key idea: More reinforcement → More behavior.

• Concurrent schedules: Allow a choice between two or more reinforcement schedules.

Matching Law in Ratio Schedules

Matching Law in Ratio Schedules

• Ratio schedules: Based on number of responses.

• Animals usually choose the schedule offering more frequent reinforcement.

• Example: FR 30 vs. FR 40 → Prefer FR 30.

• Key idea: Choose the option requiring less work per reward.

Matching Law in Interval Schedules

Matching Law in Interval Schedules

• Interval schedules: Based on time.

• Animals divide their behavior based on relative reinforcement rates.

• Example: VI 10 vs. VI 30:

  • VI 10 → 75% of responses.

  • VI 30 → 25% of responses.

• Key idea: Time spent responding matches the proportion of available rewards.

Research & Applications

Research & Applications

• Herrnstein: Pigeons distributed responses according to reinforcement rates.

• William Baum (1974): Found matching behavior in wild pigeons.

• Examples: Animals foraging, choosing crops, basketball shot selection.

• Limitation: Matching law does not perfectly predict every choice.

Important Exam Concepts

Important Exam Concepts

• Matching law: Choices match available reinforcement.

• Ratio schedules: Usually favor the better-paying option.

• Interval schedules: Behavior is divided between options.

• Higher reinforcement rate → Greater proportion of behavior.

• Behavior depends on all available reinforcers, not just one.