Comprehensive Guide to Operant Conditioning, Reinforcement Schedules, and Observational Learning

Principles of Operant Conditioning and Behavioral Shaping

• Operant conditioning governs intentional, goal-directed behaviors through environmental rewards and consequences, contrasting with involuntary reflexive actions.
  • Reinforcement triggers neurochemical responses, specifically releasing dopamine and serotonin in the brain, which signals the organism to repeat the preceding action.
• Behavioral shaping is the process of reinforcing successive approximations toward a target behavior through small, incremental, planned steps.
  • Example (Swimming/Diving Protocol):
  • Initial Phase: The trainer stands in the shallow end of a pool, catching the young subject (Angel) as she jumps in, praising her immediately, and placing her gently in the water.
  • Progressive Distance: The trainer steps into deeper water, catching her right as she hits the surface, providing continuous verbal praise ("You're so great") to stimulate dopamine and serotonin release.
  • Independence: The subject jumps in submerged, gets brought back up, and eventually learns to jump in and surface on her own.
  • Creative Efficiency: The subject develops an adaptive strategy of allowing herself to sink completely to the bottom of the pool and kicking off the floor to push herself back to the surface.
  • Head-First Transition: Initial attempts at diving result in an inverted U-shape hop. Reinforcement is delivered immediately upon execution.
  • Fine-Tuning: The subject is coached to lay out her body at a safe angle to prevent belly flops and avoid the unpleasant outcome of physical impact against the water.
  • Example (Canine Training - Skinner the Dog):
  • Paw Shake: The trainer physically grabs the dog's paw, pumps it manually, and pairs the movement with immediate reward ("shake food, shake food"). Over time, the verbal cue "shake" prompts the dog to raise its paw independently.
  • Roll Over: Requires establishing prerequisite behaviors (sit and lie down).
    • Lying Down: Achieved by reinforcing successive approximations (crouching down, then fully lying down).
    • Verbal Cues: Optimal canine training utilizes single-syllable commands (maximum two syllables); "over" is used instead of "rollover".
    • Execution: A treat is held at the dog's nose and moved across its body. Successive approximations (turning slightly, rolling onto back) are reinforced.
    • Student Demonstration: During a psychology lab demonstration, a student commanded "Down. Over.", and the dog executed the full sequence immediately despite weeks of incremental practice with the primary trainer.
  • Example (Laboratory Rat Protocols - Regina and Renata):
  • Subjects: Two rats assigned to ensure completion if one fails to survive the term.
  • Stress Reduction: New environments prompt grooming behaviors in rats to reduce anxiety and stress.
  • Deprivation Protocol: During the first two weeks, rats have free access to food and water to establish baseline mass. Water is then restricted until the rat reaches $85\%$ of its free-access body weight, ensuring water acts as an effective reinforcer.
  • Operant Chamber Mechanics: Pressing a lever activates a dispenser delivering water.
  • Shaping Sequence: Reinforcement (water click) is delivered when the rat looks at the lever $\rightarrow$ moves toward the lever wall $\rightarrow$ approaches the lever $\rightarrow$ touches the lever $\rightarrow$ depresses the lever. Manual control is then relinquished to the automated chamber.
  • Individual Temperament Variations:
    • Regina (Aggressive/Bold): On her first approach, she body-slammed the lever forcefully, received water, instantly formed the association, and operated the lever continuously.
    • Renata (Timid/Cautious): Gently rested her paws on the lever without depressing it. Because tapping was no longer reinforced, she failed to push the lever down, requiring temporary removal from the experiment to prevent dehydration before eventually catching up to Regina.

Classifications of Reinforcement and Schedules

• Definition of Reinforcement: Any stimulus or event following a behavior that increases the probability or strength of that behavior. Reinforcement can only be definitively identified retroactively by observing an increase in behavior.
• Positive Reinforcement: Strengthening a behavior by adding or presenting a desirable stimulus (e.g., food, water, chocolate). Chocolate is generally reinforcing, though exceptions exist (e.g., individuals with allergies).
• Negative Reinforcement: Strengthening a behavior by removing or terminating an unpleasant (aversive) stimulus.
  • Operant Grid Chamber: Operates via a metal grid floor wired to deliver electric current controlled by a rheostat (a variable resistor that adjusts electrical intensity, similar to a stereo volume knob or light dimmer switch).
  • Shock Protocol: The rheostat starts at zero and is incrementally increased until the rat displays an avoidance response (picking up paws, performing a distinct dance). Depressing the lever immediately deactivates the electric current, strengthening the lever-pressing behavior.
  • Misconception Warning: Between $40\%$ and $60\%$ of students incorrectly state that negative reinforcement weakens behavior. Regardless of the modifier (positive or negative), all reinforcement strengthens behavior.
• Primary vs. Conditioned Reinforcers:
  • Primary Reinforcers: Naturally and biologically reinforcing stimuli (e.g., food, water, play).
  • Play Behavior: Play can reinforce other behaviors (e.g., requiring a child to clean their room before being allowed to play).
  • Conditioned (Secondary) Reinforcers: Neutral stimuli that acquire reinforcing properties through repeated association with primary reinforcers (e.g., clicker training).
  • Clicker Training Protocol: Repeatedly pair the click sound with food delivery ("click food") until the clicker alone becomes reinforcing. Training must occur prior to regular feeding while the animal is hungry; feeding an animal prior to training destroys motivation.
  • Human Application: Money is the most potent secondary reinforcer for humans because it can be exchanged for primary and desired reinforcers; non-human primates (e.g., chimpanzees) do not value money.
• Negative Reinforcement Protocols:
  • Escape Behavior: The aversive stimulus is currently active (e.g., floor is electrified), and the response terminates it. The organism cannot prevent the shock, only escape it.
  • Avoidance Behavior: A conditioning cue (e.g., a light) signals an impending aversive stimulus, allowing the organism to respond before the stimulus begins.
  • Laboratory Observation: Rats in avoidance protocols remain stationary at the lever with a paw resting on it, staring at the cue light to press it instantly.
  • Extinction Resistance: Behaviors learned under avoidance protocols are extremely difficult to extinguish because the organism responds instantly to the cue, preventing them from discovering that the shock has been disabled.
  • Human Application: Seatbelt warning systems apply negative reinforcement. Early systems used loud buzzers that drivers bypassed by clipping wires under the seat; automobile manufacturers subsequently integrated seatbelt wiring into the ignition system so clipping the wire prevents the engine from starting. Modern systems utilize persistent chime bells that cease once the belt is buckled.

Reinforcement Schedules and Behavioral Response Patterns

• Continuous Reinforcement: Every target response is reinforced. Essential during the initial acquisition phase of learning.
• Intermittent (Partial) Reinforcement Schedules:
  • Fixed Ratio (FR):
  • Reinforcement occurs after a set, predictable number of responses (e.g., FR-5 delivers reinforcement on every 5th response).
  • Response Pattern: Produces the fastest overall response rate, characterized by rapid bursts of behavior interrupted by brief post-reinforcement pauses.
  • Industrial Application: 19th and 20th-century piecemeal pay structures (e.g., paying workers for every 10th telephone switch assembled). While this increased output, it led to high defect rates due to rushed assembly, prompting companies to abandon the system for quality control.
  • Variable Ratio (VR):
  • Reinforcement occurs after an unpredictable, varying number of responses centered around an average probability.
  • Response Pattern: High, steady response rates extremely resistant to extinction.
  • Human Behavior: Humans demonstrate extreme tolerance for low VR probabilities, such as purchasing lottery tickets with odds of $1 \text{ in } 250 \times 10^6$ (described colloquially as a "tax on the mathematically illiterate").
  • Gambling Mechanics: Slot machines ("one-armed bandits"), roulette, craps, and blackjack operate on VR schedules. Each pull or spin is an independent event with fixed odds (e.g., $1 \text{ in } 10\,000$).
  • Poker Distinction: Combines chance with psychological manipulation; winning relies on convincing opponents to fold through presentation and reading cues rather than hand strength alone.
  • Fixed Interval (FI):
  • Reinforcement is delivered for the first response made after a fixed, elapsed time period (e.g., FI-15 seconds).
  • Response Pattern: Produces a "scalloped" response pattern—zero responses immediately following reinforcement, with acceleration in rate and intensity as the end of the time interval approaches.
  • Human Application: Professional athletes performing in a "contract year" (the final year of a multi-year contract) dramatically increase performance metrics (e.g., doubling sack rates in football) to secure favorable contracts.
  • Variable Interval (VI):
  • Reinforcement is delivered for the first response following unpredictable, varying intervals of time (e.g., intervals of $1\,\text{s}$, $20\,\text{s}$, $5\,\text{s}$, $5\,\text{s}$, $2\,\text{s}$, $15\,\text{s}$).
  • Response Pattern: Produces a slow, steady, metronome-like response rate with high resistance to extinction.
• Strategic Implementation: When establishing a new behavior, utilize continuous reinforcement during acquisition, then transition to a variable ratio schedule to maximize resistance to extinction.

Punishment, Environmental Cues, and Superstitious Behavior

• Definition of Punishment: Any consequence that weakens a behavior, decreasing its future probability.
  • Positive Punishment: Decreasing behavior by introducing an aversive stimulus.
  • Historical Example: Raising children using a "switch bush" (supple branches used as whips). Children instructed to retrieve their own switch who brought back an insufficient branch were punished with a larger switch.
  • Negative Punishment: Decreasing behavior by removing a desirable stimulus or privilege (e.g., revoking driving privileges for a month, revoking access to landline phones, or confiscating smartphones which induces severe anxiety).
• Drawbacks of Punishment:
  • B.F. Skinner strongly opposed punishment as a behavioral control method.
  • Punishment produces temporary behavioral suppression, but the behavior tends to recover over time even with continued punishment.
  • Teaches organisms (particularly children) to suppress the behavior only when being actively observed rather than eliminating the behavior.
  • Damages the interpersonal relationship between the trainer/parent and the organism.
• Environmental Cues (Discriminative Stimuli):
  • Contextual environments signal which behaviors will be reinforced or punished.
  • Examples: Rats only press levers inside the operant chamber. Individuals refrain from using profane language in academic environments while frequently using it during highway traffic commutes.
  • Discrimination vs. Generalization:
  • Discrimination: Performing a behavior only in environments where specific contextual cues signal reinforcement.
  • Generalization: Performing a learned behavior across all environments without distinguishing contextual appropriateness.
• Superstitious Behavior:
  • Formed through accidental, non-causal pairings of a behavior with a reinforcing outcome.
  • Canine Example ("Magic Back Door"): A dog kept in a kennel due to anxiety-driven chewing (which destroyed a coffee table leg) was released into the backyard after 9 to 10 hours. Relief from biological elimination acted as negative reinforcement, while food prepared during that time acted as positive reinforcement upon re-entry. The dog erroneously concluded that going out the back door generated food, leading to persistent demands to go outside and run laps whenever denied treats.
  • Sports Examples:
  • Baseball players develop rigid pre-at-bat rituals due to the low baseline probability of success. Joe Morgan (Cincinnati Reds second baseman, 1970s) tapped his arm twice before every pitch after an early accidental pairing resulted in a hit.
  • Fans performing rituals (e.g., drinking from the exact same glass during an undefeated season, or wearing an unwashed Gamecock shirt during back-to-back college baseball national championships).

Observational Learning, Modeling, and Media Impacts

• Definition: Observational learning (modeling) involves acquiring behaviors by watching the actions of others; it requires an observer and a model.
  • Species Scope: Occurs across vertebrates (e.g., a spaniel puppy named Brittany learning the "sit" command purely by observing its mother without direct instruction).
• Albert Bandura's Bobo Doll Experiment:
  • Protocol: Children observed an adult female model physically beating and verbally abusing an inflatable Bobo doll. A control group observed non-aggressive behavior. Children were then placed individually in a room containing various toys (kitchen sets, tool sets, tetherball, fake oven, baby dolls, Bobo doll).
  • Findings:
  • Children exposed to the aggressive model displayed significantly higher rates of physical aggression toward the Bobo doll.
  • Novel Aggression: Children generated creative violent acts not demonstrated by the model, such as hitting Bobo with a hammer, pointing toy guns at its face, timing tetherball swings to strike the doll, and using a baby doll by its ankles as a weapon to beat Bobo.
  • Language: Children adopted and expanded upon the aggressive verbal phrases used by the model.
  • Media Policy and Societal Changes:
  • Establishment of "Prime Time" television broadcasting (restricting violent content before 9:00 PM when children were expected to be asleep under curfews).
  • Media watchdog groups began tallying violent acts on television (e.g., Looney Tunes ranking high due to slapstick violence against Wile E. Coyote).
  • Cancellation of popular violent network shows (e.g., CBS canceled The Wild Wild West, starring Robert Conrad who suffered concussions and broken ribs doing stunts, alongside a co-star who suffered a heart attack) to preempt federal legislation.
• Neurobiology of Modeling (Mirror Neurons):
  • Located primarily in the motor and premotor cortex, mirror neurons fire both when an individual executes an action and when they observe another executing that same action (e.g., watching Tiger Woods play golf activates motor circuits in the observer).
  • Development: Young children lack neurological inhibition, causing them to physically imitate observed actions (e.g., sticking out their tongue or grinning back).
  • Theory of Mind and Empathy: Mirror neurons support Piaget's theory of mind, spatial mental rotation, and empathetic feeling (firing similarly when observing another person in pain, such as watching violent scenes in The Passion of the Christ).
  • Atypical Mirror Neuron Activity: Observed consistently in individuals on the autism spectrum.
  • Clinical Psychopathy: Serial killers lack automatic empathetic mirror responses when watching others experience pain, but exhibit normal empathetic neural activity when explicitly instructed to imagine themselves experiencing the pain.
• Vicarious Reinforcement vs. Vicarious Punishment:
  • Vicarious Punishment: Observing a model receive punishment for a behavior strongly suppresses imitation of that behavior in observers.
  • Vicarious Reinforcement: Observing a model receive rewards (e.g., candy) for aggression produces a minor increase in imitation.
• Historical Media Codes:
  • Hays Code / Motion Picture Production Code (1933): Required all screen villains to face retribution or legal punishment.
  • Controversy: The release of It's a Wonderful Life faced regulatory opposition because the antagonist, Mr. Potter, stole $15,000 from Uncle Billy without facing legal consequences or physical punishment within the film.

Questions & Discussion

• Student Attendance Credit:
  • Question/Prompt: A student asked whether to email the instructor to be marked present after arriving late.
  • Response: Students arriving after roll call must send an email immediately following class, as attendance details are forgotten within 10 feet or 10 seconds of leaving the lecture room.
• Georgia Lottery Proceeds:
  • Question/Prompt: A student noted that playing the lottery in Georgia funds educational programs, functioning as a civic donation.
  • Response: While proceeds support education, direct charitable donations provide greater financial return by offering tax write-off benefits that lottery purchases do not.
• Marvel Cinematic Universe Villains:
  • Question/Prompt: A student expressed appreciation for unpunished, highly powerful media villains (e.g., the Void in Marvel content), desiring longer villain story arcs.
  • Response: While observers enjoy watching intense villain storylines without imitating illegal behavior due to fear of incarceration ("I like my freedom"), vicarious punishment remains the established psychological mechanism for behavioral suppression in broader society.