Comprehensive Guide to Operant Conditioning: Principles, Schedules, and Aversive Control

Law of Effect

Thorndike's principle stating that responses followed by a satisfying consequence in the presence of a stimulus have their connection to that stimulus strengthened (stamped in), whereas responses followed by an annoying consequence have their connection weakened (stamped out).

Puzzle Box

An experimental apparatus devised by Edwin L. Thorndike in the 1890s in which a hungry cat had to trigger a latch or lever to escape and access food outside, showing learning occurs through gradual trial-and-error rather than sudden insight.

Discrete-Trial Procedure

An experimental arrangement in which a trial begins when the subject is presented with an opportunity to respond and terminates after the instrumental response is made, requiring the experimenter to remove the subject and initiate each subsequent trial.

Free-Operant Procedure

An experimental arrangement developed by B. F. Skinner in which the organism remains in the chamber throughout and is free to emit the response repeatedly at any time, allowing rate of responding to serve as the primary dependent measure.

Operant Chamber (Skinner Box)

A standardized experimental chamber containing a response mechanism (such as a lever or pecking key), a stimulus delivery system (such as a houselight), and a consequence dispenser (such as a food tray) used to study operant behavior.

Operant Conditioning

A form of learning in which the future probability or strength of a voluntary, emitted behavior is modified by the consequences that follow it.

Three-Term Contingency

The fundamental unit of operant analysis comprising an antecedent discriminative stimulus ($S^D$), a behavior or response ($R$), and a consequence ($S^R$ or $S^P$).

Discriminative Stimulus ($S^D$)

An antecedent stimulus that occurs before a behavior and signals that a particular consequence is currently available if that behavior is emitted.

Discriminative Stimulus for Extinction

An antecedent stimulus signaling that a previously available reinforcer is currently unavailable, setting the occasion for non-responding.

Positive Reinforcement

An operant contingency in which the presentation of an appetitive stimulus following a response increases the future probability of that response.

Negative Reinforcement

An operant contingency in which the removal or termination of an aversive stimulus following a response increases the future probability of that response.

Positive Punishment

An operant contingency in which the presentation of an aversive stimulus following a response decreases the future probability of that response.

Negative Punishment

An operant contingency in which the removal of an appetitive stimulus following a response decreases the future probability of that response; also known as response cost or omission training.

Operant Extinction

The weakening of a previously reinforced behavior resulting from the complete withdrawal or discontinuation of the maintaining reinforcer.

Primary Reinforcer

An event or stimulus that is innately reinforcing and requires no prior learning history to function as a reinforcer, such as food, water, or proper temperature.

Secondary (Conditioned) Reinforcer

A stimulus that acquires reinforcing properties through its pairing and association with other established reinforcers, such as a clicker or good grades.

Generalized Reinforcer

A type of secondary reinforcer that has been associated with multiple other primary and secondary reinforcers, making it durable across varying deprivation states (e.g., money, social attention, or tokens).

Intrinsic Reinforcement

Reinforcement that is directly provided by the mere act of performing the behavior itself, such as reading an engaging novel or rollerblading for enjoyment.

Extrinsic Reinforcement

Reinforcement provided by a consequence that is external to the behavior itself, such as working solely for money or studying only to earn an $A$.

Natural Reinforcer

A reinforcer that is an ordinary, typical consequence of the behavior in the environment in which it naturally occurs.

Contrived Reinforcer

An artificial reinforcer deliberately arranged by someone to modify or shape behavior, rather than being an intrinsic or typical feature of that setting.

Shaping

The creation of a novel operant behavior through the systematic reinforcement of successive approximations to the desired terminal behavior.

Schedule of Reinforcement

The specific rule spelling out exactly what a learner must do before a reinforcer is delivered.

Continuous Schedule of Reinforcement (CRF)

A schedule in which every single instance of the target response is reinforced; optimal for teaching and shaping new behaviors and identical to an $FR\ 1$ schedule.

Intermittent Schedule of Reinforcement

A schedule in which only some instances of the target response are reinforced, representing most natural circumstances.

Fixed Ratio (FR) Schedule

A schedule where reinforcement is contingent upon a fixed, predictable number of responses, characterized by high rates of responding punctuated by post-reinforcement pauses.

Post-Reinforcement Pause

A brief cessation of responding that occurs immediately following the delivery of a reinforcer on fixed schedules of reinforcement.

Ratio Strain

A disruption in responding characterized by burnout or pausing, caused by stretching the reinforcement ratio too quickly from a dense schedule to a lean schedule.

Variable Ratio (VR) Schedule

A schedule where reinforcement is delivered after an unpredictable number of responses that averages out to a set value, generating high, steady rates of responding with little or no post-reinforcement pause.

Fixed Interval (FI) Schedule

A schedule where reinforcement is provided for the first response made after a fixed, predictable amount of time has elapsed, producing a scalloped cumulative response pattern.

Variable Interval (VI) Schedule

A schedule where reinforcement is delivered for the first response made after an unpredictable amount of time that averages to a specific duration, producing a moderate, steady response rate without pauses.

Cumulative Records of Basic Schedules

Graphical records of cumulative responses plotted over time that illustrate the distinct response patterns generated by $FR$ (step-like), $VR$ (steep and steady), $FI$ (scalloped), and $VI$ (moderate and linear) schedules.

Fixed Duration (FD) Schedule

A schedule where reinforcement is contingent upon performing a behavior continuously throughout a fixed, predictable period of time.

Variable Duration (VD) Schedule

A schedule where reinforcement is contingent upon performing a behavior continuously throughout an unpredictable period of time that averages out to a specified value.

Differential Reinforcement of High Rates (DRH)

A schedule in which reinforcement is contingent on emitting at least a specified number of responses within a designated time window, reinforcing fast responding.

Differential Reinforcement of Low Rates (DRL)

A schedule in which reinforcement is contingent on waiting a specified minimum amount of time between successive responses, reinforcing slow responding.

Differential Reinforcement of Paced Responding (DRP)

A schedule in which reinforcement is contingent on emitting responses neither too fast nor too slow, reinforcing a steady, paced rate of performance.

Noncontingent Schedule

A schedule in which reinforcers are delivered independently of any behavior after a fixed ($FT$) or variable ($VT$) time period.

Superstitious Behavior

Behavior that is accidentally reinforced when a noncontingent reinforcer coincidentally follows its occurrence, creating an illusory contingency.

Conjunctive Schedule

A complex schedule in which reinforcement is delivered only after the requirements of two or more simple schedules have all been satisfied.

Adjusting Schedule

A schedule in which the response requirement changes systematically based on the learner's performance during the acquisition of the preceding reinforcer.

Chained Schedule

A complex schedule consisting of a sequence of two or more simple schedules, each signaled by its own discriminative stimulus ($S^D$), culminating in a terminal reinforcer upon completion of the final link.

Backward Chaining

A training procedure used to construct a behavioral chain by training the final link first and working systematically backward to the initial link.

Goal Gradient Effect

The increase in the strength, rate, and efficiency of responding that occurs as an organism moves closer to the terminal reinforcer in a behavioral chain.

Drive Reduction Theory

Hull's theory stating that an event is reinforcing to the extent that it is directly or indirectly associated with the reduction of a biological physiological drive.

Incentive Motivation

Motivation that is derived from properties of the reinforcer itself rather than from an internal physiological state of deprivation.

Premack Principle

The proposition that a high-probability behavior can be utilized as a reinforcer to strengthen a low-probability behavior.

Response Deprivation Hypothesis

The hypothesis stating that any behavior can function as a reinforcer provided that access to it is restricted below its preferred baseline level of occurrence.

Behavioral Bliss Point

The ideal distribution of activities that an organism freely chooses to maximize overall reinforcement when unconstrained by contingencies.

Escape Behavior

An operant response that terminates or stops an aversive stimulus that is already occurring.

Avoidance Behavior

An operant response that prevents an anticipated aversive stimulus from occurring before it starts.

Shuttle Avoidance Task

An apparatus and procedure where an animal shuttles back and forth across a barrier to terminate or prevent an electric shock signaled by a warning stimulus.

Two-Process Theory of Avoidance

Mowrer's theory stating that avoidance involves two stages: classical conditioning of fear to a warning stimulus ($CS$), followed by operant conditioning where the response is negatively reinforced by escaping the fear-eliciting $CS$.

Anxiety Conservation Hypothesis

The hypothesis proposing that avoidance responses occur so rapidly that the organism has minimal exposure to the conditioned aversive stimulus, preventing conditioned fear from fully extinguishing.

One-Process Theory of Avoidance

The theoretical view that avoidance behavior is maintained solely by an operant reduction in the overall rate or frequency of aversive stimulation, without requiring an internal state of fear reduction.

Stampfl's Phobia Analogue

An experimental model using a moving conveyor belt demonstrating that rats learn early avoidance responses to minimize exposure to a fear-associated compartment, illustrating why phobic avoidance persists indefinitely.

Exposure and Response Prevention (ERP)

A behavioral treatment for obsessive-compulsive disorder involving prolonged exposure to fear- or obsession-inducing stimuli while strictly preventing the performance of anxiety-reducing compulsive rituals.

Time-Out

A negative punishment procedure that involves the temporary loss of access to all positive reinforcers following the emission of an inappropriate behavior.

Response Cost

A negative punishment procedure that involves the removal of a specific, valued reinforcer following the occurrence of an inappropriate behavior.

Conditioned Suppression Theory of Punishment

Skinner's view that punishment does not weaken an operant association directly, but instead elicits an emotional reaction that temporarily suppresses the occurrence of ongoing behavior.

Avoidance Theory of Punishment

The proposition that punishment is an instance of avoidance conditioning, in which doing any behavior other than the punished response is negatively reinforced by avoiding the punisher.

Learned Helplessness

A decrement in learning ability and willingness to escape that results from prior, repeated exposure to inescapable and uncontrollable aversive events.

Experimental Neurosis (Masserman)

A laboratory-induced breakdown in behavior and onset of neurotic symptoms in animals caused by exposure to unpredictable, uncontrollable aversive events during basic activities like eating.