Operant Conditioning: Principles, Consequences, and Schedules
Operant Conditioning - Basics
Definition: Operant responses are behaviors emitted by an organism (rather than being initially elicited by specific stimuli), representing voluntary patterns that the organism "grows out into the environment." They apply to a wider range of behaviors compared to classical conditioning's elicited, innate, or automatic responses.
Consequences: Changes in the situation that follow the production of an operant response.
Effect on Future Behavior: These consequences then influence the probability of the response being produced again in similar circumstances in the future. If no change occurs in the situation or the likelihood of the behavior, no operant learning has taken place.
Four Types of Consequences
Consequences are primarily identified by observing how the response changes in the future (i.e., its probability of occurring), not by intention or subjective evaluation (e.g., "good" or "bad").
Reinforcement: A consequence that increases the probability of a response occurring in the future.
Punishment: A consequence that decreases the probability of a response occurring in the future.
Note: Reinforcement and Punishment are mutually exclusive; they can never be the same thing, as one increases and the other decreases behavior probability.
Positive: Indicates the addition of something into the situation that was not present prior to the response (i.e., ).
Negative: Indicates the removal of something from the situation following the response (i.e., ).
Note: "Positive" and "Negative" in operant conditioning are mathematical concepts (addition/subtraction), not evaluative terms (good/bad or liked/disliked).
Examples of Consequences
Positive Reinforcement:
Description: Adding something desirable to increase a behavior.
Example 1: A child hugs their mom and compliments her cooking ("You're the best baker ever!") after coming home. Mom, who has made cookies, gives the child a cookie. This makes the child more likely to compliment Mom's cooking in the future.
Example 2: A student studies diligently and prepares well for a test (the behavior) and subsequently receives a compliment from their mother for their performance (consequence). The student then becomes more likely to study diligently for future tests. (Note: Getting a good grade is usually a consequence, not the behavior itself).
Negative Reinforcement:
Description: Removing something undesirable to increase a behavior.
Example 1: During a class trip to the Franklin Institute, children behave well on the bus. The teacher responds by not assigning any homework for the rest of the week. This makes the children more likely to behave well in public in the future.
Example 2: A person experiences a headache (undesirable situation). They take an analgesic (behavior), and the pain goes away (removal of undesirable stimulus). The person becomes more likely to take an analgesic when experiencing a headache in the future.
Example 3: A person is at the beach under strong sun (undesirable situation). They put on sunscreen (behavior), and do not get a sunburn (removal of undesirable outcome). The person becomes more likely to apply sunscreen when in the sun in the future.
Positive Punishment:
Description: Adding something undesirable to decrease a behavior.
Example 1: A child calls their father "Dirtball." The father slaps the child across the mouth. The child does not do it again (i.e., the behavior decreases).
Example 2: A child is sitting with their feet on the car seat (behavior). The father takes the child to the police department (addition of an undesirable experience). The child becomes less likely to put their feet on the car seat in the future.
Example 3: Students in a class misbehave (behavior). The substitute teacher assigns them to write out an entire page of the dictionary (addition of an undesirable task). The students become less likely to misbehave in that class.
Negative Punishment (Omission Training):
Description: Removing something desirable (or a privilege) to decrease a behavior. It is important that the target behavior is actively occurring; one cannot punish a behavior that is already at zero (e.g., cannot punish "laziness" if the child is not actively doing something that could be reduced).
Example 1: A teenager violates curfew. They lose driving privileges for one month. The teen does not violate curfew as often (i.e., the behavior decreases).
Example 2: In animal training, an animal might be receiving food periodically. If the animal performs a particular unwanted response, the reinforcement (food access) is canceled for a period, leading to a decrease in that unwanted response.
Extinction in Operant Conditioning
Definition: Extinction occurs when a previously reinforced operant response is no longer followed by reinforcement. This is distinct from negative punishment, where a desirable item or privilege is contingently removed following a behavior to decrease it.
History: It's specific to circumstances where there has been a previous history of reinforcement for the behavior (e.g., social approval, a reward, or removal of something undesirable).
Effect: The likelihood of the response decreases, and it may eventually be eliminated. For extinction to be considered successful, the behavior's likelihood must be decreasing over time.
Example: If a child's tantrum behavior was previously reinforced by parental attention (even negative attention like yelling), an extinction approach would involve ignoring the tantrum (no longer providing attention). If the attention was the reinforcer, the tantrum behavior would likely decrease over time.
Spontaneous Recovery: Even after a behavior is seemingly extinguished, it can sometimes reappear. This indicates that the original association was weakened, not entirely undone, and can resurface especially if the individual encounters the stimulus situation again after a break.
Influence of Schedule: The schedule of reinforcement originally used can significantly influence the rate and pattern of extinction. Behaviors reinforced on variable schedules are generally more resistant to extinction compared to those on fixed or continuous schedules.
Schedules of Reinforcement
Schedules dictate when and how often reinforcement is provided after an operant response. These schedules differ along two main dimensions.
Dimension One: Basis for Reinforcement
Ratio Schedule: Reinforcement is contingent on the number of responses produced. The frequency of the response is what determines when reinforcement is delivered.
Interval Schedule: Reinforcement is contingent on the timing of a response after a specific period. The interval is a time during which reinforcement is not available; the first response after this interval elapses is reinforced. During the interval, responses do not lead to reinforcement.
Cost Perspective: Interval schedules can limit the overall amount of reinforcement an organism receives over time, making them potentially more cost-effective in some settings compared to ratio schedules, where continuous responding can lead to continuous reinforcement.
Dimension Two: Consistency of Conditions
Fixed Schedule: The conditions (number of responses or time interval) for reinforcement remain constant throughout the training process. These schedules are very predictable.
Variable Schedule: The conditions for reinforcement change unpredictably throughout the training process. The exact number of responses or time interval varies, usually around an average. Access to reinforcement is much less predictable.
Four Specific Schedules of Reinforcement
Fixed Ratio (FR) Schedule:
Description: The same, fixed number of responses is required to gain access to reinforcement. The number remains constant throughout training.
Example: A rat must press a lever times () to receive one food pellet. A vending machine operates on an schedule, where one response (pressing a button after inserting money) yields one reinforcement.
Response Pattern: Typically characterized by a high rate of responding followed by a post-reinforcement pause (a brief break after receiving reinforcement), especially with high response requirements. This creates a "jerky" response pattern.
Special Case: is known as continuous reinforcement, where every single response is reinforced. While efficient for initial learning, continuously reinforced behaviors extinguish quickly once reinforcement is removed.
Variable Ratio (VR) Schedule:
Description: A different (and unpredictable) number of responses is required each time to gain access to reinforcement. The average number of responses may be known, but the exact number for any given reinforcement varies.
Example: Slot machines provide reinforcement (a jackpot) after an unpredictable number of button presses (e.g., presses, then , then ). The unpredictability makes individuals persist.
Response Pattern: Supports very high and steady rates of responding without significant pauses, and the pattern is smooth, not jerky. This is because the organism never knows when the next reinforcement will occur.
Resistance to Extinction: Variable schedules (including VR) are generally more resistant to extinction than fixed schedules because the uncertainty of reinforcement keeps the organism responding persistently, even if reinforcement is not immediately given (e.g., comparing persistence at a vending machine versus a slot machine).
Fixed Interval (FI) Schedule:
Description: The same, fixed amount of time must pass before a response will trigger access to reinforcement. Reinforcement is available only for the first response made after the fixed interval has elapsed.
Example: Baking cookies. Each batch takes minutes. Checking the oven before minutes will not yield warm cookies. Only the first check after the -minute interval has passed will be reinforced (with warm cookies).
Response Pattern: Often leads to a "scallop" pattern of responding: a very low rate of responding at the beginning of the interval, gradually increasing in frequency as the interval ends, with the highest frequency just before reinforcement becomes available. After reinforcement, the rate drops again.
Real-life Application: Student study habits before scheduled exams often follow a fixed-interval pattern, with little studying at the beginning of the semester and an intense increase as exam dates approach.
Variable Interval (VI) Schedule:
Description: A different (and unpredictable) amount of time must pass before a response triggers reinforcement. Reinforcement is available for the first response after each variable interval.
Example: Asking Dad on the phone to play catch. The time Dad becomes available to play varies unpredictably, so the child might ask at random intervals, knowing that eventually, an interval will pass where the request is reinforced.
Response Pattern: Supports a moderate and steady rate of responding. Similar to VR schedules, it produces a smooth response pattern because the time of the next reinforcement is unpredictable. However, the maximal level of response is not as high as with ratio schedules.
Problems with Punishment
While punishment can decrease unwanted behaviors, it often comes with significant negative consequences, especially coercive forms involving pain infliction. Furthermore, problems arise when intentions do not align with actual behavioral changes; if a behavior increases following an intended punishment, it was, by definition, actually reinforcing. Behavioral professionals generally do not endorse punishment strategies that entail pain infliction due to these side effects.
Intense Emotional Responses: Coercive punishment (especially involving pain) automatically triggers intense negative emotions such as anxiety, fear, frustration, or anger. This heightened emotional arousal can interfere with learning and rational decision-making abilities, as the individual becomes primarily focused on escaping or avoiding the punishment.
Aggression Toward Others: The anxiety, fear, and frustration induced by punishment can lead to increased irritability and defensiveness. This, in turn, may trigger aggressive interactions with others, often redirected toward less powerful or less threatening targets (e.g., a child hitting a sibling or a pet after being spanked).
Classical Conditioning: The punisher themselves might become a conditioned stimulus (CS) for fear. The individual may associate the punisher with pain or discomfort, leading to avoidance or fear responses even in their presence. This can damage social relationships, as respect can be confused with fear.
Observational Learning: Children, through observational learning, might emulate the aggressive behaviors they witness. If they observe powerful figures (like parents or teachers) using "hands-on" or aggressive methods of punishment, they may then apply similar aggressive behaviors with their peers or in other social contexts, perceiving it as an effective way to control others or express frustration.
Alternatives to Punishment
To avoid the problematic side effects of positive punishment, especially coercive forms, several alternative strategies are available that are generally more effective and healthier for behavioral change.
Negative Punishment Procedures (e.g., Timeout):
Description: Removing something desirable (or a privilege) to decrease a behavior. Time-out procedures are a common application.
Effective Application of Time-out:
Designated Location: There should be a specific, safe time-out location in the home with reduced opportunities for social interaction or access to preferred items (e.g., a bottom step of stairs, a separate spot in a room away from toys).
Clear Explanation: When placing a child in time-out, clearly state the specific behavior that led to the time-out.
No Interaction: Do not interact with the child during the time-out period.
Duration: The general rule is one minute per year of the child's age (e.g., a -year-old gets a -minute time-out, a -year-old gets a -minute time-out). Longer durations for young children are often counterproductive, leading to increased agitation and distress without improving learning.
Consistency: Time-out procedures must be used consistently to be effective. Inconsistent application (e.g., sometimes using time-out, other times smacking or yelling) decreases its impact.
Benefits: Rarely leads to problematic side effects seen with positive punishment.
Extinction Procedures:
Description: Removing the reinforcer that was previously maintaining a problematic behavior. This is effective for behaviors that were unintentionally reinforced (e.g., a child's annoying habit reinforced by parental attention).
Considerations: If the behavior was reinforced on a variable schedule, extinction might take more time and persistence. It's crucial to identify the actual reinforcer. It's also important to ensure that removing reinforcement for one behavior doesn't severely reduce overall reinforcement opportunities for the individual, which could lead to other problems.
Reinforcement of Alternative Behaviors:
Description: Actively reinforcing helpful, useful, or functional behaviors. By increasing the probability of desirable behaviors, individuals have less opportunity to engage in problematic ones.
Approach: Instead of solely telling individuals what not to do, it's essential to also guide and reinforce what to do. This proactive approach focuses on building positive behavioral repertoires.
Overall Benefits: These strategies lead to less problematic outcomes and promote healthier learning environments. Awareness of how these procedures work allows for deliberate and effective behavioral management.