Chapter 6 - Classical Conditioning, Operant Conditioning, and Observational Learning

Two Main Behavioral (Conditioning) Theories
  • Two Main Behavioral Scenarios:

    • A child throws a tantrum in order to manipulate his/her parents to buy him/her the toy.

    • Q#1: How did the child learn to like the toy so much? (Classical Conditioning Theory explains that).

    • Q#2: How did the child learn that throwing tantrum will get him/her the toy? (Operant Conditioning Theory explains that).

    • A child pretends to be sick in order to get out of doing house chores.

    • Q#1: How did the child learn to dislike house chores? (Classical Conditioning Theory explains that).

    • Q#2: How did the child learn that pretending to be sick would get him/her out of doing house chores? (Operant Conditioning Theory explains that).

I. Classical Conditioning Theory
  • Also called respondent conditioning theory or Pavlovian conditioning Theory.

  • Explains how organisms develop involuntary reactions (mostly emotional [like, dislike, fear, craving, etc.] or physiological in nature) toward different environmental stimuli.

A. Key Concepts

  1. Innate Reflexes:

    • All organisms are born with a set of reflexes. Reflexes are fixed reactions to highly specific stimuli (e.g.: a hungry dog will salivate to smell of meat).

  2. Stimulus and Response Designations:

    • Any stimulus that naturally elicits a reflex is called an unconditioned stimulus (UCS) for that reflex. The reflex itself, when elicited by the UCS, is called an unconditioned response (UCR) to that stimulus.

      • e.g.: For a hungry dog, smell of meat = UCS; salivation to the smell of meat = UCR.

    • All other stimuli that do not naturally elicit that reflex are called neutral stimuli (NS) with respect to that reflex.

      • e.g.: A hungry dog will not naturally salivate to the sound of a bell. So, the bell sound is an NS with respect to the salivation response.

  3. Definition of Learning:

    • Learning is defined as the generalization of a reflex (UCR) from a stimulus that naturally elicits that reflex (UCS) to other stimuli that do not naturally elicit that reflex (NS).

      • e.g.: Learning is said to have taken place if somehow we 'make' a hungry dog salivate to the sound of a bell.

  4. The Acquisition Process and S-S Associations:

    • Learning can be achieved through the process of acquisition: The repeated presentation of an NS and a UCS to the organism at about the same time.

    • Through this process, a neurological link between the NS and the UCS, called stimulus-stimulus association or S-S association, is formed within the nervous system of the organism.

    • This association allows the NS to act (neurologically speaking) as if it were the UCS to trigger the reflex (UCR).

    • Neurological Mechanism:

      • Smell Centre: Centre that Processes Smell of food → Salivation reflex, hardwired. Activation of 'smell' centre always triggers salivation.

      • Bell Centre: The centre that processes sound of bell: Before stimulus pairing, there is no neurological association with the centre that processes smell of food. Does not lead to salivation.

      • After Pairing: Through the process of acquisition, S-S association is formed between the centre that processes sound of the bell and the centre that processes smell of food. Now, through a chain reaction, sound of bell will elicit salivation (Hears bell →→ Salivates).

  5. Conditioned Stimulus and Response:

    • Once an S-S association is formed, the NS, which now has the power to elicit the reflex, is renamed as the conditioned stimulus (CS).

    • The reflex, when it is elicited by the CS alone (i.e.: without the presence of the UCS), is called the conditioned response (CR).

B. The Process of Acquisition: How to Do So Correctly?

  1. Must be Consistent:

    • During training/learning, the UCS should always be paired with the NS – and only the NS.

      • e.g.: Always precedes food (UCS) with the SAME bell sound (NS) in order to teach the dog to salivate to the sound of the bell.

  2. Make sure the Intensity of the UCS is Strong:

    • All things being equal, the stronger the UCS during stimulus pairing, the faster the formation of S-S association, and the stronger the S-S association.

      • e.g.: Make sure the dog is very hungry, so that the food (UCS) has great power to elicit salivation, before pairing the bell (NS) with it.

  3. Make sure the NS is novel to the organism:

    • If the organism is already familiar with the NS, it will be difficult for the organism to develop a different reaction to it through classical conditioning. This phenomenon is called latent inhibition.

      • e.g.: If a dog has heard a bell sound many times before in different situations, it will be difficult to teach the dog to salivate to the sound of that bell.

C. Extinction and Spontaneous Recovery

  • Extinction:

    • By itself, most S-S associations do not maintain their strength forever.

    • If the CS is repeatedly presented to the organism without the UCS, it will eventually lose its power to elicit the CR. This is called the process of extinction.

  • Spontaneous Recovery:

    • On the other hand, an S-S association can rarely be completely unlearned.

    • Even after extinction, the neurological link remains dormant within our nervous system.

    • Under the right condition, an extinct S-S association may regain its strength and the CS may, once again, have the power to temporarily elicit the CR. This is called spontaneous recovery.

II. Operant Conditioning Theory
  • Operant conditioning theory explains how the environment controls our seemingly voluntary behaviors through reward (reinforcement) and punishment.

A. Thorndike's Law of Effect and Skinner's Elaboration

  • Operant conditioning is based on Thorndike's Law of Effect, which simply states that behaviors that are followed by "satisfying" consequences will increase in frequency and intensity over time.

  • Skinner further elaborated/expanded on the Law of Effect and stated that behaviors that are either NOT followed by any “satisfying” consequences, or are followed by "dissatisfying" consequences, will reduce in frequency and intensity over time, until they eventually disappear.

  • Based on these two simple concepts, Skinner proposed four ways of controlling behaviors.

B. Four Ways of Controlling Behaviors

  • Whether a consequence constitutes a reinforcement or a punishment strictly depends on how it impacts the target behavior. If the consequence increases the frequency of occurrence of the target behavior, it should be considered as a reinforcement for that behavior, no matter what the intent of the consequence is. The reverse is also true.

  • The Four Methods:

    1. Positive reinforcement: Give something pleasurable to the organism after the organism executes a behavior.

    2. Negative reinforcement: Take away something unpleasant from the organism’s environment after the organism executes a behavior.

    3. Positive (Aversive) punishment: Administer something unpleasant to the organism after the organism executes a behavior.

    4. Negative punishment (response cost): Take away something valued by the organism after the organism executes a behavior.

  • Behaviors that are reinforced should always increase in frequency and intensity over time; behaviors that are punished should always decrease in frequency and intensity over time and eventually disappear.

  • Effectiveness Comparison:

    1. Reinforcement (both positive and negative) is generally more effective than punishment (both types). Punishment is problematic for 4 reasons:

    • a. Punishment only suppresses undesirable behaviors, but does not teach new, more adaptive behaviors (whereas reinforcement can).

    • b. Punishment can create anger, anxiety, and fear, all of which inhibit learning.

    • c. Punishment encourages subversive behaviors: People focus on avoiding punishment rather than on doing the right thing.

    • d. Punishment, especially physical punishment, teaches children that aggression is acceptable.

    1. Positive reinforcement is more effective than negative reinforcement because negative reinforcement involves first subjecting the organism to some type of unpleasant stimulus, which can also create anger, anxiety, and fear.

    2. Therefore, positive reinforcement should be used as much as possible when trying to control behavior.

C. Three Types of Positive Reinforcer

  1. Unconditioned / Primary Reinforcers:

    • Unconditioned (primary) reinforcers are stimuli that are naturally reinforcing because they satisfy biological needs.

    • The use of primary reinforcers to control behavior has 2 limitations:

      • a. They are effective only if the organism is deprived of, or has not had enough of, the reinforcer.

      • b. Unconditioned reinforcers must be consumed in order for them to be reinforcing, and so it works only if the situation allows for the consumption of the reinforcer.

  2. Conditioned / Secondary / Generalized Reinforcers:

    • Conditioned (secondary) reinforcers are stimuli which have acquired their reinforcing properties by having been paired with some unconditioned (primary) reinforcers or other conditioned reinforcers through the process of classical conditioning.

      • e.g.: First pair a bell with food to get a dog to associate the bell sound with food, then use the bell to reinforce the behavior you want.

    • A generalized reinforcer is a secondary reinforcer that has been paired with a variety of primary reinforcers. As a rule, the more generalized a secondary reinforcer, the more powerful it is.

      • Examples of generalized reinforcer: Money; praise by loving parents.

  3. Preferred Activity as a Reinforcer (Premack Principle):

    • Research by David Premack (1962) shows that preferred activities can be used as effective reinforcers of less preferred activities. This is known as the Premack Principle.

      • e.g.: Allowing a child to play wii (preferred activity) contingent on his/her finishing homework (less preferred activity).

  • Contingency Rule:

    • Regardless of the type of reinforcer used, contingency is the key to success: The reinforcer is given to the organism immediately, consistently, and only after the organism has executed the desired behavior.

D. Shaping

  • Shaping is a technique used to teach organisms behaviors that they have never emitted because the behaviors are not part of the organisms’ behavioral repertoire and are too complex for the organisms to execute without some kind of learning.

  • Key Steps in Shaping:

    1. Provide an environment in which behavioral choices / distractions are limited.

    2. Divide the target behavior into different components in such a way that:

    • a. The first component should be fairly easy for the organism to learn, and,

    • b. Successful learning of previous components should facilitate the learning of the subsequent component.

    1. Chaining: Begin by reinforcing successful execution of the first component, and then successively changing the criteria of reinforcement so that the organism is reinforced only if it executes the next component, etc., until the entire behavior is learned. (wtf this all mean)

E. Schedules of Reinforcement

  1. Continuous Reinforcement Schedule:

    • The organism is reinforced after executing the desired response every time.

    • It is the schedule of choice when trying to teach the organism a new behavior.

  2. Intermittent (Partial) Reinforcement Schedule:

    • With intermittent reinforcement schedule, not every desired response is reinforced.

    • Intermittent schedule is best for maintaining a desired response.

    • The transition from continuous reset

      • Reinforcement to partial reinforcement is called fading.

  • Four Types of Intermittent Reinforcement Schedules:

    • Fixed Ratio Schedule (FR-n):

      • With Fixed Ratio Schedule, the organism is reinforced after a fixed number of responses have been emitted.

        • The requirement to get a reward never changes—it requires a set quantity of work every single time

        • The rate of reinforcement depends entirely on speed. The faster the action is repeated, the faster the reward arrives

      • A behavior controlled by fixed-ratio schedules is typically bi-valued. That is, the organism tends to pause after completing the ratio and receiving the reinforcer.

        • Bi-valued" means the behavior flips between two distinct extremes: zero activity (a pause) or high-speed activity (rapid responses)

        • Immediately after receiving the reward, there is a predictable slowdown or complete break before starting the next set of actions

        • Once a reward is handed out, the brain recognizes that the next reward is at its maximum distance away—you have to start the whole counter over from zero. This creates a moment of fatigue or temporary hesitation before diving back into the work

    • Variable Ratio Schedule (VR-n):  The organism is reinforced after a changing number of responses centered around an average (e.g., VR-10). Maintains a high and steady response rate that is extremely difficult to extinguish. Long VR schedules often induce schedule- and extinction-induced aggression.

      • You get a reward after doing an action a variable number of times, but it averages out to n. For a VR-10 schedule, you might get paid after 3 attempts, then 18, then 7, then 12—which averages out to a reward every 10 attempts

      • The reward is unpredictable. You never know which exact attempt will pay off, but doing the behavior faster brings rewards sooner

      • Because the next attempt could be the winner, there is no post-reinforcement pause. The rate of response is fast, constant, and resistant to stopping (high resistance to extinction)

      • If the required number of attempts becomes too high (e.g., VR-200) or stops paying out altogether, the unpredictability creates intense frustration, leading to aggressive outbursts aimed at the machine or environment

    • Fixed Interval Schedule (FI-t):  Reinforcer is delivered for the first correct response made after a fixed period of time (t) has elapsed. Responding shows a positively accelerated response pattern ("scalloping").

    • Variable Interval Schedule (VI-t):  Reinforcement is delivered for the first correct response made after a variable period of time has elapsed since the last reinforcement. Executes at slow but steady rates; extinction is gradual.

  • Observational Learning Theory

    • Cognitive Conditioning:  Introduces the concept that our interpretation of the situation can affect the effectiveness of the conditioning process.

    • Observational & Latent Learning:  We do NOT have to personally be reinforced or punished to learn what to do in a situation. We learn simply by observing other people's experience of reinforcement and punishment ( observational learning ). Learning how to behave without having to try out the behavior ourselves is called latent learning.

    • Mirror Neurons:  The physiological basis of observational learning is mirror neurons—neurons that become active when we perform an action and when we simply watch others perform the same action.

Other Limitations of Learning through Conditioning

  1. Evolutionary Preparedness:

Innate, evolutionary tendencies exist that are difficult (if not impossible) to overcome through classical or operant conditioning.

Example:  It is a lot more difficult to teach a rat to develop a conditioned fear response to a visual NS than to an auditory NS.

Example:  It is difficult to teach a rat to become afraid of sugar water through classical conditioning because sugar water is innately appealing to rats.

  1. Instinctive Drift:

In cases where an innate behavior is successfully suppressed (through punishment), the behavior will likely return once punishment stops.

Similarly, when an alternative behavior is encouraged over an innate behavior through reinforcement, the innate behavior will return soon after reinforcement stops.


________________________________________________________________________

TEXTBOOK NOTES

Chapter 6 - Learning

Learning - A change in an organism’s actions, thoughts, or emotions as a result of experience

Habituation - The process by which we respond less strongly over time to repeated stimuli

  • It is the simplest and probably earliest form of learning to emerge in humans

  • When researchers apply a gentle vibration to the mother’s stomach, the fetus jerks around in response to the stimulus, but stops moving after repeated vibrations

  • Habituation makes good adaptive sense

  • We typically don’t habituate to powerful stimuli, like extremely loud tones or painful electric shocks

Sensitization - Responding more strongly over time to stimuli

  • Most likely when a stimulus is dangerous, irritating, or both

6.1 - Classical Conditioning

Conditioning - Forming associations among stimuli

  • In most cases, the CR is fairly similar to the UCR, but it’s rarely identical to it (ex. usually salivating less)

  • Classical condition may not even require the animal to be conscious

    • Classical conditioning can occur even among people who are in a vegetative state

Three phases of classical conditioning - acquisition, extinction, and spontaneous recovery

  • Acquisition - gradually learn or acquire the CR

    • Backward conditioning - when UCS is presented before the CS - can be difficult to achieve

      • For conditioning to work efficiently, the CS must forecast the appearance of the UCS

  • Extinction - CR decreases in magnitude and eventually disappears when the CS is repeatedly presented alone

  • Spontaneous recovery - a seemingly extinct CR reappears (often in somewhat weaker form)

    • Renewal effect - when we extinguish a response in a setting different from the one in which the animal acquired it

    • Stimulus generalization - the process by which CSs that are similar, but not identical, to the original to the CS elicit a CR. The more similar to the original CS the new CS is, the more stronger the CR will be

    • Stimulus discrimination - occurs when we exhibit a less pronounced CR to CSs that differ from the original CS

    • Higher-order conditioning: the process by which organisms develop classically conditioned response to previously neutral stimuli that later became associated with the CS


Conditioned compensatory response (CCR) - an automatic physiological reaction triggered by environmental cues that is opposite to the effect of a drug or substance

6.2 - Operant Conditioning

Insight - grasping the underlying nature of the problem. Thorndike concluded that cats don’t learn by insight

Skinner box - an operant chamber which electronically records an animal’s responses and prints out a graph of the animal’s activity

Discriminative stimulus - any stimulus that signals the presence of reinforcement

  • Finger snapping for a dog to approach to be petted is an example of a discriminative stimulus that indicates the availability of positive reinforcement, encouraging the dog to engage in the desired behavior.

Shaping by successive approximations - Conditioning a target behavior by progressively reinforcing behaviors that come closer and closer to the target

6.3 - Cognitive Models of Learning

Cognitive maps - mental representations of how a physical space is organized

6.4 - Biological Influences on Learning

Preparedness - evolutionary predisposition to learn some pairings of feared stimuli over others owing to their survival value

6.5 - Learning Fads: Do They Work?

Learning styles - an individual’s preferred or optimal method of acquiring new information


Stimulus-organism-response (S-O-R)