Chapter 6: Aversive Control of Behaviour

Aversive Stimuli

• Events that organisms escape from, evade, or avoid

• Primary aversive events:

• Avoidance was adaptive; animals that avoided these events more likely to survive and reproduce

• Examples: loud noises, bright light, foul odours, intense heat/cold, insect stings

• Not learned, inherently aversive

• Conditioned aversive events

• Examples: verbal threats, public criticism, a failing grade, a frown, and verbal disapproval

Primary Aversive Events

• Events that organism comes biologically prepared to avoid or escape

Conditioned Aversive Events

• Events acquire aversive properties by being associated with primary aversive stimuli

Aversive Control of Human Behaviour

• Humans extensively use and arrange aversive stimuli to control the behaviour of others at individual, societal, and institutional levels

• Excessive reliance on use of aversive control

Punishment

• Punishment, like reinforcement, is defined solely by its effect on behaviour

• Any consequence, made contingent upon a behaviour, that decreases the future probability of that behaviour is punishment

Positive Punishment

• Decrease is an operant behaviour when a stimulus is presented following the operant

• Spanking

• If a consequence increases the future occurrence of a behaviour then it is reinforcement

• Punishment, by definition, always works

Spanking

• Only a punishment if it decreases the future likelihood of the behaviour

Negative Punishment

• Decrease in an operant behaviour when a stimulus is removed following the operant

• Stimulus/activity that a person enjoys is removed contingent upon behaviour (TV is turned off when child watching TV, jumps on the sofa)

Timeout

• Loss of access to reinforcement for a specified time period contingent upon a behaviour

• Exclusionary timeout

• Non-exclusionary timeout

• Effectiveness depends upon the classroom activities being reinforcing for the student, the behaviours that lead to time out being clearly specified, and the method (exclusionary/non-exclusionary)

• Teacher should keep precise records to evaluate if the behaviour is decreasing

Exclusionary Timeout

• Physically remove the person from the situation (time out room, a barren school hallway)

Non-Exclusionary Timeout

• Use a stimulus (pin) correlated with ignoring of behaviour or withdrawal of a specific positive reinforcer

Response Cost

• Behaviour decreases due to the removal of a stimulus (something of value) contingent upon the behaviour

• Loss of money, privileges

Relativity of Punishment

• Premack Principle: reinforcement involves making a high probability behaviour contingent upon a low probability behaviour, low probability behaviour increases

• Punishment: making a lower probability behaviour contingent on a higher probability behaviour, higher probability behaviour decreases

Does Punishment Work?

• Originally, Thorndike's law of effect was symmetrical, reinforcing consequences increase behaviour and punishing consequences decrease behaviour

• Later Thorndike, revised this formulation to reinforcement works, punishment does not

• Why?

  • Estes suggested that punishment suppresses responding; however, no more so than aversive stimuli delivered independently of behaviour

  • Effect of exposure to aversive stimulation is to generally suppress behaviour whether or not the occurrence of aversive stimulation depends on a particular behaviour

  • Question becomes:

    • "Does a contingent relation between responses and aversive stimuli result in a greater suppression than when aversive stimuli are merely presented independent of responding?"

Contingent vs Non-Contingent

• Train a rat to press a lever and to pull a chain using reinforcement

• After training, present only one of the two responses (chain for some, levers for others)

• Present shock contingently on available behaviour for half the rats and non-contingently for other half

• Test for rate of responding in the presence of both the lever and the chain after these procedures

• Measure suppression of both behaviours

• If suppressive effects of punishment are general, then expect equal suppression of both trained behaviours regardless of whether aversive stimulation was contingent or not

• If suppressive effects of punishment are specific, then predict little or no suppression of response not available and considerable suppression of response available during punishment training

• Results:

  • Contingent shock: little suppression of untrained response (SR [suppression ratio] = .4), but greater suppression of trained response (SR = .2)

  • Noncontingent shock: suppression ratios of ~ .4 for both trained and untrained responses (a little suppression)

  • Punishment selectively reduces the responses that the aversive stimulation is contingent upon and does not work through a general reduction in all responding

Contingent vs Non-Contingent Shock

• Goodall (1984) degree of suppression varies with strength of response-shock contingency

• Strong response-shock contingency (responses frequently followed by shocks) produces greatest reduction in response rate

• Weak response-shock contingency produces reductions to a lesser degree

• Non-contingent shock produced little reduction in responding regardless of frequency of shocks

Using Punishment Effectively

1. Abrupt, not gradual, introduction

2. Intensity

3. Immediacy

4. Schedule of punishment

5. Reduce effectiveness of positive reinforcement for the response

6. Combine punishment with reinforcement of alternative behaviour

7. Combine punishment with extinction

8. Combine punishment with stimulus control

9. Prevent escape

Abrupt, Not Gradual, Introduction

• Abrupt introduction of a moderate intensity shock (80 V) produces lasting suppression of behaviour

• Pigeons continue to respond at intense levels of shock (130 v) when shock was introduced at a low intensity (60 v) and gradually increased,

• Gradual increases in intensity produce adaptation (behaviour recovers) and consequently punishment suppresses behaviour less

Intensity

• Punishment should be introduced at a moderate to high intensity on the first occasion

• Severity: greater the intensity of the aversive stimulation, greater the suppression

• Low intensity punishment may leave the behaviour relatively unaffected; intense punishment can completely eliminate behaviour

• Punishment only produces temporary suppression of behaviour; behaviour can recover to pre-suppression levels

Immediacy

• Celerity: more immediately the aversive consequence follows the behaviour, the greater the suppression

• As delay between the response and the occurrence of punishing consequence increases, degree of suppression rapidly diminishes

• Immediate positive punishment can also elicit emotional behaviours that prevent the occurrence of the operant

Schedule of Punishment

• Certainty: the greater the probability that a response will be punished, the greater the suppression of that behaviour

• Punishment is most effective when the punishing stimulus is applied every time that the behaviour occurs (CRF/FR1)

• Intermittent schedules of punishment produce less suppression than continuous schedules of punishment

• Schedule effects with punishment are opposite of the patterns produced by positive reinforcement

• FI schedule of punishment superimposed over responding on a VI schedule of positive reinforcement produces an inverse scallop

• Each occurrence of a punisher is followed by a high rate of response that decreases as the time for the next punishment approaches

Reduce Effectiveness of Positive Reinforcement for the Response

• Arzin, Holz & Hake (1963)

• Trained pigeons to peck a key on a VI 180-s schedule of food reinforcement

• Introduced shock punishment on FR 100 schedule

• Different groups were food deprived to different percentages of free-feeding body weight: 60, 65, 70, 75, or 85%

• Results:

  • More efficacious the food reinforcement was (greater deprivation), the less the suppression of key pecking

  • 85% body weight group stopped pecking

  • 60% body weight group emitted a high and stable rate of pecking

  • Satiation for reinforcement maintaining behaviour would increase the suppressive effect of punishment

Combine Punishment with Reinforcement of Alternative Behaviour

• Suppression associated with punishment is more effective and lasts longer if punishment of one behaviour is combined with reinforcement of an alternative behaviour that is incompatible with the punished behaviour

• Punished response rapidly decreases when an alternative behaviour that produces the same reinforcement is available

Combine Punishment with Extinction

• Punishment of a behaviour combined with removing sources of reinforcement for that behaviour is more effective at rapidly reducing the behaviour than either punishment or extinction alone

Combine Punishment with Stimulus Control

• Punishment is more effective when used with stimulus control (e.g., warning labels, signs, rules) that makes conditions under which punishment will occur as explicit as possible

• Problem: person administering the punitive consequences becomes a discriminative stimulus for punishment (S^DP), behaviour is suppressed only as long as that person is present

Prevent Escape

• Arrange the context so that escape does not occur

• Escape is a natural reaction to exposure to an aversive stimulus, opportunity for escape must be precluded

Permanence of Punishment

• Can punishment by itself permanently eliminate a behaviour?

• No! Punishment only temporarily suppresses behaviour while the punishment contingency is in effect

• Punishment in combination with extinction and reinforcement of alternative behaviours can produce lasting suppressive effects

Paradoxical Effects of Punishment

• More and Kelleher (1977): monkey responding on VI schedule of food reinforcement, then superimposed FI 10 min shock schedule, subsequently, food reinforcement was eliminated so that lever pressing only produced shock on FI schedule

• Paradox: monkey continued to lever press

• Interpretation: not that shocks function as positive reinforcement

Advantages of Punishment

1. Rapidly suppresses behaiour

2. Facilitates discrimination

3. Instructive to peers (vicariously

Rapidly Suppresses Behaviour

• In a situation where the behaviour of an individual is harmful to themselves or to others, a contingency of punishment can be used to rapidly suppress a behaviour

• E.g., a 5 year old child was inducing epileptic seizures by waving his hands rapidly in front of his face

Facilitates Discrimination

• Punishment of behaviours under specific conditions is informative to individuals (about what not to do)

• When punishment is paired with clear discriminative stimuli (e.g., rules, signs), it assists the youngsters to discriminate unacceptable behaviours more rapidly

• Punishment suppresses, it does not teach what to do

Instructive to Peers (Vicariously)

• Punishment of the behaviour of one individual may reduce the probability that others present will imitate that behaviour

• Deterrence

Disadvantages of Punishment

1. Promotes escape and/or avoidance

2. Causes aggression

3. Causes generalization

4. Modelling punishment

5. Teaches the punished act

6. Produces peer reactions

7. Causes negative self-statements

8. Suppression of behaviour by punishment reinforces its use by the punishing agent

Promotes Escape and/or Avoidance

• The severely scolded child runs away from home, the employee quits after being chewed out by the boss; hit and run driver

• One form of escape is lying to avoid punishment

Causes Aggression

• Subjects become aggressive in response to aversive stimulation; aggression is directed toward the individual delivering the punisher

• Schedule-induced aggression or pain-induced aggression; delivery of shocks elicit aggressive attacks against other individuals

Causes Generalization

• Emotions/behaviours associated with a punishing stimulus may also be occasioned by other stimuli (over generalizes)

• E.g., fear/anxiety associated with being punished by one individual (teacher) generalizes to other individuals (teacher)

• Necessary to inform subjects that they have been punished for specific behaviours under specific conditions to minimize generalization

Modelling Punishment

• Observers are more likely to imitate the act of delivering punishment

• Observers are learning that punishment is the way to deal with a problem rather than other methods

Teaches the Punished Act

• Once attention is drawn to the punished behaviour, it may occur under different conditions

• For example, after observing swearing be punished children may imitate punished behaviour such as swearing when the contingency manager (parent) is no longer around

Produces Peer Reactions

• When an individual is repeatedly singled out for punishment, they may themselves become stimuli to be avoided or ridiculed by their peers

Causes Negative Self-Statements

• After punishment, individuals are more likely to make negative statements about themselves

• Negative self-statements are more likely to occur if the aversive stimulation is directed at the individual rather than the behaviour

• This is important because studies have shown that what children say about themselves is related to school achievement

Suppression of Behaviour by Punishment Reinforces its use by the Punishing Agent

• Problematic if punishment is being applied without consideration of harmful side effects or when other less aversive/harmful techniques can be used to accomplish the same change in behaviour

Conditioned Emotional Response Theory

• Estes

• Response suppression occurs when a CS that is predictive of a shock elicits responses that are incompatible with the operant response

• CS elicits emotional response such as freezing that is incompatible with lever pressing

• When no explicit CS present, various stimuli (visual and spatial cues near the lever; tactile cues of the lever) experienced just before the lever press that produced the shock occurred serve this function

• Explains why longer duration and more intense shocks produce greater suppression

• Explains why response contingent shock produces greater suppression than noncontingent shock

Avoidance Theory of Punishment

• Dinsmoor proposed two factor theory of punishment

  • Stimuli that set the occasion for the instrumental response become conditioned by the aversive stimulus when the response is punished

  • Conditioned aversive stimuli motivate an escape response (negative reinforcement) which is incompatible with the punished response

  • Suppression is not a function of the weakening of the punished response rather it results from the strengthening of responses that are incompatible with the punished response

  • Considered a weak theory since it does not precisely specify what stimuli acquire conditioned aversive properties and what these incompatible escape responses are

Negative Law of Effect

• Thorndike's formulation

• Punishment acts directly to weaken the punished response; not indirectly as proposed by Dinsmoor

• Akin to a one factor theory of punishment in that it postulates that the classical conditioning factor is not an important or necessary part of the punishment process

• Punishment has a selective weakening effect on the behaviour that produces the aversive stimulation.

Negative Reinforcement

• Increase in operant behaviour due to the removal or prevention of an event or stimulus

• Negative reinforcer: any event or stimulus that increases operant rate by its removal (or prevention)

• Aversive stimulus: any event or stimulus that an organism avoids or escapes

Escape vs Avoidance

• Discriminated Avoidance Procedure

  • Discrete trials: a signal / cue / CS is presented prior to presentation of aversive stimulation (US)

  • In the presence of aversive stimulation, subject can make a response that terminates the aversive stimulation (escape)

  • In the presence of signal that predicts upcoming aversive stimulation, subject can make a response that prevents aversive stimulation (avoidance)

Escape Learning

• Escape responses are acquired more readily than avoidance responses because there is an immediate change from the presence to the absence of aversive stimulation

• Species specific behaviour elicited by aversive stimulation can interfere with escape conditioning

• Operant behaviour of lever pressing is incompatible with freezing / immobility elicited by shock

• If operant behaviour is similar to the reflexive behaviour elicited by the aversive stimulation, then escape learning occurs more quickly

• Running in a wheel to escape electric shock; running is part of the species specific responses to electric shock in rats

Discriminated Avoidance

• Two process theory of avoidance

  • Proposed by Mowrer (1947)

• Classical conditioning process

• Operant process

Classical Conditioning Process

• A transfer of fear-eliciting properties of shock to a neutral stimulus (e.g., light) that predicts the shock (UCS)

• Onset of light becomes a CS that elicits fear/anxiety

Operant Process

• Avoidance response is reinforced by reduction in fear by removal of fear-eliciting CS (response turns off the light)

• Negative reinforcement of avoidance response because it leads to the removal of the conditioned fear response elicited by the stimulus predictive of the aversive event

Fear Reduction

• Is the reduction o fear elicited by a conditioned aversive stimulus sufficient to reinforce a response?

• Brown and Jacob’s (1949) tested rats in shuttle box

  • Phase 1:

    • Experimental group rats experienced22 pairings of a pulsating light/tone CS with shock (us) in a shuttle box with second compartment closed off

    • Control group rats get same training, but without shocks

  • Phase 2:

    • Rats from both groups placed in one side and given access to the other side, the pulsating tone/light CS was presented and could be turned off by crossing over to the other side

    • CS remained on until animal crossed to other side

    • No shocks delivered

  • Results

    • Rats in experimental group cross over more rapidly than control rats

  • Supports two factor theory proposition: termination of a fear-eliciting stimulus will negatively reinforce an instrumental response

Fear and Avoidance Responding

• Two factor theory: avoidance response is maintained by fear reduction; this means that when the response is established, animals should still display signs of fear in the presence of CS predicting the shock

• Problem: animals appear less fearful as they become proficient at performing the avoidance response

• Kamin, Brimer, an Black (1963): if a CS from a shuttle avoidance procedure elicits fear then it should suppress lever pressing for food in a conditioned suppression procedure

• Rats trained to press a lever for food reinforcement on a VI schedule in an operant conditioning chamber

• Rats then trained to avoid shock in a shuttle box using an auditory CS

• After 1, 3, 9 and 27 consecutive avoidance responses in the shuttle box, auditory CS presented in the operant chamber when rat was pressing a lever for food

• Degree of suppression of lever pressing in the presence of the CS measured

• Results

  • Suppression increased between 1and 9 consecutive avoidance responses

  • Much less suppression when rats had made 27 consecutive avoidance responses

• Decline in fear as avoidance responding is acquired is problematic for two process theory

Conceptual Analysis of Asymptotic Avoidance Performance

• According to classical conditioning, when avoidance responding is established, CS that elicits fear should undergo respondent extinction (CS presented alone and not followed by the US)

• Extinction of fear-elicited properties of the CS should lead to extinction of the avoidance response (fear reduction that reinforces the response will be diminished)

• Animal should fail to make the avoidance response which leads animal to experience the shock (US) predicted by the CS

• Pairing the CS with the shock again leads the CS to require fear-eliciting properties and avoidance response should occur again

• Theory predicts avoidance response should show cycles of extinction and reacquisition

• Empirical studies show avoidance responding is extremely resistant to extinction

• Avoidance responses carry on for many, many trials without a shock occurring

• Conservation of fear

• Avoidance responding maintained by part of CS that does not undergo extinction

Conservation of Fear

• supporters of two-factor theory propose that short latency between onset of the CS and occurrence of the avoidance response means that later components of CS are not experienced and do not undergo extinction

Non-Discriminated Avoidance

• Can animals learn to avoid shock if there is no external stimulus (warning sign) signaling an upcoming aversive event?

• Sidman developed a non-discriminated or free operant avoidance procedure to address this question

Sidman Avoidance Procedure

• No warning signal precedes the shocks (non-discriminated)

• Shocks occur at regular intervals (every 5 sec)

• If no responses are made, rat receives shocks every 5 sec (S-S interval)

• If a response occurs, a shock does not occur for 30 sec; every response resets timer to postpone the shock for 30 sec (R-S interval)

• All shocks can be avoided if the subject responds at least once every 30 sec

Differences with Discriminated Avoidance

• No signal predictive of a shock

• Avoidance response can occur at any time

  • In the discriminated avoidance procedure, responses in the absence of the CS have no effect, in a Sidman avoidance procedure, each response delays the shock by 30 sec

• Speed of acquisition and rate of the avoidance response depend on S-S and the R-S intervals

• The shorter the S-S interval and the longer the R-S interval, the more likely the animal will learn the avoidance response

• Animal will not acquire the avoidance response if R-S interval is shorter than S-S interval

• If S-S interval is 20 s and R-S interval is 5 s, then making the response increases the frequency of shocks (shock occurs 5 s after press)

• Issue: R-S interval becomes a temporal CS predictive of shock, the more time since the last response, the greater the anxiety about an upcoming shock (fails to eliminate classical conditioning process)

• Making a response eliminates the anxiety