Catania 5

Behavior Without Learning

Reflexes and Environment

  • A dog's leg flexion reflex in response to a sharp stimulus like a thorn is advantageous for ground-walking animals as it pulls the paw away from the threat.
  • However, this same flexion reflex would be detrimental to a sloth hanging from trees, as pulling the limb towards the body would drive the thorn in deeper. Instead, a sloth's comparable reflex involves limb extension.
  • cf.Hart,1973,p.176cf. Hart, 1973, p. 176

The Evolution of Learning

  • The capacity for behavior to be modified, or learning, was a major evolutionary step.
  • Learning may have initially been selected within specific domains. For instance, the digger wasp's ability to learn landmarks for nest return is likely specific to that task.
  • The capacity to learn is subject to natural selection, varying across species and times.
  • Behaviors that remain constant over a lifetime need only be learned once, whereas behaviors in changeable environments require repeated learning and discarding.
  • Examples include:
    • Names of familiar people, which are relatively constant.
    • Shopping lists, which change frequently.
  • We should expect some learning to be difficult to reverse, whereas other learning remains transient and easily modified.

Imprinting and Developmental Learning

  • A newborn calf's survival depends on quickly learning to distinguish its mother, and this imprinting may be effectively permanent in a stable environment.
  • The time of life during which learning may occur is often limited.
  • As the calf matures, it learns about changeable aspects of its environment, such as predators and food/water locations.
  • The capacity to learn and remember can vary developmentally. Examples include Peterson (1962). Shaping a key peck on newly hatched ducklings had short-lived effects that became longer-lasting as the chick aged.

Evolutionary Contingencies and Learning Mechanisms

  • Evolutionary contingencies have selected for different kinds of learning, including:
    • Permanent vs. easily reversed changes in behavior.
    • Narrowly specific vs. broadly general applications.
    • Different times of operation in an organism's life.
  • The selection of nervous systems corresponds to the behaviors they produce.
  • If that evolution worked like the evolution of anatomical structures, we should not be surprised if some evolved learning systems are modular.

Learning Without Words

Consequences of Responding: Reinforcement

  • The Indo-European root sekw-, meaning "to follow," connects consequence to signal, designate, social, and association.
  • Conditioning shares its prefix con- with contingency and contiguity.
  • The Indo-European root deik- meaning to show or pronounce, relates conditioning to dictation, teach, judgment, and paradigm.
  • Contingency, from the Latin contingere, means a possibility, a condition of depending on chance, or something incidental.
  • Like contact combines the roots com-, together, and tangere, to touch.
  • Contiguity, the condition of touching or being in contact, has the same origins.
  • Contingency stresses how the likelihood of one event is affected or caused by other events.
  • Contiguity implies the juxtaposition of events in space or time without regard to causation.

Key Terms

  • Reinforce, Reinforcer, Reinforcement
  • Contingency
  • Extinction, Spontaneous Recovery
  • Free or Noncontingent Reinforcement
  • Superstition
  • Side Effects

Focus on Nonverbal Behavior

  • We will concentrate on behavior without words, including human examples where talk is irrelevant.
  • Human examples are often embedded among examples involving nonhuman organisms such as pigeons and rats, and readers often miss the use of human behavior to illustrate the relevance of findings from nonhuman research.
  • The advantage of starting with nonhuman organisms is precisely that they don't talk: unlike us, they are nonverbal creatures.
  • Resist the temptation to treat what they do as if they talk to themselves about it the way we talk to ourselves.
  • All of our behavior that involves talk is built upon nonverbal foundations that are covered over by the talk, and nonverbal organisms provide our best path to understanding those foundations.
  • We'll be in a far better position to deal with what is special about it because we first surveyed learning without words.

Behavior and Consequences

  • Behavior has consequences, and an important property of behavior is that it can be affected by its consequences.
  • We study this phenomenon by arranging consequences for behavior, but doing so involves more than just presenting stimuli. The stimuli must occur in some relation to behavior.
  • We have to arrange the environment so that responses make something happen.
  • Natural environments already provide consequences for behavior. Even before we intervene organisms change their environments by doing things or by going from one place to another.
  • We can better study how consequences affect behavior by arranging consequential environments in the laboratory.
  • Examples:
    • A maze in which a water-deprived rat finds water after making an appropriate sequence of turns.
    • A chamber in which a food-deprived pigeon produces food by pecking a key on the wall.
  • Find out whether water affects the turns the rat takes as it runs through the maze or food affects how often the pigeon pecks the key.
  • We'll explore the historical development of research on the consequences of behavior and we'll see how reinforcement is relevant not only to behavior maintained by physiologically significant consequences such as food and water but also to behavior with more subtle consequences, as when, in sensory-motor interactions, my eye movements affect what I see.

Reinforcement

  • Chapter 2 introduced Thorndike's experiments, in which animals learned to escape from problem boxes by operating a device that released the door.
  • A food-deprived animal was placed inside the box with food available outside. The animal sooner or later operated the device and was free to leave the box.
  • At first this was a low probability response, but because it opened the door its probability went up over repeated trials.
  • Thorndike described how the consequences of responding affected later responding with a principle he called the Law of Effect.
  • The law went through many revisions, but its essence was that response probability could be raised by some consequences and lowered by others.
  • Responses with satisfying effects were stamped in whereas those with annoying effects were stamped out.
  • The earliest version of Thorndike's law was called the strong Law of Effect. Later, he repudiated the second half of the law, keeping the probability increase or stamping in but discarding the probability decrease or stamping out.
  • What remained was then called the weak Law of Effect. This historical point will be relevant later, when we deal with punishment.
  • Figure 5-1 shows data from one of Thorndike's cats. To escape from its box, this cat had to pull a string running from a wire loop at the front of the box to a bolt holding the door. The first time in the box the cat took 160 s to escape. Its time decreased gradually and irregularly over successive trials until, during the last few trials, the cat reliably escaped in less than 10 s. This gradual decrease in the time taken to complete a task came to be called trial-and-error learning.
  • Köhler later contrasted this gradual change with the sudden or insightful solutions he observed with chimpanzees.

Mazes and Learning Curves

  • In later years, trial-and-error learning was studied with many different organisms in many different situations.
  • Experimenters believed that the intelligence of different species could be compared by seeing how rapidly learning went in problem boxes, mazes, runways and other apparatuses (e.g., Hilgard, 1951).
  • Apparatus design began to be dictated by theoretical questions: Does learning take place in discrete steps, on an all-or-none basis, or instead gradually and continuously? Do organisms learn movements (response learning) or properties of the environment (stimulus learning)? Do the consequences of responding lead directly to learning or only make the organism perform so as to show what it had learned in other ways?
  • Graphs showing how behavior changed during an experiment were called learning curves: time to complete a response as a function of number of trials (e.g., Figure 5-1); percentage of correct responses; proportion of animals reaching some criterion of successful performance.
  • Sometimes these measures were transformed to ease comparisons among them.
  • Converting time to run the maze to speed (the reciprocal of running time) made all three measures increase with learning.
  • But the shapes of learning curves depended so much on apparatuses and on measures taken that the progress of learning couldn't be described in any unitary way.
  • Measuring the time course over which a rat stopped entering blind alleys as it learned its way through a maze didn't show how learning proceeded at a single choicepoint. This consideration led to the gradual simplification of mazes, as illustrated in Figure 5-2.

Evolution of Mazes

  • Diagram A shows the plan of the earliest maze used to study animal learning (Small, 1899–1900), a 6-by-8-foot modification of the hedge maze at Hampton Court in England. (Curiously, such mazes also provided settings for the hydraulically operated statues that inspired Descartes to formulate his concept of the reflex; cf. Chapter 4.)
  • When a cage door at the start was lifted, rats could enter the maze; food was in the goal area at the center. As their experience grew they reached the goal area more rapidly and with fewer wrong turns along the way. But it was difficult to examine learning at any given choicepoint. Learning might have been quicker at choicepoint 1 in diagram A than at 7 either because 1 was earlier in the maze than 7 or because their floor plans differed; learning might have been quicker at 4 than at 5 either because it could be approached in two ways, from 3 or from 5, or because a rat that usually went directly from 3 to 4 only rarely encountered 5.
  • Gradually, mazes evolved into more systematic forms, as in diagram B. In this maze, sometimes called a U-maze after the form of the successive units, choicepoints were essentially the same as the rat approached each one; they differed only in where they were in the sequence and in whether left or right turns were correct. This kind of systematic arrangement made it easy to specify the correct sequence (in B, right-left-right-left-left-right) and to keep track of errors. Even here, however, position and sequence interactions complicated the analysis.
  • For example, is the rat's choice of left at 4 affected by the preceding right turn at 3 or the following left turn at 5? Would it matter if the rat approached 4 after coming back from the blind alley at 3, having made an error there, instead of after a correct right turn at 3? Does it matter that 4 is in the middle rather than near the beginning or end?

T-Maze and Runway

  • The maze would be reduced to a single choicepoint, as in the T-maze shown with a goalbox on the right in diagram C in Figure 5-2. Here, when leaving the startbox the rat has only to make a single choice of right or left. But complications were still possible. For example, suppose one rat in its first trial in this maze turns right while a second rat turns left. Should the second one be allowed to retrace its steps after reaching the empty box at the end of the left arm? If it is returned to the startbox instead, should it be forced to the goalbox by blocking the left arm, to give it time in the goalbox equal to the first rat's?
  • The next logical step was to eliminate choicepoints completely, leaving nothing but a simple runway, as in diagram D. Now no errors are possible, and measures of behavior can be reduced to just the speed of the rat's run from startbox to goalbox.

Individual vs. Group Performance

  • Average measures of the performance of a group didn't necessarily represent performances of the individuals in the group.
  • Suppose each rat running in a T-maze usually changes abruptly from making frequent errors to making consistently correct choices, but this change occurs on different trials for different rats. In a large group of rats, 65% might make correct turns by trial 5, 72% by trial 6, 79% by trial 7, 83% by trial 8, and so on, until performance becomes stable at 98% to 100% by trial 20. This group performance, giving the appearance of a gradual increase in correct choices, would completely obscure the abrupt change in errors for all individual rats (Sidman, 1952).

Simplification and Experimental Innovations

  • Even the simple runway wasn't an ultimate solution, because the speed of a rat's running down a straight alley can be affected by many trivial factors. If trials began with the opening of a startbox door, it depended on which way the rat was facing when the door opened. Other factors included the experimenter's handling of the rat when moving it between trials from goalbox to startbox, odor trails left by other rats, and even whether the goalbox had enough room to allow a running rat to slow down before banging its head against the goalbox back wall (Killeen & Amsel, 1987).
  • With mazes or runways, the experimenter had to return the rat from goalbox to startbox to begin a new trial. Thus, the experimenter rather than the organism determined when behavior occurred. Furthermore, measuring how long the rat took didn't specify what it was actually doing during that time.
  • Two experimental innovations helped to solve these problems. The first was an apparatus designed so that the organism could repeatedly emit an easily specified response without the experimenter's intervention; the second was a recording method based directly on the rate or frequency of responding rather than on indirect measures derived from response sequences or groups of organisms. These innovations, inspired partly by an interest in reducing the handling of the organism and thereby simplifying the experimenter's work, were important features of a direction of research initiated by B. F. Skinner (1930, 1938, 1950); see especially Skinner (1956) for a history of these developments.

Experimental Chambers and Cumulative Records

  • Figure 5-3 illustrates two representative apparatuses: on the left a standard rat chamber with a single lever, and on the right a three-key pigeon chamber. They share response devices, mechanisms for delivering reinforcers such as food or water, and stimulus sources.
  • In a typical arrangement, a rat that has been food deprived is placed in the chamber. A lever protrudes from one wall. Near the lever is a food cup into which food pellets can be dispensed from a delivery system on the other side of the wall; a distinctive sound accompanies each pellet delivery. The houselight provides general illumination, and noise can be broadcast from the speaker to mask any sounds from outside the chamber.
  • The first step is feeder training. Pellets are delivered into the food cup. Sooner or later, the rat finds and eats them. Once this happens, pellet deliveries continue until the rat comes quickly to the food cup from each delivery; 10 or so pellets are usually enough. Once feeder training is done, the apparatus is changed so that pellet deliveries depend on lever presses. Eventually the rat presses the lever, the press produces a pellet, and the pellet occasions eating. The rat will then probably go back to the lever and press it again. The outcome of interest is the rate at which the rat presses the lever. If it increases, we call the food pellet a reinforcer. In the type of chamber shown in Figure 5-3, other kinds of reinforcers can be substituted. For example, the pellet dispenser could be replaced by a dipper that delivers small amounts of water or milk.
  • A pigeon chamber differs from one for rats in that keys substitute for levers and the feeder works by bringing a tray of pigeon food (mixed grain or commercially available pellets) within the pigeon's reach for a few seconds. The feeder opening is centered below the keys. It is common practice to light the feeder and turn off all other lights whenever the feeder is operated. The chamber typically includes other features, such as a houselight for dim general illumination, masking noise or other auditory stimuli, and so on.
  • A pigeon key is a piece of plastic mounted behind a hole in the chamber wall. It is attached to a switch that records the pigeon's pecks if they're forceful enough (keys are routinely sensitive to forces of less than 0.1N0.1 N or Newtons, about 10g10 g ). The plastic is usually translucent or transparent, so that lamps or miniature projectors or a computer monitor behind the key can present patterns or colors on it. The chamber in Figure 5-3 contains three keys, arranged horizontally about 23cm23 cm above the chamber floor. A given experiment might use only one key, some combination of two, or all three. Keys are typically lit when they're in use. As with the rat, if a food-deprived pigeon's key pecks produce food, the rate at which the pigeon pecks the key will ordinarily increase.

Common Laboratory Organisms

  • The rat and the pigeon are common laboratory organisms. They each have idiosyncratic species-specific behavior patterns that must be taken into account, and we mustn't assume that what we observe with rats or pigeons can be generalized to other organisms (though it often can be). Nevertheless, the diet, housing, susceptibility to disease and other characteristics of these animals are reasonably well understood, and their size, relatively long lifespan and economy make them particularly convenient. Thus, they have often served in research on the consequences of responding.

Free-Operant Responding

  • Responding in apparatuses like those in Figure 5-3 has sometimes been called free-operant responding: free because the organism is free to emit the response at any time rather than waiting for the experimenter (the rat in a goalbox can't run through the maze again until it is placed back in the startbox and the experimenter opens the startbox door); and operant because the response operates on the environment.
  • Free-operant responding lends itself to a recording method, the cumulative record, that gives a convenient, detailed picture of how responding changes over time.
  • Most contemporary cumulative records are produced by computer, but in the original cumulative recorder, illustrated in Figure 5-4, a roll of paper was threaded around a roller. A motor drove the roller at a constant speed, feeding out the paper. A pen or other writing device rested on the paper as it passed over the roller, and each response (e.g., a pigeon's key peck) moved the pen a small distance at right angles to the movement of the paper. Thus, at any time during a session this record shows the total responses accumulated.

Cumulative Records

  • Figure 5-5 shows examples of cumulative records. Because the record advances at a constant speed, its slope is steeper the higher the rate of responding, as illustrated in records A and B. In the scale for Figure 5-5, rates are roughly 30 and 12 responses/min in records A and B respectively. Record C includes only a few responses; the horizontal portions indicate periods of time without responses.
  • Note that a cumulative record can't have a negative slope, because the pen can record responses only by moving in one direction across the page.
  • In record D, a magnified section of record C that includes a few responses accompanies an event record on the same time scale. For each response in the event record, a small step occurs in the cumulative record; this property of cumulative records isn't always obvious because typical response and time scales are often too small for such fine resolution. Even so, different patterns of responding are easily distinguished. For example, response rates in E and F are roughly the same, but E is steplike whereas F is relatively smooth. This means that E was produced by short high-rate bursts of responding (steep segments) separated by pauses (flat segments), whereas F was produced by more uniform responding. This property is sometimes called grain; of the two records, E has a rougher grain than F.
  • Records G and H illustrate other properties of behavior made visible in cumulative records. In G, the rate begins at roughly 25 responses/min but gradually decreases as time passes; in H, it changes in the opposite direction, increasing from a relatively low rate to roughly 30 responses/min (records in which slopes decrease over time are called negatively accelerated, those in which they increase are called positively accelerated).

Cumulative Records and Behavior Patterns

  • Figure 5-6 shows other features sometimes incorporated into cumulative records. Records A and B show how pen displacements can indicate other events besides responses. Here only some responses produced food, irregularly in A (as at a, b and c) and regularly in B (as at d and e). The repeated concave pattern in B, as between d and e, is sometimes called scalloping. In C, responding that began at f produced food at g, as indicated by the pen displacement. The pen then reset to h and the sequence repeated at h to i and so on. This type of record makes successive segments easy to compare (e.g., the segment ending at g contains more responses than the one ending at i). Record D shows how sustained pen displacements can distinguish different conditions. Here responding occasionally produced food only in the presence of a tone; the pen stayed in its normal position during the tone, in segments j, l, and n, but was displaced downward in its absence, in segments k, m and o.

Language of Behavior

  • With this treatment of free-operant behavior and cumulative records, we've explored part of the technology of the science of behavior.
  • Before we can effectively consider the findings made available through such analyses, we must attend to some properties of the language of behavior.

The Vocabulary of Reinforcement

  • Lever pressing becomes more probable when a water-deprived rat's lever presses produce water than when they don't. Key pecking becomes more probable when a food-deprived pigeon's key pecks produce food than when they don't. And perhaps a child's cries become more probable when they produce a parent's attention than when they don't.
  • These cases illustrate reinforcement: Responding increases when it produces reinforcers. The principle is simple, but as it evolved from Thorndike's initial versions of the Law of Effect to its contemporary status, it carried problems of language and logic with it. Reinforcement is not a theory. It is something that happens in behavior, and we must learn to spot it when it happens and not to confuse it with other things that happen that might seem superficially related to it.
  • Table 5-1 summarizes some properties of the contemporary vocabulary of reinforcement. This vocabulary includes the term reinforcer as stimulus and the terms reinforce and reinforcement as either operation or outcome. For example, when a rat's lever presses produce food pellets and lever pressing increases, we say that the pellets are reinforcers and that the lever presses are reinforced with pellets. The response that increases must be the one that produces the consequences. For example, if a rat's lever press produces shock and only the rat's jumping increases, it would be inappropriate to speak of either pressing or jumping as reinforced.

Technical Application of Terms

  • A reinforcer is a type of stimulus, but reinforcement is neither stimulus nor response. As a procedure or operation, reinforcement is presenting a reinforcer when a response occurs; it is carried out on responses, and so we speak of reinforcing responses rather than organisms. We say that food reinforced a rat's lever press or that a pigeon's key peck was reinforced with water, but not that food reinforced the rat or that the pigeon was reinforced for pecking or that a child was reinforced. The main reason for this restriction is illustrated in the last examples: When we speak of reinforcing organisms, it is too easy to omit the response or the reinforcer or both. The restriction forces us to be explicit about what is reinforced by what. Nor must we omit the organism; we can always say whose response it was (e.g., the child's crying).
  • Reinforcement: Responding increases when it produces reinforcers.
  • This vocabulary is appropriate if and only if three conditions exist:
    • (1) a response produces consequences;
    • (2) the response occurs more often than when it doesn't produce those consequences; and
    • (3) the increased responding occurs because the response has those consequences.

Vocabulary Details

Term Restrictions Examples
reinforcer (noun) A stimulus Food pellets were used as reinforcers for the rat's lever presses.
reinforcing (adjective) A property of a stimulus The reinforcing stimulus was produced more often than the other, nonreinforcing stimuli.
reinforcement (noun) As an operation, the delivery of consequences when a response occurs. As a process, the increase in responding that results from the reinforcement The fixed-ration schedule of reinforcement delivered food after every tenth key peck. The experiment with monkeys demonstrated reinforcement produced by social consequences.
to reinforce (verb) As an operation, to deliver consequences when a response occurs; responses are reinforced and not organisms. As a process, to increase responding through the reinforcement operation When a period of free play was used to reinforce the child's completion of school work, the child's grades improved. The experiment was designed to find out whether gold stars would reinforce cooperative play among first-graders.

Logical Difficulties and Circular Definitions

  • When a response becomes more likely because it has produced a stimulus, we say the response has been reinforced and we call the stimulus a reinforcer.
  • If asked how we know the stimulus was a reinforcer, we point to the increase in responding. If we're then asked why the increase occurred, we might say it did so because the response was reinforced.
  • Once we define a reinforcer by its effect on behavior, we create a problem of circular definition if we simultaneously define the effect by the reinforcing stimulus (Meehl, 1950).

Descriptive, Not Explanatory

  • One solution is to recognize that the term reinforcement is descriptive, not explanatory. It names a relation between behavior and environment; it doesn't explain the relation.
  • The relation includes at least three components:
    • First, responses must have consequences.
    • Second, their probability must increase (i.e., they must become more probable than when not having these consequences).
    • Third, the increase must occur because they have these consequences and not for some other reason.
  • Suppose a parent attends to an infant whenever the infant makes cooing sounds, but suppose also that the infant is more likely to make them with the parent present than with the parent absent. How do we decide whether the infant is now cooing because cooing has been reinforced or because the parent is now present and makes cooing more likely?

Reinforcement Criteria

  • The vocabulary of reinforcement requires that
    • Responding has consequences,
    • Responding increases, and
    • This increase occurs because responding has its consequences and not for other reasons.
  • Once these conditions are met, we say the response was reinforced and the stimulus was a reinforcer.

Effectiveness of Reinforcers

  • We might also assume the stimulus will continue to be an effective reinforcer in the future and will reinforce other responses in other situations. But either assumption may be incorrect. The effectiveness of reinforcers changes over time, and any consequence may reinforce some responses but not others.
  • Money is more likely than a smile to reinforce the services of a plumber or an electrician, but the opposite is likely to be the case if the behavior to be reinforced is a lover's embrace. Despite these reservations, the reinforcers used in many standard experimental situations (e.g., food with food-deprived organisms) are likely to reinforce a variety of responses; the experimenter who chooses a stimulus that will reinforce some responses but not others will sooner or later have to cope with the difference.

A Cautionary Note

  • A cautionary note is in order. Positive reinforcement can be used to change a developmentally delayed child who engages extensively in self-injurious behavior into one who has learned communicative skills and therefore has been empowered to deal in more constructive ways with his or her caregivers.
  • We might therefore think that positive reinforcement would be especially important to the practitioners of an approach called positive psychology. Unfortunately, they reject it along with the establishing events that make it effective; their rhetoric implies that contingent acts of kindness should always be replaced by random ones (Catania, 2001a; Seligman & Csikszentmihalyi, 2000, 2001).

Extinction

  • The consequences of many responses remain reasonably constant throughout life. For example, we usually touch the objects we reach for and we usually get from one floor to another when we climb a flight of stairs. But for other responses, consequences change. Responses reinforced during childhood may no longer be reinforced in adulthood. Educational systems may arrange consequences such as praise or grades for solving arithmetic problems or answering factual questions, but sooner or later these artificial consequences are discontinued, with the hope that more natural consequences will maintain the responses when the student moves on to other settings. When a response is reinforced, its probability increases. But the increase isn't permanent: Responding decreases to its earlier levels when reinforcement is discontinued.

Discontinuation of Reinforcement

  • The discontinuation of reinforcement is called extinction; when this reduces responding to its earlier level the responding is said to be extinguished. This mainly demonstrates that the effects of reinforcement are temporary. Responding is maintained while reinforcement continues but not after it stops. Thus, the decrease in responding during extinction isn't a special process requiring a separate treatment; it is one of the properties of reinforcement.

Resistance of Change

  • At one time, responding during extinction was a primary measure of reinforcement. Resistance to extinction was expressed as the time elapsed until responding dropped to some specified level. Two hypothetical records of extinction of a rat's lever presses are shown in Figure 5-7. Response rate decreases over time in both (negative acceleration), but depending on the extinction criterion either could represent more resistance to extinction. If the criterion is the time until 2 min go by without a response, then A shows more resistance to extinction than B; A doesn't include 2 min without a response but such a period begins halfway through B. If instead the criterion is total responses emitted, then resistance to extinction is greater for B than for A. Because its definition permitted such ambiguities, resistance to extinction lost favor as a measure of the effects of extinction.
  • But resistance to change, of which resistance to extinction is a special case, remains an significant property of behavior (Nevin, 1992). Two responses might each be maintained at similar rates, but one might be persistent in the face of changes in other events, such as the introduction of reinforcement arranged for different responses, whereas the other might be fragile. For example, arithmetic facts, spelling and other academic skills are said to be fluent when they have been practiced and reinforced to the point where they occur with high accuracy and short latency (Johnson & Layng, 1992). Once such skills become fluent, they also become less likely to be disrupted by changes in settings or other distractions. Thus, fluent skills are far more resistant to change than those that have not been learned to a fluency criterion.

Rate Determination

  • As a rough approximation, we can say that the total activation of responding over its reinforcement history determines its resistance to change, whereas its rate is determined by its context, including such current variables as the reinforcement of other response classes (cf. Nevin, McLean, & Grace, 2001b).

  • Whatever its starting point, extinguished responding tends to decrease by a constant proportion over equivalent times, roughly according to the equation:

    R=klog⁡tR = k \log t

  • where R is the response rate, is the time in extinction, and k is a dimensional constant (e.g., Catania, 2005b; Catania & Keller, 1981; Katz & Catania, 2005; Nevin & Grace, 2005).

  • We will later see, however, that the course of extinction can be influenced by schedules of reinforcement (Chapter 15).

Extinction vs. Inhibition

  • If extinction didn't occur, the effects of reinforcement would be permanent. Any responding engendered by reinforcement would last through a lifetime. Clearly that cannot be generally so. For example, if you wear a watch you probably often turn your wrist to look at it; the consequence of looking is seeing the time. But if you stop wearing the watch for some reason, you'll eventually stop looking; seeing a bare wrist isn't an effective reinforcer.
History and Assumptions
  • The history of the concept of extinction, however, wasn't so simple. It was long assumed that extinction actively suppressed responding. Extinction was said to have inhibitory effects, in contrast to assumed excitatory effects of reinforcement. This treatment went back to a language that had been applied to data from Pavlov's conditioning experiments (cf. Chapter 17). The effectiveness of Skinner's operant language depended in part on escaping from the implications of those earlier usages (Skinner, 1938, pp. 96-102). Once those usages had carried over to the language of consequences, they were kept because they seemed consistent with other effects that often accompanied extinction. Thus, texts on learning tended to devote separate chapters to reinforcement and extinction rather than treating them as two aspects of a single phenomenon.

Spontaneous Recovery

  • Consider spontaneous recovery. In a typical extinction session, responding decreases as the session continues. But the rate at the beginning of the next extinction session is usually higher than it was at the end of the last one. Some hypothetical cumulative records illustrating spontaneous recovery are shown in Figure 5-8. Responding at the start of a session was said to have recovered spontaneously from inhibition built up by the end of the last session; it was assumed that this inhibition, actively suppressing responding, increased within sessions but dissipated between sessions.
  • Phenomena such as spontaneous recovery were taken to mean that responding that had been reduced by extinction was somehow "there all the time but inhibited" (R. L. Reid, 1958).
  • Various accounts of extinction were formulated in terms of inferred processes such as inhibition, frustration, interference or fatigue (Kimble, 1961), but when a response was said to be inhibited in extinction, there was no way to measure what was doing the inhibiting. Later we'll consider a different variety of inhibition in which both what is inhibited and what does the inhibiting are clearly specified.

Presession Conditions

  • It wasn't necessary to assume that extinction required active suppression. For example, the effects of presession conditions such as handling may make the start of a session different from later times. If so, effects of extinction late in one session might not transfer to the start of the next session.
  • Kendall (1965) reasoned that the usual pattern of response rates in extinction sessions could be reversed under the right conditions. The key pecks of three pigeons were first reinforced in 1-hr sessions. Repeated 1-min sessions of extinction followed. The first long extinction session came only after responding had reliably decreased to zero in the brief sessions. Within a few minutes, each pigeon began to respond. Until this session, responding had never extinguished at times later than 1 min into a session; responding occurred at these later times when the opportunity was finally available. In a sense, Kendall had demonstrated spontaneous recovery within a session rather than at its start.

Regression or Resurgence

  • Another example of recovery of extinguished responding has been called regression or resurgence (Epstein & Skinner, 1980; Keller & Schoenfeld, 1950, pp. 81-82).
  • Suppose a rat's chain pulls are extinguished and then lever presses are reinforced. If the lever presses are later extinguished the previously extinguished chain pulls are likely to reappear. By analogy to clinical terminology, the phenomenon suggests regression from current behavior (lever presses) to older behavior that was once effective (chain pulls). It is of considerable practical significance. For example, in training show dogs it is sometimes necessary during a show to withhold reinforcers that had been available during training, but this must be done carefully to prevent regression to behavior that had been common earlier but would now spoil the dog's performance.

Response-Reinforcer Contingencies and Reinforcer Deliveries

  • Contingencies expressed in terms of probability relations between responses and their consequences can be summarized in graphic form much like those between stimuli and the responses they elicit (Figure 4-2). The coordinate system is illustrated in Figure 5-9. The y-axis shows the probability of a stimulus given a response, p(S/R); the x-axis shows its probability given no response, p(S/no R). Relative to Figure 4-2, the S and R terms have been reversed. The earlier figure showed effects of stimuli on responses; this one shows effects of responses on stimuli.
Probability Examples
  • At A, the probability of the stimulus is high given a response but is otherwise low, as when a rat'