Chapter 5 Operant Conditioning: Learning the Outcome of Behaviors (Terms)

altruism

In behavioral economics, an action or a behavior that provides benefit to another at the expense of some cost to the actor

basal ganglia

A brain region that lies at the base of the forebrain and includes the dorsal striatum

behavioral addiction

Pathological addiction to a behavior (rather than to a pharmacological substance)

behavioral economics

The study of how organisms allocate their time and resources among possible options

bliss point

In behavioral economics, the allocation of resources that maximizes subjective value or satisfaction

chaining

An operant conditioning technique in which organisms are gradually trained to execute complicated sequences of discrete responses

concurrent reinforcement schedule

A reinforcement schedule in which the organism can make any of several possible responses, each of which may lead to a different outcome (each of which may be reinforced according to a different reinforcement schedule)

continuous reinforcement schedule

A reinforcement schedule in which every instance of the response is followed by the reinforcer

cumulative recorder

A device used for recording responses in operant conditioning, designed in such a way that the height of the line it draws represents the total (cumulative) number of responses made up to a given time

delay discounting

The progressive reduction (or discounting) of the subjective value of a reward the longer it is delayed.

differential reinforcement of alternative behaviors (DRA)

A method used to decrease the frequency of unwanted behaviors by instead reinforcing preferred alternative behaviors

discrete-trials paradigm

An operant conditioning paradigm in which the experimenter defines the beginning and end points of each trial

discriminative stimulus (S) D)

In operant conditioning, a stimulus indicating that a particular response (R) may lead to a particular outcome (O).

dorsal anterior cingulate cortex (dACC)

A brain region that may play a role in the motivational value of pain

dorsal striatum

A region of the basal ganglia that is important for stimulus–response learning

drive reduction theory

The theory, proposed by Clark Hull, that all learning reflects the innate, biological need to obtain primary reinforcers

endogenous opioid

Any of a group of naturally occurring neurotransmitter-like substances that have many of the same effects as heroine, morphine, and other opiate drugs; may help signal the hedonic value of reinforcers in the brain

fixed-interval (FI) schedule

A reinforcement schedule in which the first response after a fixed amount of time is reinforced; thus, FI 1-min means that reinforcement follows the first response made after a 1-minute interval since the last reinforcement

fixed-ratio (FR) schedule

A reinforcement schedule in which a specific number of responses must occur before a reinforcer is delivered; thus, FR 5 means that reinforcement arrives after every fifth response

free-operant paradigm

An operant conditioning paradigm in which the animal can operate the apparatus as it chooses in order to obtain reinforcement (or avoid punishment)

hedonic value

In operant conditioning, the subjective “goodness” or value of a reinforcer

incentive salience hypothesis

The hypothesis that dopamine helps provide organisms with the motivation to work for reinforcement

insular cortex (insula)

A brain region that is involved in conscious awareness of bodily and emotional states and that may play a role in signaling the aversive value of stimuli

matching law of choice behavior

The principle that an organism, given a choice between multiple responses, will make each response at a rate proportional to how often that response is reinforced relative to the other choices

motivational value

In operant conditioning, the degree to which an organism is willing to work to obtain access to a stimulus

negative contrast

The phenomenon in which the reinforcing value of one reward is reduced because a better reward is expected

negative punishment

A type of operant conditioning in which the response causes a desirable element to be “subtracted from” the environment; over time, the response becomes less frequent.

negative reinforcement

A type of operant conditioning in which the response causes an undesirable element to be “subtracted from” the environment; over time, the response becomes more frequent.

operant conditioning

The process whereby organisms learn to make or to refrain from making certain responses in order to obtain or avoid certain outcomes

orbitofrontal cortex

A brain region that is important for learning to predict the outcomes of particular responses

partial reinforcement schedule

A reinforcement schedule in which only some instances of the response are followed by the reinforcer

pathological addiction

A strong habit that is maintained despite harmful consequences

positive punishment

A type of operant conditioning in which the response causes an undesirable element to be “added” to the environment; over time, the response becomes less frequent

positive reinforcement

A type of operant conditioning in which the response causes a reinforcer to be “added” to the environment; over time, the response becomes more frequent

post-reinforcement pause

In operant conditioning, a brief pause in responding that follows delivery of the reinforcer

primary reinforcer

A reinforcer, such as food, water, or sleep, that is of biological value to an organism

punisher

A consequence of behavior that leads to decreased likelihood of that behavior in the future

punishment

The process of providing outcomes (punishers) that lead to decreased probability of a particular behavior occurring in the future

reciprocal altruism

In behavioral economics, the principle that one organism may donate time or resources to help another in the expectation that the other will return the favor later on

reinforcement

The process of providing outcomes (reinforcers) that lead to increased probability of a particular behavior occurring in the future

reinforcement schedule

A schedule that determines how often reinforcement is delivered in an operant conditioning paradigm

reinforcer

A consequence of behavior that leads to increased likelihood of that behavior in the future

secondary reinforcer

A reinforcer that initially has no biological value but that has been paired with (or predicts the arrival of) a primary reinforcer

self-control

An organism’s willingness to forgo a small immediate reward in favor of a larger future reward

shaping

An operant conditioning technique in which successive approximations to the desired response are reinforced

Skinner box

A chamber used for operant conditioning and designed so that reinforcement or punishment is delivered automatically whenever an animal makes a particular response (such as pressing or releasing a lever)

substantia nigra pars compacta (SNc)

A part of the basal ganglia that contains dopamine-producing neurons projecting to the striatum

token economy

An environment (such as a prison or schoolroom) in which tokens function the same way as money does in the outside world

variable-interval (VI) schedule

A reinforcement schedule in which the first response after a fixed amount of time, on average, is reinforced; thus, VI 1-min means that reinforcement follows the first response made after a 1-minute interval, on average, since the last reinforcement

variable-ratio (VR) schedule

A reinforcement schedule in which a specific number of responses, on average, must occur before a reinforcer is delivered; thus, VR 5 means that, on average, every fifth response is reinforced