Psych of Learning - Test 2: Ch. 6-9

Instrumental Behavior

occurs because it was previously effective in producing certain consequences (aka operant behavior)

Operant conditioning

modifying behavior through changing the consequences of that behavior

Describe how operant behavior functions through the S^D R → S^R model.

S^D

• Discriminative stimulus that proceeds the response that signals that a certain consequence is now available (ex. tone signals that lever press will now produce food)

R

• Response that produces a certain consequence (lever pressing produces food)

S^R

• Consequences that serves to increase or decrease the probability of the response that proceeded it (consequence of food pellet increases rat’s tendency to press lever again)

What is the difference between respondent (classical) conditioning and operant conditioning?

Respondent

• elicited by antecedent stimuli (food elicits salivation)

Operant

• response is emitted by organism and maintained by the consequences (the rat emits lever presses which produce food)

What is the Law of Effect?

if response (R) in presence of stimulus (S) is followed by a satisfying event, association between S and R becomes strengthened

if the response is followed by an annoying event, the S-R association is weakened

A punisher ——- behavior. A reinforcer ——— behavior.

weakens, strengthens

What is the three-term contingency (ABC)

• Antecedent - salient stimulus in environment (thing or event that existed before or logically proceeds another)

• Behavior - response to antecedent

• Consequence - addition or removal of stimulus from environment

What are examples of unconditioned reinforcers?

Biologically determined

• Food

• Water

• Oxygen

• Escape from heat or cold

What is a conditioned reinforcer?

• A previously neutral stimulus that has been repeatedly paired with an established reinforcer

• Will function as a reinforcer

• Ex:

  • Parents’ smile, tone of voice, attention, praise

  • Grades, positive evaluations

  • Money

An aversive stimulus is also called a ————.

punisher

What are the four types of contingencies?

• Positive reinforcement

• Negative reinforcement

• Positive punishment

• Negative punishment

Difference between an appetitive stimulus and an aversive stimulus

Appetitive

• pleasant event

Aversive

• unpleasant stimulus

Free Operant Procedures

Allow animal to repeat instrumental response without constraint over and over again without being taken out of the apparatus until the end of an experimental session.

• animal is allowed to freely respond or not

• animal is not forced to respond

What is a schedule of reinforcement?

A program or rule that determines which behavior is followed by reinforcer

What is the difference between a dense (thick) and lean (thin) schedule of reinforcement?

Dense (thick)

• reinforcement is delivered frequently (ex. for every response there’s a reinforcement)

Lean (thin)

• reinforcement delivered rarely (ex. every 100th response gets a reinforcement)

Continuous reinforcement

Reinforcement is delivered after every instance of a behavior

• results in sharp increase in frequency of behavior over time

Partial Reinforcement

Reinforcement is not delivered after every occurrence of a behavior (e.g. intermittent reinforcement)

What are the four partial reinforcement schedules?

• Fixed Ratio (FR)

• Fixed Interval (FI)

• Variable Ratio (VR)

• Variable Interval (VI)

Shaping

reinforcing successive approximations of a target behavior

• ex. clapping as the subject gets closer to desired location

Establishing Operation (EO)

an antecedent event that makes a reinforcer more potent

• makes a behavior that produces that reinforcer more likely to occur at that time

  • e.g. deprivation

Abolishing Operation (AO)

an antecedent event that makes a reinforcement less potent

• makes a behavior that produces that reinforcer less likely to occur at that time

  • e.g. satiation

Natural vs. Contrived Consequences

Natural

• typically provided for a certain behavior; expected consequence of the behavior within that setting (ex. money is a natural consequence of selling merchandise)

Contrived

• reinforcers that have been deliberately arranged to modify a behavior; not typical consequence of that behavior within that setting (ex. giving candy to teach potty training, not a directly related reinforcer)

Ratio Schedule

Reinforcement contingent upon a fixed number of responses

• ex. FR 5: Rat must press a lever five times to obtain a food pellet

What is a fixed ratio schedule?

Reinforcement is delivered after n instances of a behavior

• FR 10 - the 10th response

• FR 100 - the 100th response

Post-Reinforcement Pause

Zero rate of responding that typically occurs just after reinforcement on fixed ratio

Ratio Run

high and steady rate of responding that completes each ratio requirement

Ratio Strain

if ratio requirement is suddenly increased, animal likely to pause periodically before completion of ratio requirement.

Habituation

reduction in dimension of response after repeated presentations of an unconditioned stimulus

• classical conditioning

• multiple presentations of US until they get used to it

Extinction

omitting the US or reinforcer

Forgetting

decline in responding occurs because of passage of time and does not require nonreinforcement of CS/instrumental response

• Memory, cognitive

Classical Extinction

• presentation of CS without the US

• results in decreased CR in response to CS

Operant conditioning extinction

• A previously reinforced behavior no longer produces reinforcement

• involves withholding consequences

Side effects of Operant Extinction

• Aggression

• Extinction burst

  • big spike in responding happening before rapidly decreasing

• Variability

• Resurgence

Extinction burst

temporary increase in frequency and intensity of responding when extinction first implemented (try harder to obtain the reinforcer)

Increase in Variability

increase in variability of a behavior (try different ways of obtaining the reinforcer)

• trying out different behaviors to get the same reinforcer

Resurgence

reappearance during extinction of other behaviors that had once been effective in obtaining reinforcement (trying an old way of obtaining the reinforcer)

Factors Affecting Resistance to Extinction

Partial Reinforcement Effect

History of Reinforcement

Magnitude of the Reinforcer

Degree of Deprivation

Previous Experience with Extinction

Distinctive Signal for Extinction

Partial Reinforcement Effect

Behavior maintained on partial reinforcement schedule will extinguish more slowly than behavior maintained on continuous schedule

History of Reinforcement

A longer history of reinforcement leads to it being hard to make extinct

Magnitude of the Reinforcer

Large reinforcers often result in greater resistance to extinction

Variable Ratio Schedule (VR)

Average number of responses required to obtain successive reinforcers

• Ex. VR 10 - Pigeon must make 10 responses to earn first reinforcer, 13 for second, 7 for third, etc.

  • Average = 10 responses per reinforcer

——— schedules = fairly steady rate

• VR

• FR

• VI

• FI

VR

Predictable pauses in the rate of responding are less likely with ——- than —— schedules.

VR, FR

• Results in extremely high and steady responding

• Easiest way to teach rapid responding

Variable Ratio Responding

What type of reinforcement usually produces a high rate of response with little or no post-reinforcement pause?

• often involved in gambling addiction

Variable Ratio (VR)

Interval Schedule

Reinforcement depends on first response after a certain amount of time has passed

Fixed Interval

• Reinforcement is delivered after the first response following a set period of time

  • Ex. FI 10 - the first bar press after 10 minutes has passed results in a food pellet

Describe what the Fixed-Interval scallop shows.

• Increase in response rate evident as acceleration in cumulative record toward end of each fixed interval

  • post-reinforcement pause followed by gradual increases rate of response

Variable Interval Schedule

Reinforcement is delivered after the first response following a random amount of time

• Subject has to respond to obtain the reinforcer with unpredictable set-up time

• ex. VI 10 - on average, the first bar press after 10 minutes has passed results in a food pellet.

  • Would vary between 1 and 20 minutes

———— schedules maintain steady and stable rates of responding without regular pauses.

• VR

• VI

• FR

• FI

VI

Motivates most vigorous instrumental behavior due to short inter-response times or relationship between response rates and reinforcement

• VR

• VI

• FR

• FI

VR

Interval between successive responses

Inter-Response Time (IRT)

Conjunctive Schedule

Requirements of two or more simples schedules must be met before a reinforcer is delivered

• e.g. wages for work are contingent upon putting in the required hours and doing the work that needs to be done during that time

Adjusting Schedule

Response requirement changes as function of performance while responding for previous reinforcer

• e.g. increasing the difficulty of material being presented in a chemistry class as students learn more

Drive Reduction Theory

• Event is reinforcing to the extent that it is associated with reduction in physiological drive

  • E.g. food is a reinforcer for going to the cafeteria because it reduces a hunger drive

Incentive Motivation

Derived from property of reinforcer vs. internal drive state

• e.g. playing a video game for the fun of it

Premack Principle

high-probability (or frequency) behavior (HPB) can be used to reinforce low-probability behavior (LPB)

• e.g. First you work, then you play

Does extinction does not erase what was originally learned?

No

Spontaneous Recovery

rest period introduced after extinction training and responding comes back; nothing specific is done during rest period to produce recovery

Resurgence

reappearance of extinguished target response when another reinforced response is extinguished

Differential Reinforcement of Other Behavior (DRO)

reinforcement of any behavior other than the target behavior that is being extinguished

• more effective than simple extinction procedures

• reduces unwanted side effects of extinction

Functional Communication Training

• A variant of DRO

• Assumes that misbehavior often occurs because the person is trying to achieve some type of reinforcer

• If so, training person to instead verbally communicate want they want should reduce the amount of misbehavior

Stimulus Control

When operant behavior is under the control of stimulus cues

Difference between escape and avoidance

Escape

• when response terminates an aversive stimulus

Avoidance

• When response prevents an aversive stimulus from occurring

Two-Factor (process) Theory of Avoidance

Classical conditioning of fear to CS + instrumental reinforcement of avoidance response through fear education

What are the problems with Two-Process Theory of Avoidance?

• Avoidance responses are extremely persistent; why do they not extinguish?

• Once the rat gets used to the procedure, it seems to show no fear but continues to avoid anyway

Why is avoidance in animals somewhat different from phobias in humans?

Avoidance conditioning in animals usually requires a few conditioning trials, but phobic conditioning in humans requires only a single trial

• avoidance conditioning in animals may eventually extinguish, but phobic conditioning in humans is extremely resistant to extinction

What are the types of punishment?

• Intrinsic Punishment

• Extrinsic Punishment

• Primary (unconditioned) Punisher

• Secondary (conditioned) Punisher

• Generalized (generalized secondary)

Intrinsic Punishment

inherent aspect of the behavior being punished; activity itself is punishing (e.g. smoking makes you feel nauseous)

Extrinsic Punishment

not an inherent aspect of behavior being punished; simply follows the behavior (e.g. being told “you smoke Is that ever disgusting”)

Primary (unconditioned) Punisher

event that is innately punishing (e.g. being jabbed with a needle)

Secondary (conditioned) Punisher

event that is punishing because of past association with other punishers (going to a doctor who often gives you a needle”

Generalized (generalized secondary) Punisher

event that is punishing because of its past association with many other punishers (e.g. a mean look from someone.)