Exam 2 Learning and Behavior

What is Thorndike’s law of effect

statement that behavior is function of consequences

State he two types of effects that Thorndike said that behavior had.

satisfying state of affairs

bx that produces this WILL be repeated

annoying state of affairs

bx that produces this NOT repeated

List the three necessary characteristics of a reinforcer

behavior must have consequence

Behavior must increase in strength (more frequent)

Increase in behavior must be result of consequence

Define and give examples of positive reinforcement

behavior followed by appearance of, or increase in intensity of, a stimulus

effect of positive reinforcer = strengthen behavior that precedes it

ex: putting money in machine, get food, more likely to put money in machine

Define and give examples of negative reinforcement

behavior followed by removal of, or decrease intensity of, a stimulus

effect of negative reinforcer = strengthen behavior that precedes it by removal of stimulus

ex: additional assignment or chores, annoying sound of alarm

Escape (Negative Reinforcement)

stimulus is removed after contacted

ex: turn off alarm when goes off

Avoidance

previously experience stimulus not contacted

ex: turning off alarm before it sounds

primary reinforcers

naturally or innately reinforcing, phylogenetically significant events

does not depend on association with other reinforcers to increase behavior

ex: food, water, sex, drugs, relief from heat, cold & pain

less effective in humans because we are rarely deprived of them

lose their effectiveness quickly (satiation)

Secondary reinforcer

depend on their association with other reinforcers

a.k.a conditioned reinforcers

e.g., praise, points, money

generalized reinforcers

paired with variety situations, e.g., money

natural reinforcers

event that follow automatically/naturally from behavior

behavior provides it’s own reward

ex: itch relief from scratching, “high” from exercise, relief from stretching

not arranged by someone else for the purpose of modifying a behavior

contrived reinforcers

arranged by someone for the purpose of modifying behavior

ex: praise, recognition, awards, promotions, money

used when explicitly teaching a new skill

State how contingency between behavior and its consequences affect operant learning.

dependency; degree of correlation between behavior & its consequence

likelihood that consequence will follow behavior

greater contingency = faster learning

State how contiguity between behavior and its consequences affect operant learning.

closeness in space or time, usually temporal

greater contiguity = faster learning

1 second rule

immediate reinforcement is more effective because no other behaviors should occur

State how characteristics of reinforcers affect operant learning.

magnitude - all else being equal, larger reinforcer more effective than smaller

relation not linear - greater increases = less benefit

qualitative differences

the preference level or desirability of the reinforcer to the learner (e.g., highly preferred food vs. less preferred food)

ex: taste - cake versus broccoli

State how characteristics of behavior affect operant learning.

behavior involving skeletal muscles (instead of smooth muscles or glands) easier to reinforce

task difficulty varies across species

ex: easier to train birds that eat seeds to peck a disc

motivating operations

anything that establishes conditions that change the effectiveness of a reinforcer

what are the two types of motivating operations

establishing and abolishing operations

establishing operations

increase reinforcer’s effectiveness

ex: food deprivation

abolishing operations

decrease reinforcer’s effectiveness

ex: pre-feeding before session

Hull’s drive reduction theory

animals & people have motivational states called drives

ex: drive for food, drive for water

reinforcers is event that reduces 1 or more drives

theory works with primary reinforcers less with secondary

drive is what we call a hypothetical construct

ex:

The Premack Principle

high probability (high-p) behavior reinforces low - probability (low-p behavior)

reinforcers are access to behavior not stimuli

ex: act of eating, not food itself, is the reinforcer

some behaviors more likely than others, different relative values

no use of physiological drives

no distinction between primary & secondary reinforcers

Response deprivation Theory

behavior reinforcing to extent individual prevented from engaging in it at normal frequency (e.g., deprived)

What matters is extent behavior occurs baseline rate

Describe Soloman and Wynne’s shuttle box experiment

dog in 1 compartment

light off, 10 s later shock through floor

jumps hurdle to 2nd compartment

light on in 2nd compartment no shock

later light out 10 s later shock on

dog jumps hurdle

soon, dog jumps instant shock began

then, dog jumps when light goes out

Advantages of Premack Principle

strictly empirical

no hypothetical constructs

disadvantages of Premack Principle

difficulty explaining secondary reinforcers

low-p behavior will reinforce high-p behavior if prevented from engaging in low-p behavior for sometime

disadvantage of Response Deprivation Theory

does not explain reinforcing effects of “yes”, “right”, “correct” etc.

state the question Soloman and Wynne’s shuttle box experiment raised about explaining avoidance.

Question on avoidance: how can something that is NOT happening control behavior

Describe the two-process theory of avoidance.

two processes involved in avoidance Pavlovian & operant

escaping shock - negatively reinforcing

avoiding shock

shock unconditional stimulus for fear

dark chamber paired with shock

dark chamber becomes conditional stimulus for fear

jumping hurdle reinforced by escape from dark chamber

many test support two-process theory

Describe the one-process theory of avoidance.

only process in avoidance is operant

both escape & avoidance reduce aversive stimulation

In Herrnstein & Hineline study rats always got shocked, but lever pressing reduced the frequency of shock

Describe and give examples of shaping

reinforcement of successive approximations of desired behavior

ex: teaching person to swing golf club, teaching dog to roll over

start with closest approximation of behavior (precursor behaviors)

as that step is mastered, up the ante

reinforce more advance behavior

stop reinforcing easier behavior

what are the best tips for make sure that shaping is

successful?

reinforce in small steps

always start with what organism can already do

immediate reinforcement ( 1 - s rule)

small reinforcers

use best approximation available (“go with it”)

Back up when necessary

Describe and give examples of a behavior chain.

a series of related behaviors the last of which produces reinforcement

ex: dialing a phone number

ex: eating out at restaurant

segments of chain completed in particular order

chaining

procedure of establishing a behavior chain

two types: forward chaining & backward chaining

chaining starts with task analysis

task analysis

the procedure of identifying the component elements of a behavior chain

each component element of chain = link

always reinforce correct sequence of links

forward chaining

trainer begins by reinforcing performance of 1st link in chain

repeated until 1st link performed without hesitation

trainer then requests subject to perform 1st and 2nd link in order

not always most effective

backwards chaining

begin by reinforcing performance of last link in chain

work backwards towards first link

e.g., if 10 links, train link 10 first. then train 9 & 10. then 8, 9, & 10.

chain is NEVER performed backward

more effective than forward chaining

with chaining, only last link produces reinforcer

Problem

situation in which reinforcer is available but behavior necessary to produce is not

Describe the studies by Harlow and state how it showed that problem solving could be understood in terms of operant conditioning.

Monkey with food under 1 of 2 lids

lids varied somehow (e.g., color, size, shape

in one trial series, prize under larger lid

another trial series, under square lid

success on second trial in series if selected same lid produced prize on 1st trial

gradual learning, no insight

same results with children ages 2 -7

Harlow’s monkeys solved problems better over time because correct choices were reinforced with food, showing that learning and problem solving can be explained by operant conditioning and reinforcement

Describe the studies by Epstein, and state how it showed that problem solving could be understood in terms of operant conditioning.

taught pigeons to:

push small box toward green spot in chamber

climb on box beneath toy banana and peck it

then hung banana from ceiling & placed box elsewhere in chamber

did NOT train bird to push box toward banana

yet pigeons learn to do it

pigeons not trained to push box don’t learn it

insight depends on previous reinforcer of separate behaviors required for solution

Epstein’s studies showed that what looks like insightful problem solving can actually be explained by operant conditioning, where animals combine behaviors that were previously reinforced to solve a new problem.

Describe how superstition can be understood in terms of operant conditioning.

superstitious behavior - any behavior that occurs repeatedly even though does not produce reinforcer that maintains it

no contingency between behavior and consequences

e.g., Skinner’s pigeons when food delivered at fixed intervals, regardless of behavior

birds later turned counterclockwise, raised heads toward corner of cage, swung heads

product of accidental/coincidental reinforcement

This explains human superstitions too. People may repeat behaviors (like wearing “lucky” clothes or doing certain rituals) if something good happened after they did it, even if the two events are not truly related.

✅ In short: Superstitions form when a behavior is accidentally reinforced, leading someone to believe that the behavior causes a reward.

Describe how creativity can be understood in terms of operant conditioning.

involves novelty, original behavior

to get novel behavior reinforce novel behavior → increase creativity

creativity is characteristic of behavior that can be reinforced

e.g., porpoises produced more novel behaviors after unusual tricks were reinforced

e.g., relative to controls, school children came up with creative uses for items when novel responses were reinforced

In operant conditioning, individuals produce many different behaviors or ideas through variation and trial-and-error. When one of those behaviors leads to a reward or successful outcome, it is reinforced and becomes more likely to occur again.

Over time:

1. A person tries many possible actions or ideas.

2. Some ideas are rewarded or successful.

3. Those successful responses are reinforced.

4. The reinforced behaviors are repeated and refined.

This process can produce behaviors that appear new or original, which we describe as creativity.

Creativity can emerge when many behaviors are tried and the ones that work are reinforced, gradually shaping new and effective solutions through operant conditioning.

Describe how Seligman’s study demonstrates learned helplessness.

dog in harness paired tone with shock

inescapable shock

next, put dog in shuttle box, sounded tone, shock on 1 side of box

dogs don’t jump; instead whimper, lie down, take shock

make escape easier

still take shock

Seligman’s study demonstrated learned helplessness by showing that animals exposed to uncontrollable stress eventually stop trying to escape or change the situation, even when escape becomes possible.

learned industriousness.

reinforce high level effort & persistence despite difficulty

learned helplessness

tendency to give up on problems because of previous experience with insoluble problems

schedule of reinforcement

a rule describing the delivery of reinforcement for a behavior

continuous reinforcement

every response produces reinforcement

ex: each time answer correctly, get a point

ex: every time put $ in vending machine, get item

1 response: 1 reinforcer

produces rapid increases in rate

ideal for teaching new behavior

rare in natural world

intermittent reinforcement

only some (not all) responses produce reinforcement

Define and give examples of fixed ratio

behavior reinforced when it occurs fixed number of times

number after fixed ratio = ratio requirement

ex: fixed ratio has ratio requirement of 5; 5 responses required for 1 reinforcer

after reinforcer delivery counter resets

Fix ratio example piecework (worker paid for units produce)

Pause at start of ratio (after previous reinforcer) followed by rapid response rate (break and run pattern)

Post reinforcement pause, a.k.a. pre-ratio pause, between ratio pause

Define and give examples of variable ratio schedule. Describe the patterns of behavior that this schedules produce.

vary response requirement around some average

produces steady performance, shorter pauses, consistent run rates (pattern)

pauses influenced by average ratio, lowest ratio

VR usually more behavior than comparable FR (pause shorter)

common in nature

commission, gambling, predatory bx

Define and give examples of fixed interval schedule. Describe the patterns of behavior that this schedules produce.

reinf. delivered following bx but only when bx occurs after period of time

interval starts, timer set, once interval collapses, next response → reinforced

e.g., FI 2-min, rat gets food for 1st response after 2 min

pattern: produces pauses & scallop - shaped curve

e.g., checking on cookies in the oven

Define and give examples of variable interval schedule. Describe the patterns of behavior that this schedules produce.

bx reinforced 1st time occurs after interval elapses, but interval varies around specific average

e.g., VI 30” (indiv intervals of 20, 50, 40, 10 but avg = 30)

pattern: produces moderate/high steady run rates

longer wait, greater probability response will pay off

e.g. hunter waiting for game, checking phone for text, refreshing courseweb for grades

Define and give examples of extinction

previously reinforced bx never reinforced

causes a gradual decrease in bx

1. Classroom example
A student used to shout out answers because the teacher gave attention.
If the teacher starts ignoring the shouting and only calls on students who raise their hands, the shouting will eventually stop.

Define and give examples of extinction burst

immediate sudden increase in in intensity/rate responding @ start of EXT

Classroom behavior
A student calls out answers for attention.
When the teacher ignores it, the student may call out even more frequently or loudly before stopping.

Define and give examples of resurgence

reappearance during EXT of some other previous reinforced bx

Classroom example
A student used to call out answers (old behavior).
They are taught to raise their hand instead and get praised (new behavior).
If the teacher stops praising or calling on them for raising their hand, the student may go back to calling out.

Define and give examples of spontaneous recovery

after EXT, brief reappearance of extinguished bx

A student used to interrupt a lot but stopped after the teacher ignored the behavior (extinction).
A few weeks later, the student suddenly starts interrupting again for a short time—this is spontaneous recovery.

Define and give examples of fixed duration schedule

requires continuous performance of bx for period time

e.g., 20 minutes of piano practice = FD 20 min

Define and give examples of variable duration schedule

requires continuous performance of bx for period time on average

e.g., VD 30 min of studying

Define and give examples of fixed time schedule

reinforcer delivered after given period of time w/o regard to bx

no response requirement

Classroom example
A teacher gives the class a short break every 30 minutes, regardless of how students are behaving.

Define and give examples of variable time schedule

reinforcement delivered at varying intervals regardless of what organism does

no response requirement

Classroom example
A teacher gives out praise or small rewards at random times during class, not based on specific student behaviors.

Define and give examples of differential reinforcement of high rate

Bx reinforced if occurs minimum # times or more in given period

increases rate of bx

A teacher tells students they will earn a reward if they answer at least 5 questions during class.
Students who meet or exceed that number get reinforced.

Define and give examples of differential reinforcement of low rate

bx reinforced only if it occurs no more than specified # times in given period

Classroom talking
A student talks out of turn too often.
They earn a reward if they interrupt no more than 2 times during a class period.

Describe progressive ratio schedules

requirement for reinf. increases in predetermined way immediately after each reinf.

e.g., rat presses lever 5 times for first reinforcer, 10 timesfor the second, 15 times of the third, and so fourth

Break point

point at which rate of bx falls dramatically or stops

Define ratio strain and describe how it can be avoided by stretching the ratio.

disruption of pattern of bx due to leaning ratio too quickly

Ratio strain can be avoided by stretching the ratio, which means gradually increasing the number of responses required for reinforcement instead of increasing it all at once.

If the requirement jumps too quickly (like from 5 responses to 50), the behavior may stop because it feels too difficult. But when the increase is slow and step-by-step, the person or animal has time to adjust and stays motivated.

Describe and give examples of multiple schedule.

2 or more simple schedules (components) alternate

each component assoc. w/own stimulus (i.e. signaled)

components can be discriminated

e.g., pigeon’s key pecks reinforced on a FR 10 schedule when light is green and a VR 10 schedule when light is red

Describe and give examples of mixed schedule.

2 or more simple schedule (components) alternate

no components associated w/own stimulus, (i.e., unsignaled)

components cannot be discriminated

Studying example
A student sometimes rewards themselves after completing 10 problems (fixed ratio) and other times after studying for 20 minutes (fixed interval), but they don’t follow a clear pattern.
Since there’s no signal, they keep working consistently.

Describe and give examples of chain schedule.

compound schedule consisting of two or more simple schedules, each associated w/own stimulus presented in series

reinf. only after last schedule completed

simple schedules called links

Classroom assignment

• Link 1: Teacher gives instructions (stimulus), student listens

• Link 2: Student completes worksheet

• Link 3: Student turns it in

• Reinforcement: Grade or praise only after everything is done

Each step has its own cue, but the reward comes at the end.

Describe and give examples of tandem schedule.

compound schedule consisting of two or more simple schedules

reinf. only after last schedule completed

simple schedules called links

unsignaled

Classroom example
A student must:

• Complete a certain number of problems

• Then continue working for a certain amount of time

But there is no clear signal when the requirement changes from “number of problems” to “time spent.”
They just have to keep working until they eventually get rewarded.

Describe and give examples of concurrent schedule.