3 - Instrumental Conditioning (NOT INCLUDING TEXTBOOK)

Chaining

Used to develop a sequence of desired behaviours; each correct response is reinforced with the chance to perform the next step in the sequence; step by step, a chain of responses is built leading to the final sequence of behaviours.

Continuous Reinforcement (CRF)

A reinforcer follows every correct response made by the individual.

Contrast Effects

Changes in the value of a reward lead to shifts in response rate; negative contrast: a response originally receiving a high reward is shifted to a lower reward, resulting in reduced responding; positive contrast: a response originally receiving a low reward is shifted to a higher reward, resulting in increased responding.

Cumulative Recorder

Records the cumulative response rate during an instrumental conditioning experiment.

Discriminative Stimuli

A signal to the organism when a given response-reinforcer relationship is valid; can indicate either the presence (S+) or absence (S−) of the relationship.

Fixed Interval (FI) Schedule

The first correct response that occurs after a fixed interval of time is reinforced.

Fixed Ratio (FR) Schedule

Reinforcement follows after a fixed number of responses.

Law of Effect

A response followed by a satisfying effect is strengthened and likely to occur again in that situation; a response followed by an unsatisfying effect is weakened and less likely to occur again in that situation.

Mirror Neuron

A cell that responds in the same way when performing an action as it does when the organism possessing that cell observes someone else perform the action or even imagines performing the action; believed to play a key role in observational learning.

Operant Chamber

Also referred to as a Skinner Box; a special chamber with a lever or other mechanism by which an animal could respond to produce a reinforcer.

Overjustification Effect

A newly introduced reward for a previously unrewarded task can alter an individual’s perception of that task; a task previously regarded as having intrinsic value (an activity pursued because it is, in and of itself, rewarding) becomes viewed as work with extrinsic value (an activity undertaken only because it leads to reward coming from other sources).

Partial Reinforcement (PRF)

The reinforcer follows only some of the responses.

Post-Reinforcement Pause

A period during which the organism momentarily stops responding before starting up again; occurs after reinforcement on a fixed ratio schedule.

Primary Reinforcer

A reinforcer with intrinsic value, such as food, water, or a mate.

Secondary Reinforcer

A reinforcer that can be exchanged for a primary reinforcer; money is the most commonly used for humans.

Shaping

Used when a desired behaviour is too complex for a subject to discover on their own in a single step; the behaviour is broken down into smaller, easier steps, eventually leading to the more complex behaviour.

Variable Interval (VI) Schedule

Reinforcement follows the first correct response to occur after a variable interval of time has passed; the average time required characterizes a particular VI schedule.

Variable Ratio (VR) Schedule

Reinforcement follows after a variable number of responses have been completed; the average number of responses required characterizes a particular VR schedule.

instrumental conditioning

learning a contingency b/w a behaviour and a consequence

Thorndike’s experiment

placed a hungry cat in a box with food outside

only way the cat could escape was by pulling a rope

Thorndike hypothesized that as trials went on, the cat would get progressively faster at escaping

stamped in

favourable behaviours are “stamped in” because they are followed by favourable consequences

stamped out

unfavourable behaviours are “stamped out” because they did not lead to favourable consequences

Law of Effect

• behaviours w/ positive consequences are stamped in

• behaviours w/ negative consequences are stamped out

classical vs instrumental conditioning

instrumental conditioning considers overt behaviours that are operated by an actor, leading to a reinforcer

instrumental conditioning is also referred to as

operant conditioning

reinforcer

• any stimulus that is presented after a response that impacts the frequency that the response is performed at

reward training

presentation of a positive reinforcer

• leads to increase in behaviour

punishment training

presentation of a negative reinforcer

• leads to decrease in behaviour

omission training

removal of a positive reinforcer

• leads to decrease in behaviour

escape training

removal of a negative reinforcer

• leads to an increase in behaviour

  • e.g. child is forced to do the dishes daily, but told if they do their homework every day, they can skip the chore on the weekend

to have the best effect on the behaviour, a reinforcer should be presented/removed when?

directly after

the response rate for a given behaviour can be visualized using a ________ _______

cumulative recorder

auto shaping

learning without direct guidance

• e.g. if you put a pigeon in a cage, and every time it pecks the keyhole a seed gets released, the pigeon will eventually realize the contingency w/o any guidance

shaping by successive approximation

used for behaviours that are too complex to be autoshaped

• smaller approximations that eventually build up to the full response

chaining

technique used to develop a sequence of behaviours

• each behaviour is reinforced w/ the opportunity to perform the next behaviour in a sequence

  • e.g. learning the alphabet → each letter informed you of what letter came next

in shaping by successive approximation, a behaviour is reinforced only if it is a closer approximation of the desired final behaviour than the behaviour last reinforced. This is also thought as reinforcing on the basis of __________.

improvement

Chaining reinforces the behaviour so long as it is performed in a defined ____.

order

discriminative stimulus (SD/S+)

a contingency is valid

e.g. environment of parents home becomes an SD for vegetable eating behaviour (for a girl who is rewarded w/ dessert every time she eats her vegetables at her parents house)

Sδ/S-

a contingency is not valid

e.g. environment of grandparents home becomes an Sδ for vegetable eating behaviour (the girl will learn that under these conditions, eating vegetables will not lead to a dessert reward)

SD generalization gradient

literally same thing as the generalization gradient in classical conditioning

when the SD and Sδ are in the same modality (e.g. colours of light or tones), introduction of training w/ an Sδ leads to better stimulus _________ and fine tuning of behaviour that is more sharply directed to the SD.

discrimination

CS+ vs SD

CS+: paired w/ US and elicits a response reflexively

• response is involuntary + automatic

SD: paired w/ response reinforcer outcome; SD DOES NOT reflexively elicit the response

• SD sets the occasion for a voluntary response

CS- vs S-

CS-: presents the absence of the US

• eventually leads to learning of an inhibitory response

S-: signifies that response reinforcer contingency is not valid

continuous reinforcement

a response leads to a reinforcer in every trial

partial reinforcement

a response leads to a reinforcer in some trials

ratio schedule of reinforcement

based on the # of responses made by a subject, which determines when reinforcement is given

e.g. the pigeon on an FR-1 schedule is rewarded with food for each pecking response

the pigeon on an FR-2 schedule is rewarded with food for every 10th pecking response

interval schedule of reinforcement

based on the time since the last response that was reinforced

e.g. the pigeon on an FI-1 min schedule is rewarded w/ food for the first pecking response after a 1-minute period. Over an hour, the pigeon has the potential to earn 60 food pellets

on the FI-10 min schedule is rewarded w/ food for the first pecking response after a 10-minute period. Over an hour, the pigeon has the potential to earn 6 food pellets.

ratio and interval schedules can be either ____ or ______

fixed, variable

fixed schedule

constant schedule

variable schedule

random schedule

e.g. for VR-10 schedule, the pigeon must peck an average of 10 times over all the trials to get the food, but each trial the pigeon doesn’t necessarily need to peck exactly 10 times

FR vs FI vs VR

F - fixed

R - ratio

I - interval

V - variable

4 basic schedules of reinforcement

1. fixed ratio (FR)

2. fixed interval (FI)

3. variable ratio (VR)

4. variable interval (VI)

ratio strain

caused by a stingy FR schedule (too many responses required for the reward, e.g. the pigeon needs to peck 500 times before they get the food…too many times)

subject will stop responding

pause and run pattern

type of cumulative record that participants placed on a fixed ratio schedule display

• following reinforcement, the participant will pause with inactivity before beginning the next run of respondinga

a lack of motivation leads to procrastinating behaviour in a ____ ____ schedule

fixed ratio

a cumulative record of responses reinforced on a variable ratio schedule looks like what?

a VR-10 schedule will have a ______ slope than a VR-40 schedule

steeper

fixed interval schedules produce a cumulative record with a characteristic _____ pattern. Following reinforcement, there is a ____ period, in which responding drops, then slowly starts picking up again and peaking just before the next reinforcement is scheduled to be delivered following a response

scallop, lull

variable interval schedules produce a cumulative record with a very _____ rate

steady

learning is best on a _____ rather than continuous reinforcement schedule

partial

variable schedules are more resistant to extinction compared to ____ schedules

fixed

drug effects with repeated administration lead to ______

tolerance

if a drug is given to someone in the same vs different environment, how would the person react (compared to the control which is giving the drug in the same environment)? What about giving a mock drug?

same: drug would have same effect

different: drug would have more effect

mock drug: drug would have less effect

autoshaping

learning a simple behaviour without external guidance

overjustification effect

newly introduced reward for a previously unrewarded task (task was previously viewed as something that was rewarding in itself, but now is only regarded rewarding because there is an actual reward at the end)

3 ways to do instrumental conditioning

1. chaining

2. autoshaping

3. shaping by successive approximation