PSY 3103: Final Exam

Lecture 6-12

Thorndike (1874-1949)

* Was interested in animal intelligence
* Cautioned abt interpreting animal behaviour w/o critical analysis
* Warned against:
* Using anecdotes abt animals as facts
* Assuming animal behaviour was intelligence

Thorndike’s Law of Effect

* **Responses to a situation that are followed by satisfaction are strengthened; responses that are followed by discomfort are weakened**
* Reward responses are increased
* Punished responses decrease and possibly disappear\*
* Wasn't sure how to handle punishment but felt it was unique and different from reward

Skinner (1904-1990)

* Skinner a student aware of Thorndike's law of effect
* Skinner followed current thinking in psych @ the time: **behaviour could best be analyzed as though it were a reflex**
* Realized a scientific analysis of behaviour required finding a procedure yielding regular patterns of behaviour, i.e., well-controlled experimental conditions
* Set out to develop such a procedure

Skinner Box

* Rat is **able** to earn food pellets by pressing a response lever or bar
* *Free operant procedure*
* Rate of behaviour is controlled by the conditions in the box
* Many of the principles of operant conditioning were discovered w/ the use of the Skinner box
* Free operant paradigm - the animal is allowed to move around, more methodological

2 Types of Behaviour

1. **Involuntary/Reflexive Behaviour**
* Classical conditioning
* *Elicited behaviours controlled by preceding stimuli*
* i.e., involuntary behaviour like reflexes, where you're not able to ctrl much and a stimulus from that environment pulls that out of you
2. **Voluntary/Operant Behaviour**
* Operant conditioning
* *Voluntary behaviours influenced by consequences*

Operant Conditioning

* Consequence based


* **Behaviours that lead to favourable outcomes are more likely to be repeated than those that do not lead to favourable outcomes**
* Skinner deemed consequences as being satisfiable or annoying but he felt this interpretive piece was not very scientific

Operant Conditioning Process: ABC’s

1. **A discriminative stimulus (an** __**A**__**ntecedent event)**
* Not always present and not needed
* Difficult to find a situation where an antecedent doesn't exist tho bc there are often environmental cues
2. **A response (a** __**B**__**ehaviour)**
* Must be present
3. **The consequence (a** __**C**__**onsequence)**
* Must be present

Operant Behaviours

* **Operant behaviours (operants) are 'classes' of responses** = actions that can be performed in many ways


* **A behaviour is considered an operant if it results in a consequence AND the consequence affects the future probability of emitting the operant**
* Do operant behaviours occur bc we choose to emit them
* not necessarily
* operant behaviour comes to be ctrlled by the contingencies of reinforcement + punishment that follow the behaviour
* there is sometimes only an illusion of voluntariness

Consequences: Reinforcers

Consequences of a behaviour that:


1. Follow a behaviour, and
2. As a result, the future probability of that behaviour **increases**

i.e., answering a question in class (R) → prof responding positively (SR)

What does ‘R’ stand for?

response

What does ‘SR’ stand for?

stimulus reinforcer

Consequences: Punsihers

A consequence of a behaviour that:


1. Follow a behaviour, and
2. As a result, the future probability of that behaviour **decreases**

i.e., answering a question in class (R) → prof bangs head against wall (SP)

What does ‘SP’ stand for?

stimulus punisher

Operant Conditioning Perspective

* Behaviour is being reinforced or punished
* NOT the person or animal
* Behaviour increases of decreases in frequency
* Consequences are **defined entirely by their effect on behaviour**
* **Reinforcers** increase the frequency of behaviour
* **Punishers** decrease the frequency of behaviour

Reinforcement & Punishment

* *The* ***process or procedure*** *by which a certain consequence changes the strength of a behaviour*
* **Reinforcement** = the use of praise to increase the strength of tidying up
* **Reinforcer** = the praise itself
* **Punishment** = the process of prof banging head on wall that discourages answering Q's in class
* **Punisher** = the head banging

Means of Weakening Behaviour

* Punishment is not the only way to weaken behaviour
* Once a behaviour has been strengthened by reinforcement, **withdrawal of the reinforcement** (extinction)

Discriminative Stimuli (Antecedent)

A stimulus in the presence of which responses are either reinforced or punished

Discriminative Stimuli (Antecedent): Example

son’s voice in the AM: dog whines → dog gets put outside

SD R SR

silence in AM: dog whines → dog gets scolded

SD R SP

Discriminative Stimulus for Extinction

The stimulus signals the unavailability of a previously available reinforcer

Discriminative Stimulus for Extinction: Example

lever pressing is followed by food only when a tone is sounding and NOT when a buzzer is sounding

tone: *lever press* → food pellet

SD R SR

buzzer: lever press → no food

SΔ R ----

What does ‘SΔ’ stand for?

discriminative stimulus

Operant & Classical Conditioning

A stimulus can act as both an SD and a CS

* Example:

operant conditioning:

tone: lever press → food

SD R SR

\
classical conditioning:

tone: food → *salivation*

NS US UR

tone → *salivation*

CS CR

Classical Conditioning vs. Operant Conditioning: Classical

* occurs when: **2** **stimulus (US & NS/CS) are paired**
* association required: **CS → CR**
* nature of response = *i****nvoluntary*** **(elicited by a stimulus**)

Classical Conditioning vs. Operant Conditioning: Operant

* occurs when: **a response (R) is followed by the consequence; either a reinforcing stimulus (SR) or a punishing stimulus (SP)**
* sometimes we have an antecedent/discriminatory stimulus (SD)
* association required: **R → SR** __or__ **R → SP**
* nature of response = ***voluntary (*****emitted by the organism)**

4 Types of Contingencies

* positive
* negative
* reinforcement
* punishment

Determining Type of Contingency

1. Does the consequence serve to strengthen or weaken the behaviour?
* **Reinforcement or punishment**
2. Does the consequence consist of something being presented or withdrawn?
* **Positive or negative**

Negative vs. Positive

* Behaviour that results in…
* *Something being removed or subtracted =* ***negative***
* *Something being presented or added =* ***positive***

Reinforcer vs. Punisher

* Behaviour has…
* Strengthened as a result of the consequence = **reinforcer**
* Weakened as a result of the consequence = **punisher**

Positive Reinforcement: Example

wave @ person → the person waves at you

R SR

Negative Reinforcement: Example

take aspirin → eliminate headache

R SR

Negative reinforcement involves 2 types of behaviour:

1. **Escape** = results in the termination (stopping of an aversive stimulus)
2. **Avoidance** = occurs before the aversive stimulus is presented, prevents its delivery

Positive Punishment: Example

swat at the wasp → get stung

R SP

Negative Punishment: Example

hit friend → friend won’t play

R SP

Immediate vs. Delayed Reinforcement

**The more immediate the reinforcer, the stronger its effect on the behaviour**

Example:

* give a treat to a child while she is still playing quietly to reinforce quiet playing, don’t give it to her after

Primary Reinforcers

* **A** ***primary reinforcer*** **is an event that is innately reinforcing**
* Tend to satisfy bio or species-type needs
* Effectiveness tied to state of deprivation/satiation
* Examples: food, water, proper temperature, sexual contact

Secondary Reinforcers

* **An event that is reinforcing bc it has been associated w/ some other reinforcer**
* Examples: good marks, nice clothes, nice car

Conditioned stimuli that have been associated w/ appetitive unconditioned stimuli (US's) can also…

function as 2ndary reinforcers

* Example: a metronome that has been associated w/ food can be a 2ndary reinforcer for the operant responses of lever pressing

What does categorizing a reinforcer depend on?

It depends on the state of the organism, but also some reinforcers can be both primary + 2ndary

* Example: fettuccini may serve as food but may also remind you of a special person you always eat it w/

Generalized Reinforcer

**A type of 2ndary reinforcer that has been associated w/ 7vral other reinforcers**

* i.e., money being assoc. w/ many things such as shelter, clothing, entertainment, etc.

Intrinsic Reinforcement

**the mere act of doing a behaviour is rewarding in itself**

* i.e.,  driving a car because you like to

Extrinsic Reinforcement

**something that is provided by an external factor to the behaviour**

* i.e., driving to get somewhere is why you'd get in the car

\

A meta-analysis found that external rewards can undermine established intrinsic motivation when:

* The reward is expected
* The reward is tangible (i.e., $ over praise)
* The reward is given for simply performing the activity

Natural Reinforcers

**Reinforcers that are naturally provided for a certain behaviour (typical)**

* Examples: food is a natural reinforcer for pecking in pigeons, $ is a natural consequence of selling merchandise

Contrived Reinforcer

**Reinforcers that have been deliberately arranged to modify a behaviour (artificial)**

* Example: receiving chocolate for tidying up

Natural & Contribed Reinforcers depend on the behaviour:

i.e., watching tv is a natural reinforcer for turning on the TV but a contrived reinforcer for studying

Natural contingencies tend to produce more efficient behaviour patterns than do contrived contingencies

Example: throwing a ball where you want it to go is more important than throwing a ball and receiving praise

Contrived reinforcers are often used to establish natural contingencies which are likely to yield more efficient behaviour patterns

Example: giving child praise for using a good technique for throwing the ball can help establish the natural contingency of being able to throw the ball where the child wants it to go, a natural reinforcer

Intrinsic reinforcers are:

always natural reinforcers

Extrinsic reinforcers can be:

either natural or contrived

Shaping

**The gradual creation of new operant behaviour through reinforcement of successive approximations to that behaviour**

* Wanting to do a behaviour but not doing it yet so we have to create this behaviour
* example: teaching a dog to catch a frisbee

Our Shaping

Most of our behaviours have, to some extent, been learned or modified through shaping

Examples of Shaping IRL

* learning to eat w/ utensils
* learning to write
* learning to dress ourselves

Schedules of Reinforcement

**Indicates what exactly has to be done for the reinforcer to be delivered**

* What response is required to obtain reinforcement
* Different response requirements can have dramatically diff effects on behaviour

Continuous Reinforcement Schedule (CRF)

Each specified response is reinforced

* it is very useful when a behaviour is 1st being shaped or strengthened

What is the quickest way to teach a new behaviour?

continuous reinforcement schedule (CRF)

Limitations of CRF

* Not usually realistic long term
* Requires continuous supervision of the subject
* Behaviour may be subject to extinction if no transition to intermittent schedule

Intermittent Reinforcement Schedule

**Only some responses are reinforced**

* After behaviour is est., often switch to intermittent schedule to maintain or strengthen it
* behaviour on intermittent schedule is more resistant to extinction

What schedule of reinforcement is more characteristic of everyday life?

intermittent reinforcement schedule

What is an intermittent reinforcement schedule made of?

* # of responses made = ratio (how many times)
* time elapsed = interval (how much time going by)

What are the 4 basic types of intermittent reinforcement schedules?

1. fixed ratio
* fixed = know exactly how many
2. variable ratio
* variable = know an average amount
* don’t know when reinforcement is coming, but we know an average
3. fixed interval
4. variable interval

Fixed Ratio (FR) Schedules

**Reinforcement is contingent upon a fixed, predictable # of responses**

* x number of responses before reinforcement is given

FR Schedules produce:

produce high rates of response until the reward is received, followed by a short post reinforcement pause

An FR 1 schedule is the same as a

CRF schedule

* i.e., Layla gets $1 when she makes her bed 7 days in a

row

Responses to FR Schedules

* Usually a high rate of response w/ a short post reinforcement pause (could be due to super predictability
* **Break and run behaviour** helps stop the post reinforcement pause that follows a response
* Increased ratio requirements = increased post reinforcement pauses (longer pauses)

“Stretching the Ratio”

**Moving from a low ratio to a high ratio requirement**

* i.e., FR 5 to FR 100


* Should be done gradually to avoid **ratio strain**
* Aka: burnout, inconsistent output, or elimination of behaviour

Ratio Strain

Disruption in responding due to a demanding response requirement

Variable Ratio (VR) Schedules

**Reinforcement is contingent upon a varying, unpredictable # of responses**

* i.e., on a syllabus, it says you will have 7 quizzes through the semester, they will be administered in class throughout the semester w/o warning

Responses to VR Schedules

* produce a high + steady rate of response
* little or no post reinforcement pause

VR Schedules & Maladaptive Behaviour

VR schedules help account for the persistence of certain maladaptive behaviours

* i.e., Parenting - parents laughing at a child swearing (kid never knows when the parent is going to laugh when they swear)

Fixed Interval (FI) Schedules

reinforcement is contiengent upon the 1st response after a fixed predictable period of time

* i.e., start looking down the road for your bus a few

mins before it's due

Responses to a fixed interval schedule consist of:

a post reinforcement pause followed by a gradually increasing rate of response as the interval draws to a close

Responses to FI Schedules

* Individual is trained to time the interval + then target their behaviour toward the end of the interval
* Short intervals = short post reinforcement pauses
* Long intervals = long post reinforcement pauses

Variable Interval (VI) Schedules

**Reinforcement is contingent upon the 1st response after a varying, unpredictable period of time**

* i.e., checking social media

Responses to VI Schedules

Produces a moderate, steady rate of response w/ little or no post reinforcement pause

4 Basic Schedules Review

* Characteristics of response patterns
* Response rate:
* high = VR, FR
* moderate = VI
* increasing = FI
* Post-reinforcement pause:
* no = VR, VI
* yes = FR, FI

Other Simple Schedules: Duration Schedules

Reinforcement is contingent on performing a behaviour **continuously** throughout a period of time

* fixed duration (FD) schedule
* variable duration (VD) schedule

Fixed Duration (FD) Schedule

* Behaviour must be performed continuously
* For a fixed, predictable period of time

Variable Duration (VD) Schedule

* Behaviour must be performed continuously
* For a varying, unpredictable period of time (average)

Duration Schedules & Human Behaviour

* Sometimes useful in modifying certain human behaviours
* They can be rather **imprecise**
* What constitutes "continuous performance of behaviour"
* Reinforcing the mere performance of an activity w/ no regard to level of performance might undermine a person's intrinsic interest in the activity

Response-Rate Schedules

**Reinforcement is directly contingent upon the organism's rate of response**

* These are differential reinforcement schedules

Differential Reinforcement

1 type of response is reinforced while another is not

3 types of response-rate schedules:

1. differential reinforcement of high rates
2. differential reinforcement of low rates
3. differential reinforcement of paced responding

Differential Reinforcement of High Rates (Response-Rate Schedules)

Reinforcement is provided for a high rate of response and not for a low rate

Differential Reinforcement of Low Rates (Response-Rate Schedules)

**Reinforcement is provided for responding at a slow rate**

* Responses that occur during the interval have an adverse effect in that they prevent the reinforcement from occurring
* i.e., children cramming all the food in their mouth very fast and to get them to slow down, we tell them if they eat their food slower and chew they can have dessert after

Differential Reinforcement of Paced Responding (Response-Rate Schedules)

Reinforcement is provided for responding neither too fast nor too slow

* i.e., in a marathon, if you go super fast at the beginning you won't finish since you need to pace yourself

Noncontingent Schedules

* **The reinforcer is delivered** ***independently*** **of any response**


* A response is not required for the reinforcer to be obtained

2 types of Response-Independent (Noncontingent) Schedules:

1. Fixed Time
* The reinforcer is delivered:
* Following a fixed, predictable period of time
* Regardless of the organism's behaviour
2. Variable Time
* The reinforcer is delivered:
* Following a varying, unpredictable period of time
* Regardless of the organism's behaviour

Noncontingent Reinforcement and Superstitious Pigeons (Skinner 1948)

Using noncontingent reinforcement schedules whatever behaviour happened to be occurring when the reinforcer delivered was strengthened

Superstitious Behaviour in Humans

* **Noncontingent reinforcement** may account for some forms of superstitious behaviour
* Behaviours may be accidentally reinforced by the coincidental presentation of reinforcement
* Hopes of reproducing that performance
* Superstitious behaviour can be seen as an attempt to make an unpredictable situation more predictable

The Upside of Noncontingent Schedules

* Noncontingent schedules can be an effective means of reducing the frequency of maladaptive behaviours
* Carl Rogers' unconditional positive regard can be seen as a form of noncontingent social reinforcement, which can indeed have beneficial effects

The Downside of Noncontingent Schedules

When a noncontingent schedule is superimposed on a response-contingent schedule, the level of response on the respondent schedule will decrease

Complex Schedules of Reinforcement

**A combination of +2 simple schedules**

they include:

* Conjunctive Schedules
* Adjusting Schedules
* Chained Schedules
* Multiple Schedules
* Concurrent Schedules

Conjunctive Schedule (Complex Schedule of Reinforcement)

**Type of complex schedule in which the requirements of +2 simple schedules must be met before a reinforcer is delivered**

* Commonplace in daily life (i.e., earning a wage for a job is an example of having to work to get money)

Adjusting Schedule

**Response requirement changes as a function of the organism's performance while responding for the previous reinforcer**

* Shaping usually works on an adjusting schedule where gradually more is required to receive the reinforcer

Chained Schedule

A sequence of +2 simple schedules

* Each schedule has its own SD
* The last results in a **terminal reinforcer**

Responses to a Chained Schedule

* The earlier links of the chain are assoc. w/ weaker responding
* The terminal reinforcer is more immediate = more influential
* The **goal gradient effect** is an increase in the strength and/or efficiency of responding as one draws near to the goal (terminal reinforcer @ the end of the chain)

Backward Chaining

* Training the final link 1st and the initial link last can make the chain more effective
* Very long chains of behaviour can be est. w/ this method
* Shaping + chaining are thus the basic means by which circus and marine animals are trained

Chaining & Human Behaviour

* Most human endeavors involve response chains, some of which are very long
* For humans, response chains are often est. through instructions
* It is best to chart your progress bc terminal reinforcers are often extremely distant, so behaviou is easily disrupted during the early part of the chain

Biological Constraints on Operant Conditioning

* instinctive drift
* autoshaping
* both are instinctive behaviours + unnecessary in the context (not rewarded and actually delays the reinforcer)