PSYC-2600

Learning

• Change in behaviour • Occurs as result of experience • Relatively permanent • Cannot be attributed to temporary body states (e.g., fatigue)

Behaviour

• Patterns in time (movements, vocalizations, release of stress hormones) • Observable, measurable activity of an organism • Overt/External • Covert/Internal

Behaviour Modification

• Identifying behavioural problems • Applying learning principles to change the behaviour and solve the problem • Change the environment to solve the problem • Involves the systematic application of learning principles and techniques to assess and improve individuals’ overt and covert behaviours to enhance their functioning.

Moghul Emperor Akbar the Great

• 16th century • Moghul emperor Akbar the Great • Hypothesis: • Babies that do not hear speech would be unable to speak.

Plato

nature (nativist) • person’s characteristics are inborn • knowledge is innate

Aristotle

nurture (empiricist) • characteristics are learned

Darwin

• On the Origin of Species • Natural selection • Learning as an adaptive trait

Ivan Petrovich Pavlov

• From physiology to “psychic reflexes” -Laws of association -Observation of simple reflexes

Edward L. Thorndike

Law of effect... consequences

Appetitive

• wanted or approached • Often pleasurable, but not necessarily

Aversive

• avoided • unpleasant

Overt

Directly observable

Covert

Indirectly observable

Temporal contiguity

\-close in time

Spatial contiguity

\-close in space

Contingency

• Sequence of stimuli • Predictive relationship between two stimuli • A contingent on B • Extremely important aspect of learning

Habituation

• Decrease in response to a specific stimulus that is repeatedly presented • Not fatigue • Not sensory damage

Habituation Adaptive Purpose

• Ignore biologically unimportant stimuli • Conserves resources • energy • attention • If stimuli occur w/o consequences: • response diminishes

Habituation Parameters

• Stimulus frequency • Stimulus intensity • Spontaneous recovery • Reinstatement of the response • Due to the passage of time

Habituation Importance of spacing:

• rapid/close presentations = fast habituation -Short-term • Spaced presentations = slow habituation -Long-term

Generalization

Reacting to similar stimuli the same way

Sensitization

• Increase in response to a stimulus that is repeatedly presented. • Usually temporary • Stronger stimulus à longer effect • Highly susceptible to generalization

Opponent-Process Theories

• Homeostasis • Balance between habituation and sensitization • Basis of emotion • An account for drug addiction

• Emotions are biphasic • Primary reaction followed by opposite afterreaction • Primary reaction becomes weaker with repetition • After reaction becomes stronger

Opponent-Process Theory of Motivation

Proposes that emotional event elicits two competing
processes:


1. a-process (or primary process) that is directly elicited
by the event; closely correlates with emotional event
2. b-process (or opponent process) elicited by the aprocess but serves to counteract the process (and
maintain homeostasis); slow to increase and decrease

Characteristics of the opponent process

• The a-process correlates closely with the stimulus • The b-process is slow to increase and slow to decrease • With repeated stimulation, the b-process increases in both strength and duration • The a-process and b-process tend to be hedonically opposite from each other

Memory Theory (of Habituation)

• “Comparator” • Does the representation of the stimulus already exist in memory? • Active in memory = ignored

Neural Circuits

• Aplysia • Simple nervous system • Can easily trace the neuronal underpinnings

Classical Conditioning

• A learning process in which one stimulus that does not elicit a response is associated with a second stimulus that does • Consequence: first stimulus now elicits a response • An initially neutral stimulus is paired with a stimulus that elicits a response • After pairing(s), the neutral stimulus elicits the response

Classical Conditioning Ivan P. Pavlov

• Studied salivation • Discovered a phenomenon!

Elements of Classical Conditioning

• Unconditioned stimulus (US) -naturally elicits a response • Unconditioned response (UR) -response naturally elicited by US • Neutral Stimulus (NS) -initially led to no response, becomes the CS • Conditioned stimulus (CS) -comes to elicit a response because it is associated with US • Conditioned response (CR) -response elicited by CS

Little Albert

• Watson & Rayner (1920) • “Little Albert” less than 1 year old • No fear of a white rat • After several pairings, became afraid of the white rat • Evidence of generalization • Evidence for acquired phobias

Conditioned Fear

• Neutral stimulus paired with a fear-eliciting event (US).

Conditioned Suppression

• Reduction/suppression of ongoing behaviour • E.g., suppression of lever pressing for food in rats

Suppression Ratio

\# of CS responses/(# of CS responses + # of pre-CS responses)

Conditioned Taste Aversion (CTA)

• The aversion developed by animals to tastes that have been followed by illness • Form of Classical Conditioning • Interestingly, • One trial learning • Interval is long (violates temporal contiguity) • We’ve likely all experienced CTA

Operant Conditioning

• Study of goal oriented behaviour • Refers to changes in behaviour that occur • Consequence of a behaviour will impact its future likelihood

Edwin L. Thorndike Operant Conditioning

Interest in animal intelligence • Systematic investigation • Formulated the Law of Effect

Thorndike’s Conclusions

• The escape behaviour was not due to intelligence/insight WHY? Steady decline because of gradual “stamping in” of the correct behaviour and “stamping out” of the incorrect behaviours.

Law of Effect

• Behaviours that lead to satisfactory states of affairs are strengthened • Behaviours that lead to unsatisfactory or annoying states of affairs are weakened

Skinner Box

• Systematized operant conditioning • Devised methods that allowed for systematic evaluation of repeated behaviours • Lever/bar pressing in rats • Key pecking in pigeons

Reinforcer

An event/consequence is a reinforcer if it follows the behaviour and INCREASES the future probability of that behaviour.

Punisher

• An event/consequence is a punisher if it follows the behaviour and DECREASES the future probability of that behaviour.

Behavioural contingency

1\.Discriminative stimulus (SD) that precedes the response and signals that a certain consequence is now available (Ex. tone signals that lever press will now produce food) 2.Response (R) that produces a certain consequence (Ex. lever pressing leads to food availability) 3.Consequence that serves to either increase or decrease the probability of the response that preceded it (Ex. consequence of food pellet increases rat’s tendency to press lever again)

Positive Reinforcement

• Reinforcement

* Increases likelihood of behaviour
• Positive
* Presentation of a stimulus
* Appetitive stimulus

Negative Reinforcement

• Reinforcement

* Increases likelihood of behaviour
• Negative
* Removal of a stimulus
* Aversive stimulus

Positive Punishment

• Punishment

* Decreases likelihood of behaviour
• Positive
* Presentation of a stimulus
* Aversive stimulus

Negative Punishment

• Punishment

* Decreases likelihood of behaviour
• Negative
* Removal of a stimulus
* Appetitive stimulus

Operant antecedents

Discriminative stimuli - Discriminative stimuli for reinforcement -signal that when present, responses are reinforced -when absent, responses are not reinforced

\
Discriminative stimuli for punishment -signal that when present responses are punished -when absent responses are not punished

Extinction

• Discontinuation of reinforcement resulting in a decrease or weakening of the response. • Decrease of a response because of absence of a reinforcer.

Primary Reinforcer

• aka - unconditioned reinforcers • event/stimulus innately reinforcing • Ex. treat given to dog for coming when called

Secondary Reinforcer

• aka - conditioned reinforcers • was associated with another reinforcer • Ex. saying “good dog” for coming when called • “good boy” previously associated with food

Intrinsic Reinforcer

• reinforcement provided by the mere act of performing the behaviour • Ex. Playing hockey for fun

Extrinsic Reinforcer

• reinforcement provided by a consequence external to the behaviour • Eg., Playing hockey for money

Continuous Schedule of Reinforcement

• CRf • Each specified response is reinforced • Works best for new responses • Also, when reinforcers are “small”

Intermittent Schedules of Reinforcement

• Intermittent (partial) • Only some responses are reinforced • More like everyday life • Leads to behaviours that are more resistant to extinction

Ratio schedules

Dependent on the number of responses

Interval schedules

Dependent on a response after a period of time

Fixed schedules

Regular

Variable schedules

Changing

Fixed Ratio (FR)

• Reinforcement contingent upon a fixed # of responses • Ex. FR 5: Rat must press a lever 5 times to obtain a food pellet. Light (SD): Lever pressFR 5 (R) à Food (SR) • Generally produces high rate of response with a “post-reinforcement pause” after each reinforcer

Stretching the ratio

– increasing the response requirement from a very rich schedule to a very lean schedule • Must be done gradually.

Ratio strain

breakdowns in behaviour due to increasing response requirement too quickly or too high • Commonly known as “burnout” in humans

Variable Ratio (VR)

• Reinforcement is contingent upon varying, unpredictable number of responses • e.g., VR 50: requirement might vary between 1 and 100 responses with 50 being the average • Usually produces high rate of response with little or no post-reinforcement pause • Often involved in gambling addiction

Fixed Interval (FI)

• Reinforcement contingent upon the first response after a fixed, predictable period of time • e.g., FI 30-sec: First response after 30 seconds produces a reinforcer • Often produces “scalloped” pattern of responding • Few responses after the reinforcer, but a gradual increase with the passage of time • Very high responding at the time point just before delivery of the reinforcer

Variable Interval (VI)

• Reinforcement contingent upon the first response after a varying, unpredictable time period • e.g., VI 60-sec: time period might vary between 1 and 120 seconds with 60 seconds being the average • Usually produces a moderate, steady rate of response with little or no post-reinforcement pause

Shaping

• Gradual creation of new operant behaviour through reinforcement of successive approximations to that behaviour. • Use of conditioned reinforcers is useful • Provides guidance and direction for change in behaviour - directed to a specific goal behaviour. • Build a desired behaviour not yet in the repertoire.

How to use shaping

1. Identify a goal behaviour
2. Identify present level of performance
3. Identify a series of steps towards the goal behaviour
4. Reinforce the first behavioural step until mastered
5. Then move on to step 2 and only
reinforce for successful completion
of step 2

Chaining

• Constructing a sequence of behaviours. • Reinforcement delivered only after the final response in the sequence.

Forward chaining

• Chains can be trained in the forward direction • Practicing the first response in the chain • Then adding successively the next elements • Causes overtraining of the first response

Backward chaining

• Chains can be trained in the backwards direction • Practicing the last response in the chain • Then adding successively the next elements • Causes overtraining of the last response

Whole chaining

• Chains can be trained using the entire chain simultaneously (whole-chain method) • Causes more total errors • But, equal practice on all links

Drive Reduction

• Proposed by Hull (1943) • An event is reinforcing to the extent that it is associated with a reduction in some type of physiological drive • Physiological drives: • Hunger • Thirst • Sex • Pain avoidance • Etc.

Incentive Motivation

• Reinforcers actually increase drive rather than reducing it. • Reinforcers are incentives that elicit responding • Perform in order to get the reward...

Brain Stimulation

• An actual reward center in our brain that is stimulated by reinforcers.

Premack Principle

• A high-probability activity will reinforce a low probability activity. • Responses can reinforce other responses • Behaviours can be ranked in term of preference • Ex. Going to the movies Vs. Studying

TOKEN ECONOMIES

• Token: • Conditioned reinforcers that can be accumulated. • Token Economy: • A program in which a group of individuals earn tokens for a variety of desirable behaviours and can exchange them for backup reinforcers.

Common Side Effects of Extinction

1. Extinction burst
2. Increase in variability
3. Emotional behaviour
4. Aggression
5. Resurgence
6. Depression

Extinction Burst

• Temporary increase in the frequency and intensity of responding when extinction is first implemented.

Increase in Variability

Attempts to obtain reinforcement with other responses.

Emotional Behaviour /Aggression

• Increase in behaviours that suggest “frustration” • Increase in aggressive responses • frustration-induced aggression

Resurgence

Reappearance during extinction of other behaviours that were once effective in obtaining reinforcement.

Depression

Extinction can lead to depressive symptoms. • Low levels of activity

Resistance to Extinction

Persistence of responding after extinction has been implemented.

Factors influencing resistance to extinction

1. Schedule of reinforcement
2. History of reinforcement
3. Magnitude of the reinforcer
4. Degree of deprivation
5. Previous experience with extinction
6. Distinctive signal

History and Magnitude

Number of received reinforcers (history) is related to resistance • Typically, a large reinforcer (magnitude) will cause greater resistance

Degree of Deprivation

Deprived vs Satiated • Change in motivation

Previous Experience with Extinction

Having previously experienced extinction could accelerate new extinction

Distinctive Signal

• Signal for the onset of extinction • Discriminative stimulus for extinction

Spontaneous Recovery

Reappearance of the extinguished behaviour after a period of time (“rest period”).

Differential Reinforcement of Other Behaviour (DRO)

Reinforcement of alternate behaviour during extinction of a target behaviour Functional Communication Training: Assumes that misbehaviour often occurs because the person is trying to achieve some type of reinforcer. -If so, training person to instead verbally communicate want they want should reduce the amount of misbehaviour.

Developing Healthy Habits

Habits are behaviours that are under strong stimulus control.
To create a habit, (Ex. doing sit-ups each day), you simply
need to:


1. Establish a strong cue for the behaviour (Ex. a certain time and
place for when and where you will do sit-ups)
2. Repeatedly practice the behaviour in the presence of that cue.
• Note that the important thing is to develop the habit of
starting to perform the behavior, such as starting to do situps. Therefore, even if you are busy or don’t feel like doing
sit-ups, try to do at least one.
• Persist in doing this day after day and you will soon notice
that starting to perform the behavior will begin to feel more
and more automatic and less and less effortful.

Escape Behaviour

Response that terminates an aversive stimulus • Negative reinforcement

Avoidance Behaviour

Response that prevents the aversive stimulus from occurring. • Also a form of negative reinforcement

Two-Process Theory of Avoidance

Avoidance behaviour is a result of both classical and
operant conditioning


1. Classical conditioning:
Light (NS): Shock (US) à Fear (UR)
Light (CS) à Fear (CR)
2. Operant conditioning:
Light (SD): Cross barrier (R) à Reduction in fear/anxiety (SR)

Obsessive-Compulsive Disorder (OCD)

Two-process theory explanation: • Obsession has been classically conditioned to elicit anxiety. • Compulsive behaviour is negatively reinforced by reduction of anxiety. Exposure and response prevention (ERP) therapy: • Graduated prolonged exposure to events that elicit the obsessive anxiety while not engaging in the compulsive behaviour.

Punishment

Event immediately AFTER a response occurs that causes behaviour to be reduced/eliminated. • Give aversive stimuli (Positive) • Withdraw reinforcers (Negative)

Negative Punishment

Time-out: loss of access to a positive reinforcer for a brief period • Response Cost: removal of a specific reinforcer

Rules of Effective Punishment

1. It should be immediate.
2. It should be consistent.
3. It should be intense enough to stop the behaviour.
4. Negative punishment (e.g., time-out) is generally
preferable to positive punishment.
5. If possible, it should be accompanied by an explanation.
6. Most important, it should be combined with positive
reinforcement for appropriate behaviour.

Possible Benefits of punishment

1. Increase in social behaviour
2. Improvement in mood - paradoxical
3. Increase attention

Possible problems of punishment

1. Punishment of inappropriate behaviour does not increase
appropriate behaviour
2. Possible generalized suppression of other behaviours.
3. Person delivering the punisher can become an SD
4. May cause avoidance of the person delivering the punisher
5. Elicits a strong emotional response
6. Can elicit aggressive behaviour
7. Possibly modeling punishment as an acceptable means to control
behaviour.
8. Use of punishment can be reinforcing for the person delivering it