Intro Psych - Week 8: Learning

Stages of learning in classical conditioning

Acquisition:
- Phase in which CS comes to evoke CR
- Repeated trials (pairings of CS + UCS) establish and strengthen CR
- CS-UCS order and interval important
Extinction:
- CR weakens when UCS does not follow CS
Spontaneous recovery:
- Extinguished CR reappears after rest period

If an organism is in a trial, and the UCS does not follow CS, thus weakening the CR, why might they try the CR (conditioned response) again after a rest period?

Because from an evolutionary standpoint, it makes sense to check in again at a future date.
For example, if an animal finds themselves at a nearby watering hole but it has dried out, they might return in a few months, because water may have returned after a season change.

Generalization

Tendency for stimuli similar to CS to evoke similar responses (ex. a different bell is used and it sounds differently, will the dog still salivate?).

Discrimination

In classical conditioning, the ability to extinguish between the conditioned stimulus and other stimuli that do not signal unconditioned stimulus after repeated exposure to both.

Generalization gradient

If something is similar to the original conditioned stimulus (ex. something is close to the sound of the original bell), a response will occur. The further that the stimulus wanders from its originality, the less strong the reaction will be.

What makes operant conditioning different from classical conditioning

Unlike classical conditioning, an organism must perform a response in order for a consequence to occur (ex. coins in vending machine -> chocolate bar. Walking carefully in the snow -> avoid painful fall).

Reinforcement vs. punishment

Reinforcement: Increasing likeliness of response
Punishment: Decreasing likeliness of response

Law of effect.
Who was this extended by?

Thorndike's principle that behaviours followed by favourable consequences become more likely, and behaviours followed by unfavourable consequences are less likely.
Extended by B.F Skinner.

Operant behaviour

- Complex or "voluntary" behaviours (ex. push button, perform complex task).
- Operates (acts) on environment
- Produces consequences

Operant vs. respondent behaviour (reflex).

Respondent behaviours:
- Occur as an automatic response to stimulus.
- May be learned through classical conditioning.

B.F Skinner

- 1904-1990.
- Developed behavioural technology which is still important and used today.
- Many practical methods are based on Skinner's operant conditioning (ex. Skinner box).
- Techniques still widely used today.

Operant chamber/Skinner box

- Sound-attenuated chamber with bar/key that animal presses/pecks to release food or water reward.
- Contains device to record responses.

Define the following terms when referencing operant conditioning:
a) Reinforcer
b) Primary reinforcer
c) Conditioned reinforcer
* What is important to remember about reinforcers?

a) Event that strengthens the behaviour (ex. the response) that it follows (this doesn't mean that it's "good" behaviour).
b) Innately reinforcing stimulus. Satisfies biological need.
c) Secondary reinforcer. Learned through association with primary reinforcer (remember UCS from classical conditioning) (ex. money - meaningless biologically, but we can use it to satisfy biological needs such as buying food).
* What constitutes a reinforcer may differ among individuals.

True or False:
Primary reinforcers satisfy biological needs

True

Discriminative stimulus

- Signals that the schedule of reinforcement is in effect.
- Response -> reinforcer relationship only happens in certain environments (ex. vending machine only works when the light is on).
- Over time habit learning may occur where the discriminative stimulus and response become associated in a manner that is independent of the original reinforcer.

Adding some kind of stimulus:

Positive

Removing some kind of stimulus:

Negative

What are some issues with punishment?

Punished behaviour isn't forgotten.
- Suppressed behaviour returns when punishment no longer imminent.
Fear can generalize to desirable behaviours/stimuli through classical conditioning.
- Fear of school if being yelled at (as punishment) during school.
Does not necessarily guide toward desired behaviour.
- Reinforcement tells you what to do.
- Punishment only tells you what not to do.
- Combination of punishment and reward can be more effective than just punishment.
Seemingly aversive event may not be punishment.
- Loud reprimands from teachers may actually increase behaviour, whereas quiet reprimands decrease target behaviours.

Avoidance vs. escape

Avoidance: Avoiding situation altogether, maybe because of past traumas related to it (development of phobias).
Escape: When you've already found yourself in a situation, and leaving.

What is an issue with avoidance?

We have generalized the entire situation which we avoid with one particular outcome.
For example, if we avoid every single dog because we were bit by one as a child, we're unable to form better memories and get over this negative mindset with dogs that don't bite.

How do we unlearn behaviours associated with avoidance strategies?

- Flooding (exposure therapy, ex. put someone in a room full of dogs).
- Systematic desensitization (ex. gradually introduce more and more dog-like things until someone is able to be placed in a room full of dogs).

Fixed ratio (FR)

- Reinforce response only after specified number of responses.
- Faster responding gets more rewards.
- Different ratios.
- Leads to very high rate of responding.

Variable ratio (VR).
Examples?

- Reinforce response after unpredictable number of responses.
- Very difficult to extinguish because of unpredictability.

Examples: Gambling, fishing.

What does FR10 mean?

Fixed ratio of 10:1

True or False:
The number of responses in fixed ratios is always 1.

False.
This is the number of outcomes in a fixed ratio.
# of responses : # of outcomes.
# of responses : 1.

Which type of ratio has a predictable number of responses?
Which does not?

Fixed ratio (FR): Predictable # of responses.
Variable ratio (VR): Unpredictable # of responses.

What does variable ratio depend on?

Average

What does VR10 mean?

Variable ratio of 10 (average of 10).

Is FR or VR harder to extinguish?
Why is this?

VR is harder to extinguish.
This is because it's unpredictable, and we might not know for a while if we're unlucky or if it's ex. broken (in reference to slot machines).

Fixed interval (FI).
Example?

- Reinforce response only after specified time has elapsed.
- Response occurs more frequently as anticipated time for reward draws near.

Example: Waiting for a bus at a bus stop. If there are many people at the bus stop, you know the bus hasn't come yet and it should soon.

Variable interval (VI).
Example?

- Reinforce response after unpredictable time intervals.
- Produces slow steady responding.

Example: Checking emails frequently without notifications.
(There's a certain amount of time you have to wait, you're just not sure what it is).

Cognition and operant conditioning.
Example?

Cognitive map
- Mental representation of layout of environment.
Example: After exploring a maze, rats act as if they've learned cognitive map.
Latent learning
- Learning that occurs, but is not apparent until there is incentive demonstration

Acquisition
Example?

Initial stage of learning.
- Repeated trials (pair response and reinforcement) establish and gradually strengthen response.

In OC, strengthening of reinforced response works best with immediate rather than delayed reinforcement

Example: A drug taking a while to work is less reinforcing than a drug that works immediately.

Extinction
Example?

- Diminishing of conditioned response.
- In operant conditioning, this will gradually occur when the response is no longer followed by reinforcement.

Example: If a rat presses a lever, they get a food pellet. Remove food pellet, rat will eventually stop pushing the lever.

What is shaping?

- Conditioning procedure in which reinforcers guide behaviour toward closer approximations of desired goal.
- Reward behaviours that increasingly resemble desired behaviour.
- Reward for each new step, and stop giving rewards for old steps.

Continuous reinforcement

- Reinforcing the desired response each time it occurs.
- Learning occurs rapidly.
- Extinction occurs rapidly (organism knows quickly).

Partial reinforcement

- Reinforcing only some responses.
- Results in lower acquisition.
- Greater resistance to extinction.
- Different schedules

Different schedules of reinforcement

- Fixed vs. variable
- Response-based (ratio) vs. time-based (interval)

Why might variable schedules be beneficial?

Although they make it hard for the organism to learn, extinction is slower.

What is learned in operant conditioning?

Relationship between response and outcome (R -> O)

(Operant conditioning)
What happens in a "devaluation" situation where outcome value is changed after operant training?
(In reference to food poisoned experiment).

- Step 1: Lever press produces food reinforcer (R -> O)
- Step 2: Food is poisoned (taste of food is associated with illness - classical conditioning)
- Step 3: Will animal press lever associated with food? (Ex. retain the response learned in step 1?)

True or False:
Organisms will learn to modify a response in the absence of a direct association between the response and a consequence.

True

Repeated exposure to a stimulus in non-associative learning:

Changes behaviour

How is behaviour changed in repeated exposure to a stimulus (non-associative learning)?

- Sensitization: Increased response with repeated exposure to a stimulus.
- Habituation: Decreased response with repeated exposure to a stimulus.

Two most fundamental forms of learning

- Classical (Pavlovian)
- Instrumental (operant) conditioning

Thorndike's law of effect

When a behaviour has a:
- Positive (satisfying effect): It's likely to be repeated in the future.
- Negative (painful/annoying effect): It's less likely to be repeated in the future.

Effects that increase behaviour

Reinforcers

Effects that decrease behaviour

Punishers

Prediction error

The chance that a conditioned stimulus won't lead to the expected outcome.

Which form of conditioning involves choice?

Operant

Which experiment discussed is an example of observational learning?

The Bobo Doll Experiment (children treat the Bobo dolls in a similar manner as they've observed adults treating them).

Nonassociative learning

Single repeated exposure leads to a change in behaviour

Habituation

When our response lessens with exposure

Sensitization

When our response inreases with exposure

Which form of conditioning is "stimulus-stimulus" learning?

Classical

Which form of conditioning is "stimulus-response" learning?

Operant

Metacognition

Describes the knowledge and skills people have in monitoring and controlling their own learning and memory.

What is the main bottleneck that we face when we're using our knowledge?

Being able to access said knowledge