EXAM 2 STUDY!!!!!!!!!!

Exam 2-EXDP

Associative Learning

1\.Classical Conditioning (Pavlonian Conditioning)


2. Operant Conditioning

Teacher-Student tree (Also called Pavlovian Conditioning)

Pavlov Yakolev→Pablo Rakic→Pat Levitt→Liz Hammock→Watson→K. Lashley→ T. Schneiria→ Dani Lehrman→ David Crews→ Hammock→ Jay Rosenblats

Associative Conditioning

* Repeated pairings, or single pairing if the input is strong enough, of 2 or more stimuli become associated.
* Allows us to predict future things based on what's related in the past

Three Steps for Classical Conditioning

1\.      What is the naïve state of the organism?

a. Naïve state - everybody comes wired to have a specific stimulus elicit a specific response

b. Thinking about reflexes

           i.  Think in terms of sensorimotor reflex

             ii.   We also have emotional reflexes

c.      Unconditioned stimuli (US/UCS) elicit unconditioned responses (UR/UCR)

d.      Food in your mouth (US) -> Salivation (UR)

e.      Food in the mouth is the naïve state, the vision stimulus of food is a conditioned stimulus

3 steps of CC:Unconditioned Step

a.      Pairing to stimuli

b.      Pair a neutral stimulus (aka the conditioning stimulus) with UCS -> UCR

      i.   UCR because the UCS is still around

   ii.     Only when the UCS is taken away, the UCR becomes CR

c.      "Bell" (metronome) (NS) + food in the mouth (UCS) -> salivation (UCR)

Unconditioned Response

Salivation ellicited by Food in Mouth (U.R or U.CR)

Three Steps for Classical Conditioning

3) Conditioned State

a.      Conditioned Stimulus (CS) (no longer a NS) -> Conditioned Response (CR)

    i.  Once NS is paired with the UCS, there is associative baggage and It becomes a CS

b.      "Bell"(CS) -> Salivation (CR)

Appetitive Conditioning

* Usually associatively conditioned to things that you typically "like"
* Ex. Pavlov's dogs

Aversive Conditioning

Classical Conditioning to things that bring displeasure (e.g. Little Albert)

1) Naive State: UCS →UCR

2) Conditioning Step: UCS+CS→UCR

3) Conditioned State: Cs→CR

Aversive Conditioning

* Associative conditioning to things we typically don't like
* Ex. Little Albert (J.B. Watson)

Aversive Conditioning

* Little Albert


* Native state

    i.    Very loud noise (UCS) -> Fear/Arousal response (UCR)

* Conditioning step

      i.   Very loud noise (UCS) + white rabbit (NS) -> Fear/Arousal response (UCR)

* Conditioned state

     i.  White Rabbit (CS) -> Fear/Arousal response (CR)

 

General Principles of Classical Conditioning

* The conditioned stimulus will never elicit the full strength response that comes from the UCS
* A conditioned response is weaker than an unconditioned response

General Principles of Classical Conditioning

* During step 2, pairing the NS with the UCS, each presentation of the CS + UCS is called a conditioning trial

General Principles of Classical Conditioning

* Intertrial Interval
* The time lapse between conditioning trials
* Interstimulus Interval
* The time lapse between the CS and the UCS

General Principles of Classical Conditioning

* Delay Conditioning
* We have the presentation of the CS, and with a little bit of time delay with the CS around the UCS comes around and overlaps
* The most effective because associative conditioning is to predict something is coming
* Seeing the bunny and then predicting something bad (loud noise) is coming (crying)

General Principles of Classical Conditioning

* Trace Conditioning
* The CS comes first but the UCS comes with a delay and they don't overlap
* Effective sometimes

Ex. Food poisoning

General Principles of Classical Conditioning

* Simultaneous Conditioning
* The CS and the UCS have identical onset and offset where they completely overlap

Relatively ineffective because it doesn't serve as a predictor

General Principles of Classical Conditioning

* Backward Conditoning
* The UCS comes on first followed by the CS
* Not effective - not predictive !

General Principles of Classical Conditioning

* How do you know if CS is acquitting the association with UCS?
* Jump to step 3, present CS alone and see if CR is present
* A "test trial"
* Present CS alone to see if CR is present
* If no CR is present, return to step 2 and add more conditioning trials
* During step 2
* If a response is showing up earlier and earlier and seemed to be a response to the

Two theories to help explain Kamin's results and classical conditioning in general

* Rescorla-Wagner Theory
* Aka an unconditioned stimulus modulation theory
* With intense UCS, you have lots of "room" and capacity for learning
* Learning is distributed among available CS (ex. Compound CS)
* It's a prediction error model

Two theories to help explain Kamin's results and classical conditioning in general

\

* Rescorla-Wagner Theory
* The intensity of the UCS is important and the unpredictability of the UCS will drive learning
* Did you know the UCS was about to happen?
* Is it a huge stimulus?
* If you did not know it was coming, this significantly increases the probability of associating UCS with available CS
* Does not help understand "CS-" conditioning
* Prediction error is reduced with each conditoning trial as CS begins to "predict" the UCS

* Mackintosh Model

* A CS modulation theory
* Helps understand CS+ and CS- conditioning
* Focuses on the attention the organism pays to the CS
* Makes an assumption that attention is limited

* Mackintosh Model

* You're going to pay attention to the biggest and most novel CSs
* CSs  compete with one another
* Intensity and novelty drive learning
* Helps explain a "NS" is not neutral is you have had prior CS- experience with it
* CS -> UCS --> UCR

* Mackintosh Model

* Mackintosh model under the CS with intensity or novelty of CS drives learning
* Rescorla-Wagner under the UCS with intensity and predictability of UCS drives learning

Higher Order Conditioning

* When a well conditioned CS is used as a UCS in future conditioning trials

1\.      Naïve state

o    Food in mouth -> salivation

Higher Order Conditioning

1\.      Conditioning trials

o    Bell + food -> salivation

Higher Order Conditioning

1\.      Conditioned state

o    Bell -> salivation

* Before Pavlov got his dogs:
* Food in mouth -> salivation
* Food in bowl (as a visual CS) + food in mouth -> salivation
* Food in bowl -> salivation
* Higher order conditoning
* Food in bowl (CS1) -> salivation (CR)
* Bell (CS2) + food in bowl (CS1) -> salivation (CR)
* Bell (CS2) -> salivation (CR)

 

Operant Conditoning

* Actions have consequences
* A form of associative conditioning
* Helps predict what we should do, what our behavior should be

Operant Conditioning

* Edward Thorndike
* The first to measure operant behavior
* Used critters in puzzle boxes
* Had traps with pulleys, levers, buttons, etc.)
* Put critter in the box -> a discrete trial
* Measured how long it took to escape
* During these trials, he argued that these critters are having training experiences

Operant Conditioning

* Stimulus in a puzzle box and trial and error different things to get out of the box
* There's a behavioral structure of the stimulus situation and then a response you could choose and an outcome
* Stimulus -> response -> outcome
* This structure is called SRO

Operant Conditioning

* After many trials, the stimulus -> response relationship has changed
* Thorndike was very interested in association between stimulus and response
* Argued that the strength between the stimulus and response was determined by the outcome
* The outcome determines the strength of the S -> R relationship/association
* Called this the Law of Effect

Discrete Trail Paradigms

* The experimenter determines how many trials or opportunities are available
* Ex. The puzzle box -> you get one trial when you first get into the box
* Ex. A maze
* The behavior response that you track overtime is latency to do the thing

Measured in latency to perform response

Free Operant Procedure

* The organism determines the response rate
* Ex. They can press a lever many times as it wants without constraint
* Ex. Skinner box (operant chamber)
* Measured in response number
* How many responses are emitted
* Ex. How many lever presses?

What features are shared between operant and classical conditioning?

* Both are forms of associative conditioning
* Both include a stimulus -> response relationship
* The strength of the relationship is improved with more conditioning trials
* Until you reach that asymptote (upper limit of response)

What features are shared between operant and classical conditioning?

* Both can be reduced (the S->R relationship) through an extinction procedure
* Extinction in classical conditioning uses exposures of CS alone
* Extinction in operant conditoning uncouples the outcome from the previous S -> R relationship (ex. Something that used to be rewarded is no longer rewarded)

What features are shared between operant and classical conditioning?

* Both show spontaneous recovery
* Timing of stimulus presentation matters
* S -> R -> O (O is also a stimulus)
* Long delays between stimulus presentations impair learning

What features are shared between operant and classical conditioning?

* Show blocking where prior learning can interfere with new learning
* A fully predictive stimulus (or response in O.C) interferes with learning a new association

What features are shared between operant and classical conditioning?

* Both show latent learning/inhibition
* Prior learning about inconsequential stimuli interferes with learning new things about those stimuli

What features are shared between operant and classical conditioning?

* Both show associative bias
* We have biological baggage that makes it harder or easier to learn something
* The biological baggage that makes some thing easier or harder to learn

 

\#1 Thing - Never punish a behavior that comes out of classical conditioning

Differences Between Classical and Operant Conditioned Behaviors

* Classical
* Respondent behavior
* The stimulus elicits the behavior which is reflexive
* Not controlled by consequences

 

Differences Between Classical and Operant Conditioned Behaviors

* Behavior is elicited by the stimulus
* Mostly related to consummatory behaviors
* Or innate patterns of behaviors (fixed action patterns)
* Feelings as unconditioned responses can be elicited by UCS
* Can be associated with CS to elicit CR

Differences Between Classical and Operant Conditioned Behaviors

* UCS determines the behavior
* Cause you use the UCS as a part of the training to generate the behavior

Differences Between Classical and Operant Conditioned Behaviors

* Consequences is irrelevant
* Can't punish or change feelings about reflexes
* S+S -> Response
* Making the association between the two stimuli
* There's no control over the outcome

* Operant

* Operant behavior
* Stimulus evoked behavior that is voluntary
* Controlled by the consequences 
* Behavior is emitted by the organism
* Mostly related to appetitive aspects of behavior
* Mostly not innate

Operant

* Operant behaviors regulate our exposure to UCS and CS that elicit feelings
* Using the operant behavior to control the probability of being exposed to a stimuli that elicit certain feelings
* The stimulus is setting the stage for the behavior
* There are different outcomes that you could choose
* Ex. Being grossed out by the squash or going to the room

Operant

* The behavior is influenced by the outcome/consequence
* What do I want to happen?
* S (that sets the stage) -> R (what behavior do I make) -> O (outcome)
* Outcome depends on the response
* Outcome also drives learning

What are the strategies for changing behavior that is classically conditioned?

* Two Ways:
* Reduce the strength of the conditioned response, use an extinction procedure
* Leave the squash on the plate but don't make her eat it
* You get the CS exposure as a CS-
* The CS isn't predicting that the squash is coming so it reduced the CR
* CS presentation without the UCS (CS-)
* Increase conditioning responding (CR)
* Need more conditioning trials until you reach asymptote

What features are shared between operant and classical conditioning?

* Both can be reduced (the S->R relationship) through an extinction procedure
* Extinction in classical conditioning uses exposures of CS alone
* Extinction in operant conditoning uncouples the outcome from the previous S -> R relationship (ex. Something that used to be rewarded is no longer rewarded)
* Both show spontaneous recovery

What features are shared between operant and classical conditioning?

* Timing of stimulus presentation matters
* S -> R -> O (O is also a stimulus)
* Long delays between stimulus presentations impair learning
* Show blocking where prior learning can interfere with new learning
* A fully predictive stimulus (or response in O.C) interferes with learning a new association
* Both show latent learning/inhibition

What features are shared between operant and classical conditioning?

* Prior learning about inconsequential stimuli interferes with learning new things about those stimuli
* Both show associative bias
* We have biological baggage that makes it harder or easier to learn something
* The biological baggage that makes some thing easier or harder to learn

 #1 Thing - Never punish a behavior that comes out of classical conditioning

Differences Between Classical and Operant Conditioned Behaviors

* Classical

* Respondent behavior
* The stimulus elicits the behavior which is reflexive
* Not controlled by consequences
* Behavior is elicited by the stimulus
* Mostly related to consummatory behaviors

Differences Between Classical and Operant Conditioned Behaviors

* Classical

* Or innate patterns of behaviors (fixed action patterns)
* Feelings as unconditioned responses can be elicited by UCS
* Can be associated with CS to elicit CR
* UCS determines the behavior
* Cause you use the UCS as a part of the training to generate the behavior

Differences Between Classical and Operant Conditioned Behaviors

* Classical

* Consequences is irrelevant
* Can't punish or change feelings about reflexes
* S+S -> Response
* Making the association between the two stimuli
* There's no control over the outcome

Differences Between Classical and Operant Conditioned Behaviors

Operant

* Operant behavior
* Stimulus evoked behavior that is voluntary
* Controlled by the consequences 
* Behavior is emitted by the organism
* Mostly related to appetitive aspects of behavior
* Mostly not innate
* Operant behaviors regulate our exposure to UCS and CS that elicit feelings
* Using the operant behavior to control the probability of being exposed to a stimuli that elicit certain feelings

Differences Between Classical and Operant Conditioned Behaviors

Operant

* The stimulus is setting the stage for the behavior
* There are different outcomes that you could choose
* Ex. Being grossed out by the squash or going to the room
* The behavior is influenced by the outcome/consequence
* What do I want to happen?
* S (that sets the stage) -> R (what behavior do I make) -> O (outcome)
* Outcome depends on the response
* Outcome also drives leanring

What are the strategies for changing behavior that is operantly conditioned?

* Four Ways:

* Increasing the frequency of operant behaviors/response (reinforcing)
* Stimulus -> response -> outcome
* Positive Reinforcement
* The outcome could be a reward
* Getting something good as the outcome
* Present an appetitive stimulus as an outcome

What are the strategies for changing behavior that is operantly conditioned?

* Four Ways:

* Negative reinforcement
* Taking something away to increase behavior
* Remove something bad

Withdraw an aversive stimulus as an outcome

What are the strategies for changing behavior that is operantly conditioned?

* Four Ways:

* Avoidance behavior -> performing a response before being exposed to an aversive outcome
* Escape -> perform a response after the outcome has already started

What are the strategies for changing behavior that is operantly conditioned?

* Four Ways:

* Decrease the frequency of operant behavior/response (punishment)
* Stimulus -> Response -> Outcome
* Positive punishment
* Get something bad as an outcome
* Present an aversive stimulus as the outcome

What are the strategies for changing behavior that is operantly conditioned?

* Four Ways:

* Negative punishment
* Removing something good
* Withdraw an appetitive stimulus as an outcome
* Presenting or removing something to control behavior
* "good" = appetitive outcomes
* "bad" = aversive outcomes

What are the strategies for changing behavior that is operantly conditioned?

* Four Ways:

* Does the outcome increase or decrease the behavior?
* Increase -> reinforcement
* Decrease -> punishment
* Does the outcome get presented or withdrawn?
* Presented -> positive
* Withdrawn -> negative
* Reinforces and punishers are defined by their effect on behavior
* Not just what we think about what will it do
* It has to actually increase or decrease the behavior, not just in theory

Natural vs Artificial Consequences

* Natural

* The behavior that's chosen, that you're performing, causes the outcome directly
* Behavior -> outcome
* Ex. Searching for food -> get food
* Ex. Keeping your room clean -> toys lasting longer
* Ex. Studying -> learning
* All of the outcomes are natural consequences to the behavior
* That can then reinforce the behavior

Natural vs Artificial Consequences

Artificial

* Outcomes are contrived
* They're made up
* Ex. Grades -> being paid for the A's
* Ex. Doing chores -> being paid
* They don't inherently happen naturally
* Natural and Artificial can be reinforcers and punishers

 

Punishment

* To decrease the frequency of a behavior
* Increases the variability in behavior
* May increase other unwanted behaviors

Punishment

* Need to also use differential reinforcement of other behavior (DRO)
* Essentially telling a kid, "not that and do this"
* Punish a specific behavior while also reinforcing desired behaviors

Punishment

* Punishment can backfire if the organism views it as an appetitive stimulus
* Ex. A kids finally getting some attention even though you're yelling at them

Punishment

* We need to be aware of concurrent reinforcement
* Behavior you are trying to punish may also be simultaneously reinforced by someone or something else
* Ex. You have a disruptive kid -> punished by the teacher and reinforced by classmates

Punishment

* Punishment is still under the S->R-> O
* Punishment as an outcome can encourage cheating or stimulus specificity
* Ex. I can speed under these conditions (when the cop cars aren't around)
* The initial intensity of the punishment needs to be appropriately intense (based on the context)

Punishment

* Most effective the first time
* Individuals can habituate to mild punishers (aversive stimuli)
* Common punishments corporal punishment or spanking
* Illegal in many countries and some US States

To summarize:

* The appropriate application of punishment is difficult
* Punishment should be applied immediately, not delayed
* Intense at first application (because pf habituation to mild aversive stimuli)
* But not so intense that it's unethical
* Consistently applied each time response is emitted

To Summarize

* Negative punishments (like a time-out) are preferred over positive punishments (like spanking)
* For people, it works best with a clear explanation of the specific behavior being punished
* Punishment of unwanted behaviors is not effective when coupled with differential reinforcement of other behavior (DRO)

Stimulus Discrimination

* A discriminative stimulus (SD)
* A stimulus that sets the stage
* How we think about something
* Tells the organism if the behavior is going to be reinforced or punished
*  remember S -> R -> O

* Discriminative stimulus for extinction (S delta)

 

* Indicating a particular response will no longer be reinforced
* Tells if you do the behavior, the response will no longer be reinforced
* Signals that a specific response will no longer be reinforced
* Ex. See your notes as a stimulus where you might get a better outcome. But when the semester is over, the outcome is different so you look at the stimulus differently

Stimulus Discrimination

* Discriminative stimulus for punishment (S^Dp)

Stimulus indicates that response will be punished

Reinforcers

* Primary reinforcer
* No learning is required
* An outcome that you don't have to learn anything about
* Needed for survival
* Tied directly to physiological/psychological needs
* Ex. Food, water, comfort, shelter, alleviation from boredom

Secondary Reinforcers

* Have a conditioned history
* Conditioned to be associated with primary reinforcers
* Acquired through classical conditioning
* Not required for survival
* Ex. Money, clickers for pets, stickers for kids

Reinforcers

o How do we acquired associative conditoning with money


1. UCS (food) -> UCR (satiation)
2. UCS (food) + NS (Money) -> UCR (satiation)
3. CS (money) -> CR (satiation)

Reinforcers

o Money can be used as reinforcers in operant conditioning


1. Secondary reinforcers are being presented in the outcome
o Responses in operant conditioning can acquire reinforcing properties themselves due to their association with appetitive outcomes

Reinforcers

1. Ex. Helping behavior -> learned industriousness -> behaviors that require high effort can start being reinforcing on their own • Performing actions that require high effort can start to be reinforcing on their own

        Performing actions that require high effort can start to be reinforcing on their own

* Instinctive drift

1\.      When conditioned secondary reinforcers begin to elicit behaviors usually reserved for primary reinforcers

·         Ex. Racoon trained to accept coins for a reinforcer and they will start to wash the coins (washing is something they do with food (a primary reinforcer))

* Autoshaping

1. When we use classical conditioning to generate a behavior that we can then bring under voluntary control (operant control)
2. Use first a classical conditioning step and then an operant conditioning step
3. Use this idea when we want to potty train dogs

Autoshaping: Classical Conditioning

1. UCS (full bladder) -> UCR (attention to bladder)
2. UCS (full bladder) + CS (queue word) -> UCR
3. CS (queue word) -> CR (attention to bladder)

Autoshaping: Operant Conditioning

1. S -> R -> O
2. Queue word -> can make the choice of holding it or going -> if they hold it they wait outside and if they pee, it's verbal praise.

Shaping

* Changing operant behaviors in increments
* Need to identify the target behavior
* What do we want the organism to do

Shaping

* Assess the current behaviors with the idea of understanding "how far do we have to go to get the desired behavior"
* Divide the training into smaller steps (successive approximations)
* Reinforce behavior that are in then direction of target behavior
* Stop reinforcing simpler behaviors, reinforce the next more elaborate behavior
* Ex. Training a rat to play basketball

Shaping: Use Chaining

Training an organism on a series of behavior responses performed in order each time

Shaping: Backward Chaining

* Train an organism to perform a series of behaviors training order is reversed (i.e last behavior is trained first)
* A key component for any type of OC, an active animal is a trainable animal
* An active organism is an easier to train organism

 

Reinforcement Schedules

* Describes the pacing of reinforcement delivery
* Learning is fastest when the interval is short between when the response is given and the outcome
* Learning is facilitated by a short time between the response and outcome
* Individuals differ in this difference and how long you can go

Reinforcement Schedules

* Self-regulation and self-control can extend the window of learning

Delayed gratification concept

Reinforcement Schedules

* Cumulative response curve
* Graphs that basically show (y-axis) number of responses and (x-axis) time
* Illustrates the accumulation of responses
* Adds responses over time to the foundation of responses that already exist
* How many responses are here and all times before?
* These curves never slope down -> they can be flat though

Reinforcemnt Schedules: Continuous Reinforcement

* Reinforcer is given every time the desired behavior is performed (or when the response is emitted)

Reinforcement Schedules: Partial Reinforcement Schedule

* Fixed Ratio
* Where the reinforcer is provided after a fixed (or specific) number of responses
* Ex. FR3 -> After three responses performed, a reinforcer is provided
* Ex. FR1 -> continuous reinforcement (reinforcement given after every response)
* You have a period of time where you don't accumulate response after the reinforcement (but then it'll come back)
* Post reinforcement pause
* "taking a break" and the pulling the specific number of times

Fixed Interval

* Where the reinforcer is provided if the behavior/reponse is emitted in a unit of time
* The cumulative response curve has a scallop shape
* Looks like swoopy stair steps
* Promotes procrastination

Vaiable Ratio

* The reinforcer is provided after a variable (average) number of responses
* Ex. VR5 -> reinforcement given at sometimes 3rd, 4th, 5th, 6th (have to average out to 5) response
* Ex. Slot machine

Variable Ratio

* There is a smooth curve
* Because the accumulation of responses is continuous, even after the reinforcer is provided
* "Maybe it'll be this time, or this time, or this time"

Variable Interval

* Where the reinforcer is available after a variable (average) amount of time
* Because of the varibale nature, you don't know when the time is going to be up
* There's a smooth curve
* Ex. Pop quiz -> you don't know exactly which class session will result in pop quiz so the accumulation of studying is more smooth

Variable Interval

* Fixed -> Specific Number
* Ratio -> the number of reinforcers based on responses

Progressive Ratio

* Use it for fixed and variable ratio
* You progress through harder and harder reinforcement schedules
* Maybe start with FR1 and once the behavior is established, you move to FR2 and so on
* With training you increase the number of behaviors required to earn a reinforcer

Break Point

* Ratio at which the organism stops responding
* Use the break point as a measurement to see the motivational behavior of an organism