exam 3 psyc 340 - matthew alwood

instrumental vs operant learning . what is the general criteria?

instrumental: imposes behavioral change by rewarding/punishing stimulus

all operant learning is instrumental, but not all instrumental learning is operant

General Criteria

1. The modification depends on neural plasticity.

2. The modification depends on the organism's experience.

3. (a) The modification extends beyond the circumstances used to induce it. (b) The experience has a lasting effect on performance.

4. Imposing a temporal relationship between a response and an outcome alters the response

what is the two additional criteria for operant learning vs. instrumental?

Additional Operant Criteria

1. behavior can increase positively or negatively (no constraints)

2. reinforcer can be positive or negative (no constraints)

discrete trial method

behavior earns response at a certain time

free-operant method

behavior earns response whenever

- no restrictions, can have reinforcer occur at any time

reward learning

behavior is rewarded

punishment learning (positive punishment)

addition of a punishment

- behavior is associated with consequences

ex.) dog barks, so it gets a shock

omission (negative punishment)

behavior removes positive stimulus

ex.) dog bites, toy is taken away

escape/avoidance (negative reinforcement)

behavior removes negative stimulus

ex.) rat moves to other side of chamber to avoid shock

what is shuttle box/wheel turn chamber used for?

escape/avoidance learning

skinner box is used for?

opportunities to engage w/ reward and punishment

learned helplessness

experiencing unavoidable aversive events will teach subject that all future avoidable aversive events are unavoidable

learned helplessness hypothesis

causes motivational and associative deficits

"Locus of Control"

Internal: I can control my own future.

External: My life is directed by forces outside my control.

secondary (conditional) reinforcers

pavlovian association is formed between neutral stimulus and primary reinforcer

marking stimuli

brief auditory/visual cue distinguishing target response from other behaviors

- NOT valued

exceptions to contiguity are:

secondary (conditional) reinforcers and marking stimuli

do ratio or interval schedules produce more responses?

ratio schedules

- interval schedules reach an asymptote first

feedback functions

how quickly the organism learns to maximize benefit

Generalized Matching Law

takes differences in effort, reinforcer strength, and switching from one response to another

matching law

distribution of responses depend ONLY on the schedule of reinforcement for each response

concurrent chain schedules

organism's initial choice is locked for the remainder of the trial

choice link

initial decision where the participant can select either option

terminal link

trials following initial (choice) trial where only selected option is avaible

drive reduction theory

reinforcers satisfy biological needs

issues with drive reduction theory

- learning can occur without drive reduction

ex.) rats desire something sweet with no nutritional value (no biological need)

Premack's principle

the more probable response will reinforce less likely behavior, but less probable responses will not reinforce more probable behaviors

response deprivation theory

When a behavior (e.g., playtime) is restricted below its normal rate, the individual becomes "deprived," making the opportunity to perform that behavior highly motivating

- low frequency behaviors can act as reinforcers

- restriction drives reinforcement

problems with premack's approach

- difficult to quantify time spent performing individual behaviors

place learning is

spatial/explicit

response learning is

habit/implicit

what disrupts place learning?

lesioning hippocampus

what disrupts habit learning?

lesioning basal ganglia

striatum is crucial for ..

- motivation, reinforcement, planning movements

- receives dopaminergic input from substantia nigra

substantia nigra is crucial for..

- dopamine production

- forming neuron synapses onto neurons in striatum

habit system consists of:

what do these structures do?

amygdala: basolateral nucleus encodes reward magnitude and valence

striatum: integrates positive and negative associations w/ behavior over multiple trials

orbital prefrontal cortex (OPC)

allows organism to weigh the relative value of alternative outcomes

value

gain now vs future gain

lesions to orbital prefrontal cortex (OPC) does what?

trouble w/ negative feedback

- organism dependent more on amygdala, resulting in impulsivity

two classes of striatal projection neuron

D1 receptor and D2 receptor

D1 receptor

expresses direct pathway

- reward learning and initiating action

D2 receptor

expresses indirect pathway

- punishment learning and inhibiting action

conditional place preference

- direct pathway rats preferred stimulation

- indirect pathway rats avoided stimulation

is habit learning incremental?

yes, more trials builds a stronger habit

discriminative stimulus

indicates whether the R-O relation is in effect

avoidance learning

animal learns that tone predicts shock.. if animal performs a behavior, it learns that it can avoid unpleasant stimulus

two factory theory of avoidance

1. early in training: tone (CS) paired with shock (US)

tone generates fear

2. later in training: animal avoids tone w/ a behavior, reinforcing behavior

- reinforced by non-occurrence of shock

problems w/ two factory theory

presenting CS alone should extinguish conditioned fear

- animals still show conditioned fear

animals show no true signs of fear to CS under normal conditions

- if avoidance response is block, fear response

higher magnitude shock should increase avoidance performance

- actually decreases

signal termination is lower than it should be according to the theory, shock avoidance is higher than it should be

novel test of two factor theory

ending scary tone should be the reinforcer for avoidance behavior

low imminence threat

pre-encounter behavior (anxiety)

- predator could be near

high imminence threat

post-encounter behavior (fear)

- predator is near

fight or flight

circa-strike behavior (panic)

- predator is attacking

response patterns explain why high magnitude shocks decrease avoidance because..

avoidance is a pre-encounter behavior, increasing shock increases likelihood of post-encounter response

instrumental learning becomes ... over time

habitual

two-way shuttle box active avoidance

animal learns to shuttle back and forward to avoid shock in response to a tone

*active

- defensive freezing ineffective for successful avoidance

*two way

- neither side of chamber is considered safe

Goal: We want the rat to learn response means safety, not the area

one-way shuttle bow

shuttle box with two very different chambers

active: starts on dangerous side of chamber

passive: starts on safe side of chamber

*one side of chamber is always safe

Goal: measures latency to enter other chamber

active avoidance learning involves ...

suppressing brain regions involved in freezing

freezing neurobiology involves

basolateral amygdala - involved in emotional associations (fear)

central amygdala - involved in fight-or-flight behaviors (freezing)

LA receives input from CS & US, BA routes info to CeA

avoidance learning neurobiology involves

basolateral amygdala - emotional associations (fear)

prefrontal cortex - suppresses freezing response

striatum - involved in motivated action

BA -> NAcc (nucleus accumbens)

NAcc - facilitates action based on reward

what mediates different levels of threat imminence?

different types of neurons in basolateral amygdala

stimulus control

the ability of a given stimulus to control behavior

stimulus dimension

one aspect of a particular stimuli

stimulus generalization

generalizing similar stimulus to give same response

- indicative of high stimulus control

ex.) even though a traffic light is pink, you still stop

stimulus discrimination

responding to one stimulus but not another

- ability to differentiate between stimulus

- low stimulus control

differential responding

make changes to one stimulus dimension (aspect) to study differences in responding

stimulus generalization gradient

degree to which stimulus generalization occurs

factors impacting stimulus control (there are 4)

sensory capacity - is it biologically possible for organism to perceive stimulus?

sensory orientation - is organism oriented properly to perceive stimulus?

stimulus intensity (salience) - is stimulus strong enough to be perceived?

motivation - is organism in motivational state to attend to stimulus?

stimulus filter

bias towards visual/auditory cues during specific motivational states

stimulus discrimination training

organism associates different, but similar stimuli w/ different outcomes

- form of differential conditioning

interdimensional discrimination

stimulus discrimination based on many different stimulus dimensions

- no one feature has substantial stimulus control

intradimensional

stimulus discrimination based on a single stimulus dimension

perceptual learning

habituates participants to similarities to be able to differentiate

ex.) hard to know difference between two red wines

solution: try many different red wines

acquired distinctiveness

increase in an organism's ability to discriminate between two stimuli

acquired equivalence

increase in organism's ability to generalize two stimuli

perceptual concept training (think of any example)

reinforces behavior in response to various stimuli w/ one common feature

ex.) pigeons peck for food to any photo w/ a body of water

non-perceptual concept learning (think of an example)

reinforce behavior in response to different concepts or ideas

ex.) categorizing fruits & vegetables

opponent process theory of acquired motivation

addictive behavior manifests because of two processes:

A process = fast

B process = slow

- processes can be positive or negative, but are always opposites

addiction is driven by __ reinforcement and __ reinforcement

positive; negative

early in addiction driven by..

desire for positive effects

- driven by reward system

- impulsive seeking

late in addiction driven by ..

desire to relieve withdrawal symptoms

- driven by anti-reward system

- compulsive

homeostasis

body's tendency to seek a stable equilibrium

allostasis

body's ability to anticipate and adjust to changes

allostatic load

wearing down of body from chronic stress

impulsive drug taking: reward system

striatal regions receive dopaminergic input from VTA and from neurons that release endogenous opioids

"liking"

prefrontal cortex basolateral amygdala -> extended amygdala/ventral pallidum (LAH) -> ventromedial ventral pallidum -> mediodorsal thalamus -> prefrontal cortex basolateral amygdala

impulsive drug taking: HPA stress system

hypothalamic pituitary axis -> physiological stress system

compulsive drug taking: anti-reward system

- rewires reward system

"wanting"

prefrontal cortex -> ventral/dorsal striatum -> ventral/dorsal pallidum -> mediodorsal thalamus -> prefrontal cortex

compulsive drug taking: extrahypothalamic stress system

responsible for negative emotional experience associated with stress

sites of action? what input does this site of action receive?

nucleus accumbens (region of striatum)

- input from endogenous opioids

- dopaminergic input from VTA