Neuroscience of Learning and Memory - TAMU - Unit 8; Instrumental Memory (Dopamine Systems)

What is the basis for the law of effect?

a given stimulus in the environment can elicit a variety of behavioral responses

In the law of effect, what happens if the stimulus is "satisfying"?

the connection between stimulus and response strengthens

In the law of effect, what happens if the stimulus is "annoying"?

the connection between stimulus and response weakens

What happens when the connection between a stimulus and a response strengthens?

the behavior is more likely to reoccur

What happens when the connection between a stimulus and a response weakens?

the behavior is less likely to reoccur

The law of effect is what kind of association?

stimulus-response

Who proposed the law of effect?

EL Thorndike

What is positive reinforcement?

when behavior becomes more frequent because it results in the addition of an appetitive (nice) stimulus

What is negative reinforcement?

when behavior becomes more frequent because it results in the removal of an aversive stimulus

What is negative punishment?

when behavior becomes less frequent because it results in the removal of a nice stimulus

What is positive punishment?

when behavior becomes less frequent because it results in the addition of an aversive stimulus

What is "shaping of successive approximations"?

the process by which you gradually reinforce responses that are increasingly similar to the one you are ultimately trying to condition

Are stimulus-Response (S-R) associations reflexive or goal-directed?

reflexive

1 multiple choice option

Are action-Outcome (A-O) associations reflexive or goal-directed?

goal-directed

1 multiple choice option

A-O learning leads to an (unencoded/encoded) relationship between an action and an outcome.

encoded

1 multiple choice option

What is the first step of the devaluation procedure?

rat learns to press a lever for food

What is the second step of the devaluation process?

food is paired with a noxious chemical/the rat is overfed

What is the third step of the devaluation process?

test if the rat will press the lever for food

At the end of a devaluation procedure, if the rat presses the lever for food, what kind of memory does it have?

S-R

3 multiple choice options

At the end of a devaluation procedure, if the rat does not press the lever for food, what kind of memory does it have?

A-O

3 multiple choice options

A-O learning requires ________.

cognitive resources

Once the A-O relationship is established, what does the S-R relationship do?

produces the behavior automatically, freeing up cognitive resources

At what rate does A-O learning occur?

very quickly

At what rate does S-R learning occur?

very slowly

What relationship does the dorsomedial striatum encode?

A-O

What relationship does the dorsolateral striatum encode?

S-R

What relationship is subject to devaluation?

A-O

______ is important for the initial acquisition and expression of behavior.

A-O

_____ is important for the eventual development of rote habits.

S-R

Electrical stimulation of the septum will _______ ______ a lever press.

positively reinforce

3 multiple choice options

What is the one brain area that is extremely reinforcing when stimulated?

medial forebrain bundle (MFB)

MFB contains the axons of _________ neurons that originate from the midbrain.

dopamine, norepinephrine

Where do dopamine axons from the MFB commonly form synapses?

nucleus accumbens, dorsal striatum

Increasing the intensity of stimulation of the MFB increases:

the reinforcing efficacy of MFB stimulation, the amount of dopamine released in the striatum

Blocking ____________ synthesis in the brain puts rats to sleep.

norepinephrine

Small doses of a neuroleptic can ________ the reinforcing efficacy of medial forebrain bundle (MFB) stimulation.

decrease

1 multiple choice option

Larger doses of neuroleptic drugs imitate:

extinction

Amphetamines ________ the reinforcing efficacy of MFB stimulation.

increase

1 multiple choice option

Removal of dopamine from mice will reduce the _______ the mouse takes to get the reward.

effort

What is the reward hypothesis?

dopamine release occurs following an instrumental behavior, upon delivery of the reinforcer, creating the 'hedonic impact' that makes the instrumental behavior more likely to occur again in the future

What is the incentive motivation hypothesis?

dopamine release occurs preceding an instrumental action, serving to activate that behavior. Dopamine release has nothing to do with the reward or hedonic impact of the reinforcer, once received

The activity of dopamine neurons in response to a reinforcer is perfectly explained by what theorem?

Rescorla-Wagner

What is dopamine activity in response to the reinforcer involved in?

learning

What happens to dopamine activity once learning has ended?

it disappears

Unlikely reinforcers are ________ surprising than likely ones.

more

1 multiple choice option

The activity of dopamine neurons in response to an unlikely reinforcer tends to be ________.

higher

1 multiple choice option

The prediction error signal drives ___________________ in the early stages of instrumental training.

new learning

Once a task is acquired, dopamine release is triggered by:

the earliest clear predictor of the reinforcer

Once a task is acquired, dopamine serves to:

activate or motivate the instrumental action

(T/F) Dopamine is hedonic impact.

false

1 multiple choice option

What are hedonic hotspots?

areas in which pharmacological stimulation of opioid receptors increase 'liking' expressions

What are hedonic coldspots?

areas in which pharmacological stimulation of opioid receptors abolish 'liking' expressions

What neurotransmitter is responsible for hedonic impact?

endogenous opioid

Why do rats press a lever for MFB stimulation?

MFB stimulation causes a brief burst of activity in dopamine neurons

A "model-free" reinforcement learning algorithm needs the ability to produce an:

action

A "model-free" reinforcement learning algorithm needs the ability to detect:

key aspects in the environment

A "model-free" reinforcement learning algorithm needs the ability to produce a reinforcement signal when:

a goal state is detected

A "model-free" reinforcement learning algorithm needs the ability to predict:

which actions will lead to the reinforcing goal state

What does "model-free" mean?

it has no other programming to tell it how to complete its task

What is the alignment problem?

given that AI systems are designed to respond to unpredictable scenarios in flexible ways in order to attain a pre-specified goal, how can we ensure that they behave the way we want them to?