Chapter 5: Operant conditioning

Operant conditioning

the process by which organisms learn to make or avoid response in order to obtain or avoid outcomes

Discriminative stimulus

a stimulus indicating that a particular response (R) may lead to a particular outcome (O)

Shaping

operant conditioning technique in which successive approximations to the desired response are reinforced

Chaining

operant conditioning technique in which organisms are gradually trained to execute complicated sequences of discrete responses

Reinforcer

consequence of behavior that leads to increased likelihood of that behavior in the future

Primary reinforcer

a reinforcer such as food, water, or sleep that is of biological value to an organism

Secondary reinforcer

reinforcer that initially has no biological value but that has been paired (or predicts the arrival of) a primary reinforcer

Punisher

a consequence of behavior that leads to decreased likelihood of that behavior in the future

Fixed ratio

outcome arrives after every nth response

Fixed interval

outcome arrives after a fixed set of time

Variale ratio

outcome arrives after average number of time

Variable interval

outcome arrives after average number of time

Positive reinforcement

add something pleasant after a behavior to make that behavior happen more often

Positive punishment

adding something unpleasant after a behavior to make that behavior happen less often

Negative reinforcement

removing something unpleasant after a behavior to make that behavior happen more often

Negative punishment

taking something away after a behavior to make that behavior happen less often

Continuous reinforcement

rewarding a behavior every time it happens

Partial reinforcement

rewarding a behavior only some of the time

Matching law of choice behavior

tend to spend time choosing the option that gives more rewards

Premack principle

use an activity someone enjoys as a reward for completing an activity they enjoy less

Response deprivation hypothesis

an activity becomes rewarding when you are not allowed to do it as much as you normally would

Dorsal striatum

helps you learn habits and connect your actions with shivery consequences such as rewards/punishments

Caudate nucleus

helps you learn the consequences of your actions and make decisions about what to do next

Putamen

helps you learn and repeat behaviors through practice and reinforcement

Orbitofrontal cortex

helps evaluate rewards and punishments and adjust your behavior when outcomes change

Wanting

brains motivation to seek a reward, strongly associated with dopamine pathways

Liking

pleasure experienced when receiving a reward, involves specific brain areas including parts of the nucleus accumbens, and ventral palladium

Incentive salience hypothesis

explains how brain makes rewards seem desirable and motivates you to seek them out

Endogenous opioids

chemicals naturally produced by your brain that help you experience pleasure and reduce pain

Punishment signaling in the brain

helps your recognize negative outcomes, experience unpleasant feelings and learn to avoid behaviors that lead to bad consequences

Insular cortex

recognizing unpleasant experiences

dACC

helps brain monitor outcomes, detect mistakes, and recognize when something is going wrong or when an outcome is worse than expected

Operant conditioning=

behavior produces an outcome

Thorndikes law of effect=

behaviors with desirable outcomes happen more often

Operant vs classical conditioning

operant= your action matters; classical= the outcome can occur without your action

Discriminative stimulus=

a signal taht a behavior can produce an outcome

Response (R)=

the behavior you perform

Outcome (O)=

what happens because of the behavior

Shaping=

reward small steps toward a goal

Chaining=

link behaviors together like a chain links

Reinforcer=

increases a behavior

Punisher=

decreases a behavior

Drive reduction theory=

reduce a biological need

Primary reinforcer=

naturally satisfies a biological need (food, shelter)

Secondary reinforcer=

gains value through association with other rewards (money, grades)

Positive reinforcement=

add something to increase behavior

Negative reinforcement=

remove something to increase behavior

Positive punishment=

add something to decrease behavior

Negative punishment=

remove something to decrease behavior

Immediate reinforcement=

the closer the reward is to the behavior, the easier the connection is to learn

Delayed reinforcement=

a delay makes it harder to know which behavior earned the reward

Precommitment=

set rules beforehand to help yourself stick to a goal

Continuous reinforcement=

reward every correct response

Partial reinforcement=

reward only some responses

Fixed ratio=

a set number of responses (get paid for every 10 items completed)

Variable ratio=

an unpredictable number of response (keep playing a slot-style game because the reward comes unpredictably after responses)

Fixed interval=

a set amount of time (check for weekly paycheck on payday)

Variable interval=

an unpredictable amount of time (check your phone for messages that arrive at unpredictable times)

Concurrent reinforcement schedule=

you can choose between behaviors that produce different rewards

Matching law=

you tend to respond more where you receive more reinforcement

Premack principle=

use a preferred activity to encourage a less preferred one

Response deprivation hypothesis=

restricting access to an activity can make access to it more rewarding

Caudate nucleus=

choices and goals

involved in movement initiation, goal-directed behavior, positive outcomes, and habit formation

Putamen=

movement

connected with motor and sensory processing, including aspects of pain

Orbitofrontal cortex=

OFC= outcomes and flexible choices

helps evaluate rewards and update decisions when an outcome changes

Wanting=

motivation

dopamine is involved in motivational processes that help you drive you toward a reward

Liking=

pleasure

endogenous opioids contribute to the pleasurable value of an outcome and can dampen pain

Incentive salience hypothesis=

dopamine helps explain how rewarding cues gain motivational pull- how much you want something and will work for it

Wanting and liking together

you might strongly want something without enjoying it very much

Insular cortex=

helps process the unpleasant or hedonic value of punishers (how bad does it feel?)

Dorsal anterior cingulate cortex (dACC)=

helps motivate actions to stop or avoid pain and relates to the effort made to reduce it (what effort should I make to stop it?)

How timing affects learning

immediate outcomes usually produce the fastest learning

delayed outcomes can accidentally become associated with other behaviors that occurred in the meantime

Timing and superstition

if an outcome happens after unrelated behavior, the organism may mistakenly connect the two

Timing and self-control

temporal distance matters for decision making

a delayed larger reward is often harder to choose than a smaller immediate reward