Conditioning Midterm

Purposes of animal use

1. Fundamental research (59%)

2. Development of products or devices (15%)

3. Medical or clinical studies (15%)

4. Regulatory testing (5%)

5. Education and training (5%)

Categories of invasiveness

A) Experiments on most invertebrates or live isolates

B) Experiments that cause little or no discomfort or stress

C) Experiments that cause minor stress or pain of short duration

D) Experiments which cause moderate to severe distress or discomfort

E) procedures which cause severe pain near, at, or above the pain tolerance threshold of conscious animals

The three R’s

Replacement- find ways to use non-animal methods or organisms of lower sentience

Reduction- develop strategies to use fewer animals and obtain sufficient data to answer the question posed

Refinement- modifications in husbandry or experimental protocols that reduce pain or stress in animals

Advantages of animal use

1. Minimize expectancy/placebo effect

2. Not motivated to please/displease experimenter

3. Convenient

4. Can be kept in studies for prolonged periods of time

5. Environment can be controlled

6. Easier to discern basic principles of learning in less complex organisms

Disadvantages of animal use

1. Can’t be used to study language, reading, solving of complex problems

2. Some argue not possible to generalize from animals to humans

3. Ethical concerns

Learning

Acquisition, maintenance, and change in behaviour as a function of experience

Factors besides learning that change behaviour

Fatigue, motivational or physiological state, maturation, damage to nervous system/brain tumors, evolution

Why does learning occur?

To survive a species needs to be able to change behaviour in response to changes in the environment

Selection by environment

1. Natural/Darwinian selection: occurs over generations through reproduction to increase the frequency of genes that underlie characteristics that enhance survival and reproduction

2. Selection by operant conditioning: selection of behaviour of an individual within their lifetimes by the environments within which they live

3. Cultural selection: selection of behaviours of groups that endure beyond lifetime of a single individual

Continuity

Processes mediated by the nervous system, that enab;e behaviour to change in response to a changing environment are common across species

Objective of the study of learning

Descriptions of how learning occurs that that generalize across situations and species. What is learned varies across species as well as how fast learning occurs. The assumption of universal processes does not deny the diversity that may cause variations in responses between humans/animals

Ivan Pavlov

Studied physiology of digestion. Received the 1904 Nobel Prize. Considered behaviour reflexive; innate (hard wired). Discovered “conditioned reflex”- anticipatory reflexes were acquired. Any arbitrary stimulus that preceded food can be conditioned to elicit salivation

John Watson

Defined psychology as a science based on observable behaviour. 1913 “psychology as the behaviourist views it”. Thought behaviourism was the only way to build the science of psychology and that introspection had no place in science. Conditioned Little Albert to be afraid of furry things (generalized fear)

Edward Thonrdike

Affect of success and failure on behaviour. First to study operant conditioning (refers to it as trial and error). Puzzle box experiments lead to “Law of Effect”

B.F. Skinner

Natural science approach to human behaviour. Behaviour is the result of genetics and environment, not free will. Internal private events were not causes. Advocated for positive reinforcement over punishment

Hallmarks of behavioural approach

Emphasis on external events. Predominantly animal research. Goals are predictions and control behaviour

Types of behaviourism

1. Methodological Behaviourism (Watson): learning is the development of a connection between environmental event and a specific behaviour [Stimulus-Response theory]

2. Neobehaviourism (Hull): believes internal events mediate between environment and behaviour [mechanistic]

3. Cognitive (purposive) Behaviourism (Tolman): Intervening variables like cognitive processes explain behaviour

4. Social Learning Theory (Bandura): reciprocal environment influences how we think and behave, which in turn influences our environment

5. Radical Behaviourism (Skinner): external events as determinants of behaviour , rejects internal events as determinants of behaviour

Doctrine of determinism

All human behaviours are determined by physical causes. Experience of agency is a perception generated y the brain (linguistic areas)

Doctrine of Free Will

Philosophy that a non-physical entity (mind, will, soul) directs human behaviour; therefore not all human behaviours can be explained by physical causes

Functional analysis

Uses experiments to show the functional relationships between manipulated environmental events (context) and measured response. Classify a behaviour according to its response function and analyzes the environment in terms of stimulus functions

Functional relationship

Manipulate an aspect of how context interacts with an individuals behaviour. Discern how behaviour changes in tandem with changes in context. Relates to variance in context to variance in behaviour

Two types of response functions

1. Respondent: behaviour that increases or decreases based on presentation of a stimulus that precedes behaviour (elicited)

2. Operant: emitted behaviour that increases or decreases based on stimulus events that follow a behaviour (consequences)

Response class

Behaviours that serve a similar purpose or produces the same consequence/effect)

Respondent conditioning

Arbitrary stimulus that precedes an unconditioned stimulus (US) acquires the ability to elicit the same response

Operant conditioning

Any stimulus that follows a behaviour to increase (reinforcement function) or decrease (discriminative function) a response

Discriminative function (S^D)

A stimulus that is likely to elicit a response because the behaviour was reinforced in the past (predicts reinforcement)

Stimulus class

Topographically different forms of stimuli that has the same effect on behaviour (i.e. seeing pictures of different dog breeds and saying “that’s a dog”)

Positive reinforcers

Events delivered contingent upon a behaviour that increases the probability that behaviour occurs in the future

Negative reinforcers

Events removed contingent upon a behaviour that increases the probability that the behaviour occurs in the future

Establishing operations (EO)

Any environmental change that temporarily increases the effect of a reinforcement maintaining a behaviour and increases the behaviour that produces the reinforcement

Abolishing Operations (AO)

Any environmental change that temporarily decreases the effectiveness of a reinforcement maintaining a behaviour and decreases the behaviour that produces that reinforcement

Experimental analysis

Investigates the environment-behaviour relations by manipulating an independent variable (IV) and observing the effect that has on the dependent variable (DV) while holding all other variables constant.

Reversal design (ABAB)

Demonstrates environmental control over behaviour of a single organism (single subject)

Signal detection theory

Explains how we decide whether a stimulus is present amid other stimuli. Used to study perception, memory, and decision-making under uncertainty.

Hit= Signal present, detected

Miss= Signal present, not detected

False alarm= No signal, but think it’s there

Correct rejection= No signal, correctly say none

Phylogenetic behaviour

Hard wired behaviour that is related to survival and reproduction that is built into the organism

Fixed Action Pattern (FAP)

Sequence of behaviours involving the entire organism that is typical to specific species and is elicited by a releasing or sign stimulus. Known to all members of a species, not learned, rigid order of behaviours, initiated by specific stimulus

Modal Action Pattern (MAP)

FAP with variations in behaviour that implies some degree of flexibility (i.e. yawning)

Reaction Chains

Sequences of behaviour like MAP that do not go to completion once completed, only progress from one behaviour to the next if appropriate and can start and stop at any point

Laws of Reflex Action

1. Law of threshold: intensity of stimulation to elicit a response

2. Law of Intensity-Magnitude: magnitude of response varies with intensity of stimulation

3. Law of Latency: latency of response decreases as intensity of stimulation increases

Habituation

Decrease in strength of a response after repeated presentation of the eliciting stimulus

Dishabiuation

Recovery of a previously habituated response occurs when the habituated stimulus is presented along with a novel extraneous stimulus

Respondent conditioning

Stimulus-stimulus association. Behaviour control transferred from one stimulus to another by correlating the CS with the US. Conditioned response (CR) is elicited by the Conditioned stimulus (CS) and is refers to as respondent behaviour

Respondent acquisition

After initial CS-US pairing, the CR gradually appears and increases in strength. Asymptotic level: stable maximum level of conditioned responding

Two Factors that affect acquisition

1. Intensity of US affects asymptote and rate of conditioning. The magnitude of the US determines the maximum strength of the CR.

2. Intensity or salience of CS influences how rapidly the association develops

Respondent extinction

Retreated presentation of the CS not followed by US weakens the CR. Test trials= CS not followed by US, multiple test trials= extinction

Respondent level

Strength of the target response before any conditioning occurs

Spontaneous recovery

The strength of CR at beginning of subsequent extinction session is higher than at end of previous extinction session. Association is not lost, weakened, but other contextual stimuli still maintain some control over responding

Respondent generalization

Occurs when a CR happens to values of the CS that were not trained during acquisition. Generalization gradient plots changes in strength of CR with variations along a dimension on which the CS falls. CR becomes weaker the less similar the novel CS is to the acquisition CS

Respondent Discrimination

Subject responds in the presence of one value of a CS, but not in the presence of other CS values. Trained by presenting US following one CS on some trials, while on others a different stimulus is presented that is not followed by the US

Delay conditioning

CS onset precedes US onset by a brief delay. Strongest, most rapid conditioning. 0.5s delay best for quick skeletal responses, 5-30s best for autonomic responses

Simultaneous conditioning

Onset and offset of CS is same as that of US. Perfectly contiguous, yet produces no conditioned anticipatory responding (CS not predictive of US). Conditioning can be observed with second order conditioning

Backward conditioning

Onset of US precedes onset of CS. Contiguity is same as in short delay conditioning. Mixed results

Second order conditioning

Repeatedly pair a neutral stimulus with a first order conditioned stimulus (CS1) until the neutral stimulus becomes a second order CS2 that elicits the same CR as CS1

Control conditions

Shows that changes are due to association between CS and US and not something else.

Types of control conditions

Random control: presents CS and US in random order with respect to each other (CS never preceded US)

Excitatory unpaired control: presents CS and US with long interval between them so an association cannot form

Truly random control: presents CS CS and US in independent random patterns so that by chance they co-occur a few times

Classical conditioning

Associative- stimulus that occur in conjunction with an eliciting stimulus (US), come to elicit a response similar to the UR. Contiguity or correlation (predictiveness) is essential for association to form

Contiguity vs. Correlation

Used to be thought that contiguity was necessary for associations between CS and US to form. Rescorla stated predictiveness or informativeness of CS is more critical than contiguity

Rules of Rescorla’s approach

1. CS is informative or predictive of occurrence of US

2. CS is not informative about the occurrence of US

3. CS is informative or predictive of the non-occurrence of the US

Temporal coding hypothesis

CS predicts when the US will occur in time relative to the CS (before, simultaneous, after). This accounts for findings that show learning can occur with simultaneous, trace, backward conditioning

Conditioned tolerance

When both the CS that predicts the drug and the drug itself are present, the body produces a compensatory response, which reduces the overall impact of the drug

Conditioned withdrawal

When the CS that predicts the drug, but not the drug itself, compensatory opposite effect is experienced as mild withdrawal

Siegel, Hinson, Krank, & McCully (1982)

Organized rats into three groups: same context, different context, control. Rats into same and different context were given repeated doses of heroin to develop tolerance in a specific context. Half the rats then receive high dose of heroin in same context as before and others in a new context. Dependent measure= rate of mortality (32% difference)

Conditioned withdrawal experiment

Rats organize into 3 groups: self administering (administer morphine by pressing lever), yoked morphine (receive morphine when self administering does, lever present), and yoked saline (control group, saline when self administering presses lever). Self administering rats showed a higher rate of withdrawal behaviours than yoke-morphine. Lever and lever-pressing were CS that were strongly predictive of morphine but not for yoked-morphine group

Immunosuppression

Suppression of immune system elicited by a CR to a taste conditioned with cyclophosphamide

Conditioned Asthma

Guinea pigs made asthmatic through injections of egg albumin (US)- activates immune system to produce sensitizing bodies (UR). When egg albumin is later encountered, an antibody reaction occurs in bronchioles of the lungs. Disrupted breathing is UR to allergen US.

Rescorla-Wagner model

The change in associative strength of a CS on any given trial is a proportion of the difference between the maximal level of and the associative strength the CS already strength the CS already has

Rules of Rescorla-Wagner Model

1. If the maximum associative strength of the CS (Vmax) is greater than the associative strength the CS already has from previous trials (Vi), then increase in associative strength of CS presented on trial (excitatory)

2. If Vmax is lesser then Vi, then decrease in associative strength of each CS presented on a trial (inhibitory)

3. If Vmax is equal to Vi, no change in associative strength, since CS is at asymptote

Compound stimuli

Two stimuli are presented before the US. Implications for use of the Rescorla-Wagner model. Associative strength of a compound stimulus is equal to the sum of the associative strengths of its component parts

Comparator hypothesis

Assumes when CS is paired with US, subject learns 3 associations:

1. Between target CS and US

2. Between other contextual stimuli and US

3. Between target CS and other contextual stimuli

Comparator theory

Predicts that following the development of responding to CS with US, the strength of CS for eliciting CR can be enhanced by extinguishing context-US associations.

3 terms involved in Contingency of Reinforcement

Discriminative stimulus (S^D), Operant behaviour or Response (R), Concequences (S^R+)

Operant behaviour (R)

Ontogenetic behaviour. Determined, lawful, analyzed in terms of relationship to environmental events. Behaviour operates on the environment to produce consequences that affect the behaviour

Discriminative stimulus (S^D)

Stimulus that precedes an operant and sets the occasion for the behaviour. Affects the probability of a response occurring based on history of differential reinforcement (reinforced in the presence of a stimulus, but not in its absence)

Reinforcement (S^R)

If the effect of the consequence is to increase the future probability of occurrence of the response in the presence of the S^D, then the contingency is one of reinforcement. If consequences involves the removal of something, its Negative reinforcement (S^R-). If concequences involves the addition of something, it is Positive reinforcement (S^R+)

Punishment (S^P)

If the effect of the consequence is to decrease the future probability of occurrence of the response in the presence of the S^D, then the contingency is one of punishment

Need reduction rule (Hull)

All stimuli that meet a biological need will act as reinforcers. Primary reinforcers such as food, water, warmth, oxygen, avoidance of pain are all inherently reinforcing and play an important role in survival

Drive reduction rule (Hull and Miller)

Strong stimulation is aversive; any behaviour that leads to a reduction in stimulation will be reinforced by that reduction. Strong forms of stimulation referred to as “drives” are hunger, thirst, and sex

Transituationality Rule

A stimulus that is a reinforcer in one situation will be a reinforcer will be a reinforcer in other situations.

Premack principle (Premack)

A reinforcement contingency represents a relationship between two behaviours (i.e. lever pressing and eating) rather than between a behaviour and an event/stimulus (lever pressing and food)

Response deprivation hypothesis (Timberlake & Allison)

If a reinforcement schedule requires a ration between behaviours other than the ratio that occurs in the baseline conditions, the more restricted behaviour will act as a reinforcers for the less restricted behaviour

Stein, Xue & Belluzzzi (1993)

Reinforcement effects with neurons. Reinforcer: injection of a small amount of dopamine onto the surface of a neuron located in the hippocampus. Contingency: burst of action potential

Stereotype vs. Variability

Reinforcement strengthens different responses, thereby reducing variability (more uniform/stereotyped behaviour). However, reinforcement can also be used to increase variability if variability itself is reinforced.

Page and Neuringer (1985)

Pigeons had to peck 8-key sequences for food. Reinforcement only given if the sequence differed from the last 50 trials. Variability increased by 25%. When variability wasn’t reinforced, it decreased by 30%

Extinction

The procedure of withholding reinforcement of a previously reinforced behaviour

Features of Extinction

1. Extinction burst: initial increase in response rate when reinforcement is first withdrawn

2. Variability: operant behaviour becomes increasingly variable as extinction proceeds

3. Force of response: increase in response force or amplitude when reinforcement is first withdrawn

4. Emotional response: extinction induced aggression

5. Discriminated extinction: procedure in which a discriminative stimulus is used to signal a period of extinction

Resistance to extinction

Extinction occurs rapidly after a few reinforced responses, but when operants are reinforced many times resistance to extinction increases. Resistance to extinction reaches a maximum after 50 to 80 reinforced responses

Humphrey’s Paradox

A response that is intermittently followed by reinforcement is stronger than a response that is always followed by reinforcement

Partial reinforcement effect

Extinction happens faster after continuous reinforcement (CRF) than after intermittent reinforcement. This is paradoxical because more reinforcement usually means more resistance to change. Explanation: Under intermittent schedules, the organism learns that rewards aren’t guaranteed every time, so behaviour becomes more persistent when reinforcement stops.

Discrimination hypothesis

Behaviour changes in extinction only if the subject detects the change in reinforcement. Under CRF, the change to no reinforcement is easier to notice, so responding drops quickly. Under intermittent schedules, the change is harder to detect, so behaviour is more resistant to extinction. If discrimination is controlled, CRF behaviour actually shows greater resistance than intermittent.

Behavioural momentum

Different approach to assessing the strength of a reinforced responses. Assumes that a stronger reinforced response will be readily disrupted. Resistance to change depends on association between discriminative stimulus and reinforcer

Spontaneous recovery

At the end of an extinction session, rate of response may be near operant level. Next day, animal begins the session responding at higher than operant level. Amount of recovery decreases over repeated extinction sessions. Recovery is not spontaneous

Reinstatement

Recovery of responding when a reinforcement is presented non-contingently following extinction. Animals that received non-contingent reinforcement showed more recovery of responding than animals that did not

Renewal

Recovery of responding when the animal is removed from the extinction context

Deprivation/restriction

When food or water is used as reinforcement, deprivation or restriction is often used to increase motivation, a deprivation operation. Typically aim to get animals to ~85% of free feeding weight

Reinforcer habituation

Over repeated trials of reinforcement, the value of the reinforcement decreases. One reason is a decrease in deprivation. Another is reinforcer habituation (changes to the reinforcer or interruptions to the procedure can temporarily disrupt this habituation (dishabituation)

Shaping

Developing a new behaviour by reinforcing successive steps towards the desired behaviours. Start with a behaviour that resembles the target, reinforce closer versions over time. Reinforcement strengthens behaviours meeting the criterion, extinction increases variability, helping new behaviours emerge

Schedule of reinforcement

Contingent relations between the response and the consequence. Rule that determines when a response will be followed by a reinforcer

Continuous reinforcement (CRF)

Each and every response produces a reinforcer. Continues until satiation. Typically used for acquisition of new behaviours (shaping). Conjugate reinforcement: properties of reinforcement are tied to particular dimensions of a response. More effortful response results in a proportionally larger reinforcement

Response stereotype on CRF

Topography of response becomes stereotypical on CRF; only see variability with extinction. Response variability may vary inversely with rate of reinforcement. Organism becomes more variable in responding as reinforcement becomes less frequent or predictable

Advantages of Intermittent Reinforcement

1. Less likely to produce satiation

2. Produces higher rates of responding

3. Maintain behaviour for a longer period of time

4. Maintain behaviour longer under extinction