Stimulus Control, Concept Learning, Comparative Cognition, Observational Learning, Motor Skill, and Choice

Stimulus Control and Concept Learning

Stimulus Control:
- How behaviors become controlled by preceding stimuli.
Compound Stimulus:
- Reynolds (1961): Pigeons peck a red key with a white triangle.
- Treated as two separate stimuli.
Measuring Stimulus Control:
- Generalization Gradients:
  - Guttman and Kalish: Pigeons reinforced for pecking a yellow (580 nm) light.
  - Tested on other wavelengths without reinforcement.
  - Steepness indicates stimulus control.
- Causes of Generalization Gradients:
  - Innate:
    - Pavlov: Generalization is an automatic byproduct of conditioning.
  - Experience:
    - Lashley and Wade: Explicit discrimination training is necessary.
    - Generalization is learned.
    - Three groups of pigeons, VI schedule with food reward
    - Group 1 - Nondifferential training: All trials the same - key lit, 1000 Hz tone played
    - Extinction trials with different pitches
    - No discrimination training, no generalization gradient - supports Lashley and Wade
    - Group 2 - presence-absence training:
      - Two types of trials:
        Key-lit, 1000 Hz tone played, reward
        Key lit, no tone, no reward
      - S+ = 1000 Hz tone
        S+ = discriminative stimulus
        S- = discriminative stimulus for absence of reinforcement
      - Test with other pitches
    - Discriminative training → generalization gradient
    - Group 3 - intradimensional training
      - Two types of trials:
        Key lit, 1000 Hz tone played, reward
        Key lit, 950 Hz tone, no reward
      - S+ = 1000 Hz tone
      - S- = 950 Hz tone
      - Test with other pitches
    - Discriminative training → generalization gradient
Absolute vs Relational Stimulus Control:
- Chicken Study (Kohler, 1939):
  - Chicken presented with two cards, lighter (S+) and darker (S-) gray
  - After training, what has chicken learned?
    - Absolute stimulus control - lighter gray card is rewarded
    - Relational stimulus control - the lighter of the two cards is rewarded
  - Kohler trained chickens to peck at a medium gray card and not at a dark gray card
  - Then given a medium gray and lighter gray card
  - All chickens preferred to peck at the lighter gray card
  - Relational stimulus control
- Peak Shift Effect:
  - Hanson trained pigeons to peck at 550 nm light (S+) (control)
    - Another group also had S- at 590 nm (open squares)
    - Another group also had S- at 555 nm (open circles)
  - All groups tested with various wavelengths
  - For S- conditions, peak of generalization gradient shifted away from S- wavelength
  - What it tells us about absolute vs relational stimulus control
    - If absolute stimulus control, all groups should peck at the 550 nm light more frequently
    - If relational stimulus control, pigeons should learn to peck at the greener (550 nm) light than at the less green (555 nm) light. Because 540 nm is even more green, it should be pecked at with a higher rate
  - Responding to stimuli that are less like S-
- Spence’s Theory of Excitatory and Inhibitory Gradients:
  - Excitatory gradient develops for S+
  - Inhibitory gradient develops for S-
  - Net associative strength determined by adding the inhibitory gradient to the excitatory gradient
  - Peak-shift explained
- Intermediate-Size Problem:
  - What it tells us about Spence’s theory and about absolute and relational stimulus control
    - Squares with 9 different sizes
    - Chimpanzees trained with squared 1, 5, and 9 arranged in different orders
    - Rewarded if they selected intermediate sized square
    - Tested with three squares, say 4, 7, and 9
      - Relational: pick square 7
      - Spence: pick square 4
        Square 4 is closest to peak of net associative strength of 5
    - Results support relational theory
- Lazareva et al. trained pigeons with pairs of circles
  - On some trials S+ = circle 2, S- = circle 1
  - On other trials S+ = circle 6, S- = circle 5
  - Tested on circles 3 and 4
    - Circle 3 is similar to circle 2 (an S+) in size
    - Circle 4 is similar to circle 5 (an S-) in size
  - Spence’s theory says pigeons should peck at circle 3
  - Relational says pigeons should peck at larger circle (#4)
  - Pigeons preferred circle 4 over 3
Behavioral Contrast:
- Definition:
  - Responding in the presence of one stimulus changes because of a change in the reinforcement conditions during another stimulus
- Concurrent Schedules:
  - Guttman - pigeons peck on key with two discriminative stimuli
    - Noise: VI 30 schedule
    - Light: separate VI 30 schedule
  - Responds equally to the two S+
  - In phase 2, noise was switched to extinction
  - Light increases rate of response
- Positive Contrast:
  - Increase in rate of response
  - Reinforcement in one component of a multiple schedule is decreased (made less frequent or eliminated)
- Negative Contrast:
  - Decrease in rate of response
  - Reinforcement in one component of a multiple schedule is increased (more frequent or higher quality reinforcement)
- Theories of Behavioral Contrast:
  - Behavioral Reallocation:
    - Phase 2 requires fewer responses (extinction) so animal can recover from fatigue more quickly
  - Reinforcer Habituation/Satiation:
    - Fewer rewards given during phase two (extinction) so animal becomes less habituated/sated
  - Comparison of Reinforcement Rates:
    - Animal compares rate of reinforcement across the two schedules. During phase two, the reinforced schedule is more attractive than the extinction schedule
Concept Learning:
- Structure of Natural Concepts:
  - Central and Peripheral Instances:
    - Placing examples, usually of real-world stimuli, into appropriate (natural) categories
      - Boundaries of categories are not distinct
      - Central instances (central exemplars)
      - Peripheral instances / exemplars
    - How do people demonstrate that central exemplars are shared (agree about centrality of examples, list central examples first, take longer to categorize peripheral examples)
      - Agree about centrality of exemplars
      - List central exemplars before peripheral
      - Take longer to categorize peripheral exemplars
- Theories of Concept Learning:
  - Exemplar Theories:
    - Experienced examples are stored. If novel object is similar to the stored examples, the novel object is categorized
  - Prototype Theories:
    - A single, best (or average) representation is stored. If novel object is similar to the prototype, the novel object is categorized
      - Very similar to prototype - central exemplar
      - Moderately similar to prototype - peripheral exemplar
      - Dissimilar to prototype - not a member of the concept
  - Feature Theories:
    - Specific features of the category are stored. If novel object has enough of those features, it is categorized
- Animal Studies on Concept Learning:
  - Herrnstein showed pigeons 80 slides, 40 with trees (or their parts) and 40 without
  - Reinforced (VI schedule) if pecked slide with tree, no reinforcement if pecked slide without tree
  - Learned quickly
  - Tested on novel slides
  - Responded similarly to the original 80 slides
  - Pigeons trained to categorize people, water, fish, letters of alphabet
Stimulus Control in Behavior Modification:
- Study Habits:
  - Fox - poor study habits are often the result of ineffective stimulus control
    - Had students study in a particular place at a particular time of day with only study materials present to establish stimuli (time and place) associated with studying
    - Students had substantial improvement in grades
      - Improvement may be due to study skill training that the students also received
- Health Habits / Insomnia:
  - People without insomnia may have problems falling asleep in a bed other than their own; people with insomnia easily fall asleep in a bed other than their own
    - Stimulus control
  - Bed can become associated with behaviors not associated with sleeping
    - Reading, watching TV, eating
  - Only sleep in bed - if you can’t fall asleep, get out of bed until you are sleepy

Comparative Cognition

Cognition and Comparative Cognition:
- Applying concepts from cognitive psychology to animals
Memory and Rehearsal:
- Working Memory: memory that holds information for a short period of time, has limited capacity, and guides whatever tasks are being performed
  - Delay-matching to sample (DMTS)
  - Single vs two rules in DMTS
    - Single: respond to same color as sample
    - Two: if sample is red, respond red; if sample is green, respond green
  - Retroactive Interference:
    - New learning interferes with previous learning
      - Learn list A, learn list B, test list A
      - Learning list B interferes with memory of list A
  - Proactive Interference:
    - Previous learning interferes with current learning
      - Learn list A, learn list B, test list B → poorer performance
  - Spatial Memory:
    - Radial-arm mazes
      - Food cups at each arm
      - Rats do not visit the arms in a fixed order
      - Rats very good at visiting each arm once in 8 and 17 arm mazes
        8 arms: 7 or 8 correct responses
        17 arms: 15 correct responses on average
      - In 8 arm maze, after visiting four arms, rats removed for 24 or 48 hours. Remembered which arms were unvisited when returned to maze
  - Rehearsal:
    - Maintenance:
      - Repeating or processing material
      - Keeps info in working memory
    - Associative:
      - Transfers info to long-term memory
  - Directed Forgetting
    - Remember trial: pigeons 90% accurate
    - Forget trial with probe: pigeons 70% accurate
Long-Term Memory, Retrieval, and Forgetting:
- Long-term memory: very large capacity memory system that can store information for long periods of time
  - episodic memory: a type of LTM that stores episodes/events of a person’s life
    - What, where, when
    - Animals have episodic memories
- Forgetting curve:
  - Rapid rate of “forgetting” at first, slows with passage of time
  - Animals show proactive and retroactive interference
- “Forgetting” causes
  - a. Context-shift effect (context specificity)
  - Memory recovered with retrieval cue
- Chunking:
  - Organizing to be remembered material into a smaller number of meaningful categories
  - Terrace presented pigeons with 5 objects (a) that they had to later peck in the correct order
    - Objects organized in 5 different ways, some of which promoted chunking (II, IV) (all colored objects first, one chunk, black and white objects second, another chunk)
  - Chunking conditions required less practice to learn
  - Pigeons paused between chunks
Timing:
- Internal clock
  - Peak procedure
    - Train an individual on an FI schedule
    - Once trained, occasionally include probe trials in which the animal is not reinforced and which lasts twice as long as the normal FI schedule
    - Animal responds as if it knows the original duration of the FI schedule
    - Train an individual to respond by pressing one bar after a 5 second tone, but respond by pressing a different bar after an 8 second tone
    - Rats and pigeons can do this if the different in durations is at least approximately 25% of the shorter duration
  - Weber’s law
    - \frac{\Delta T}{T} = k
    - Rats and pigeons can distinguish 4 and 5 seconds: \frac{(5–4)}{4} = 0.25 (25%)
    - Rats and pigeons cannot distinguish 10 and 11 seconds: \frac{(11–10)}{10} = 0.10 (10%)
  - Theories of timing
    - Clock is a stopwatch
      - Can be started and stopped
    - Behaviors used as a clock
      - Behavioral theory of timing
Counting:
- Approximate
  - Mechner trained rats on an FR schedule
  - On half of trials, reinforced after the appropriate number of bar presses on bar A
  - On other half, reinforced only if rat presses bar A at least the appropriate number of times and then presses bar B
    - If switches too soon, counter resets to 0
  - Obeys Weber’s law
- Exact
  - Probably can count exactly for small numbers
  - Pepperberg trained her parrot Alex to say how many objects were present (from 2 to 6)
    - Alex was right about 80% of the time
Animal Language:
- Primates
  - Terrace’s (1979) summary of language in chimpanzees
    - Poor grammar
    - Frequently string signs together in random order
    - Relied heavily on imitation and prompting
    - Showed little spontaneous use of language
    - Complexity and length of sentence did not increase with additional training
    - Lacks many essential characteristics of human language
- Other species
  - Kanzi - bonobo
    - Mother - Matata, received language training with a lexigram board
    - Kanzi watched but was not reinforced
    - Spontaneously learned lexigrams
    - Learned to recognize spoken words
    - Knew over 300 lexigrams
    - Most (85 to 90%) of “utterances” were a single word
  - Chaser, a border collie
    - Learned names of 1022 objects which she could fetch from a field of 8 objects
    - Chaser did not learn about word combinations
  - Dolphins
    - Trained to respond to about 25 gestures
    - Frisbee fetch basket
    - Similar result for sea lions
  - Parrots (Alex)
    - Spoke about 50 words and could answer questions
- Conclusions about language in animals
  - Can use abstract symbols
  - Mixed evidence for productivity
  - Limited evidence for anything beyond the most basic grammar skills
  - Mixed evidence for displacement
  - Some evidence for communication
Object Permanence:
- Objects continue to exist even if they are not visible
- Piaget’s stages of object permanence
  - 0-1 months - limited awareness of objects
  - 1-4 months - continue to look where an object was
  - 4-8 months - search for partially hidden object; no attempt to retrieve fully hidden object
  - 8-12 months - able to find objects when the concealment is seen; A not B error
  - 12-18 months - cannot locate objects outside of their perceptual field
  - 18-24 months - object permanence
- Dumas’ (1992) study on object permanence in cats
  - Dogs and cats move through approximations of piaget’s six stages of object permanence
  - Cat stands behind glass, sees ball go behind wall, goes and looks for ball behind the wall
Analogies:
- A is to B as C is to D
- Sarah, a language trained chimpanzee, could often solve perceptual analogies and conceptual analogies
  - Perceptual: red square is to red circle as blue square is to blue circle
  - Conceptual: lock is to key as can is to can opener
Transitive Inference:
- If A < B and B < C, then A < C
- Gillan found chimpanzees were capable of transitive inference
  - Taught chimps that yellow < red and red < blue for amount of reward
  - Test with yellow and blue cups
- Rats, mice, pigeons, and wasps also can do transitive inferences
Tool Use and Manufacture:
- Kohler observed chimpanzees stacking boxes to reach food
- Kacelnik and colleagues studied tool use in New Caledonian crows
  - Could select stick of proper length to retrieve food in a tube
  - Could create a hook from a piece of metal to grab handle of bucket and raise it to get food

Observational Learning and Motor Skill

Theories of Imitation:
- Imitation as an instinct:
  - Basic findings (Meltzoff & Moore, 1977, Horne & Erjavec, 2007)
    - Meltzoff & Moore
      - 12 to 21 day old infants can imitate simple gestures
      - Limited to a few special behaviors
    - Horne & Erjavec
      - Found that 1 to 2 year olds can be trained to imitate gestures, but do not spontaneously imitate gestures they have not been trained on
  - True imitation
    - (imitation of a behavior it has never done before) occurs in some species - monkeys, gorillas, orangutans, rats, and quail
- Imitation as an operant response:
  - What are S, R, and O?
    - Discriminative stimulus is the behavior of the to-be-imitated individual
    - Appropriate response is the behavior of the individual
    - If imitation is reinforced, it is likely to be repeated
  - Imitation as a generalized operant response
    - Generalized imitation
      - If a child is reinforced for imitation, the imitation will likely generalize to other situations without reinforcement
- Bandura’s theory of imitation
  - Bandura’s study
    - Children watched film of an adult behaving aggressively toward a Bobo doll
      - One group saw adult being rewarded for aggressive behavior
      - One group saw adult being punished for aggressive behavior
      - One group saw adult receiving no consequences for aggressiveness
    - Children enter room with Bobo doll and play with objects
      - Consequences influenced child’s behavior
    - Children offered rewards for imitating adult’s behavior
      - All three groups have large and equal amounts of aggressive behavior
  - Why can’t generalized imitation explain Bandura’s results?
    - Why consequences to model influenced behavior of learner
    - Why some children did not imitate until rewarded to do so
  - Factors that determine if imitative behavior will occur or not
    - Attentional processes
    - Retentional processes
    - Motor reproductive processes
    - Incentive and motivational processes
- Mirror neurons and imitation
  - What are mirror neurons?
    - Have neural activity when an individual makes a certain movement or when the individual observes another individual make the same movement
  - What do they do for us?
    - Help us understand the actions, intentions, and feelings of others
    - Important for observational learning - help make connection between seeing someone else perform a behavior and then performing is ourselves
Effects of the Mass Media:
- TV
  - Can affect attitudes and behaviors of children and adults
    - Modest effect of TV violence on aggressive behavior
- Social media
  - Good and bad effects
- Video games
  - Playing violent video games increases aggressive behaviors in some studies
    - Others found no positive correlation between video game use and rates of violent crimes in the US
  - Playing video games can improve attention and perceptual skills
- Music
  - Some studies find that listening to music with violent lyrics can increase aggressive thoughts and behaviors in adolescents; some do not
Modeling in Behavior Therapy:
- Ways a model can influence behavior
  - By facilitating responses the observer already knows
  - By allowing the observer to learn new behaviors
  - By reducing fear reactions to harmless objects
- What modeling can be used for
  - Facilitation of low-probability behaviors
    - Increase the likelihood of low-probability behaviors
    - Assertiveness training
  - Acquisition of new behaviors
  - Elimination of fears and unwanted behaviors
- Video self-modeling
Learning Motor Skills:
- Variables affecting motor learning and performance
  - Knowledge of results (KR)
  - Which is more important for improving performance: information or reinforcement?
    - information
  - Knowledge of performance (KP)
    - Many tasks require a complicated sequence of behaviors before the goal state is reached
    - KP provides feedback about the performance of the component behaviors
  - Variables affecting motor learning and performance
    - Mental practice and mental rehearsal
      - Same brain areas most active during mental and physical practice
    - Observational learning with self-practice
      - Can be beneficial for learning a motor skill
    - Transfer from previous training
      - Positive transfer
        Practice on one task leads to faster motor skill learning on another task
      - Negative transfer
        Rare and usually short term
Theories of Motor-Skill Learning:
- Adams’ two-stage theory
  - Verbal-motor stage
    - Improvement depends on feedback (usually verbal)
    - Development of the perceptual trace
    - Sparrow and Summers - positioning task, but KR is only delivered on some trials while learning is in the verbal-motor stage
  - Perceptual trace
    - Internal sensation that allows the learner to discriminate a good movement from a bad one
    - Should be strengthened on KR trials, but decay on non-KR trials
  - Motor stage
    - Improvement depends on internal feedback from the perceptual trace
- Primarily for single, repetitive movements
- Schmidt’s schema theory
  - people acquire general rules (schema) as they practice
  - Individual practice examples are soon forgotten but general rule (motor schema) is remembered
  - Variability in practice
    - Important for the development of motor schema
      - Toss beanbag to target
        Group 1: target always 3 feet awat
        Group 2: target either 2 or 4 feet away
      - Test accuracy of tossing to target that is three feet away, variable practice group did better
  - Application to novel situations
Learning Movement Sequences:
- Movement sequences
  - Movements that must be made in a specific order and correct timing
- Response chain
  - The kinesthetic sensations from each movement in the sequence acts as a discriminative stimulus to start the next movement in the sequence and as a conditioned reinforcer
  - Because the behavior is reinforced multiple times during practice, the behavior should become faster
- Motor programs
  - Definition
    - A neural mechanism that controls a sequence of movements that does not rely on sensory feedback from one movement to initiate the next movement
  - Evidence for motor programs
    - Some do not rely on sensory feedback
    - Human reaction times are too slow to serve as feedback in a rapid sequence of behaviors
    - Skilled motor sequences can be made by people without sensory feedback
    - Errors made in the sequence would halt a response chain
    - Time to start a movement sequence is proportional to the number or movements in the sequence

Choice

Herrnstein’s matching law
- The matching law
  - Allowed pigeons to peck two keys
    - Key one had a VI 135 second schedule
    - Key two had a VI 270 second schedule
  - After much training, how did the pigeon distribute its pecks across the two keys
  - B = number of responses of a given type
  - R = number of reinforcers from a given response
  - Proportion of responses directed toward one alternative should equal the proportion of reinforcers delivered by that alternative
  - Rate of response does not depend solely on the schedule of reinforcement
  - It also depends on the rates of reinforcement of other activities the individual may perform
- Deviations from matching
  - Undermatching
    - Response proportions are less extreme (closer to 0.5) than reinforcement proportions
    - May be due to rapid switching between the two options or misattribution of reinforcer to response
  - Overmatching
    - Response proportions are more extreme than reinforcement proportions
    - Rare, but occurs when penalty exists for switching responses
  - Bias
    - Spending more time with one response than predicted by matching
    - Can occur if one response is easier than the other or animal prefers one side of the cage over the other
- Generalized matching law: equation, s, b
  - B1/B2 = b(R1/R2)^s
  - S < 1 for undermatching
  - S > 1 for overmatching
  - S influenced by species tested, difficulty in switching responses
  - B ≠ 1 → bias
    - B > 1 more B1 responses than unbiased
    - B < 1 more B2 responses than unbiased
  - B influenced by effort to make each response, relative attractiveness of reinforcers
- Varying quality and amount of reinforcement
  - B1/B2 = (Q1A1)/(Q2A2)(R1/R2)
  - Q is the quality of each reinforcer
  - A is the amount of each reinforcer
  - Not as accurate as other equations in some situations
Theories of choice behavior
- Optimization theory
  - Individual make choices that maximizes their satisfaction
    - Maximizes the gain in the long run
- Momentary maximization theory
  - At each moment, individual selects the response that has the highest value at that moment
    - Maximizes the gain in the short run
  - Many studies show that an individual’s behavior varies moment by moment
Self-control choices
- A choice between a small immediate reinforcer and a larger delayed reinforcer
- Preference for the two options can change across time
- Optimization theory says that the preference should not change across time
- Delay discounting
  - Value of a reinforcer decreases as its delay increases
- V = \frac{M}{(1 + kD)}
  - V = value of reinforcer
  - M = reward magnitude
  - D = delay
  - K = discounting rate parameter
  - Factors that influence k
    - IQ
    - Education level
    - Income
    - Drug abuse
- Ainslie-Rachlin theory
  - Assumptions
    - Value of a reinforcer decreases as the delay between making the choice and receiving the reinforcer increases
    - Individual chooses the reinforcer with the highest value at that point in time
    - Opposite prediction with punishers
  - Pre-commitment
    - Decision made in advance in such a way that it is difficult to change
- Self-control in children
  - Factors affecting self-control in children
    - Child could terminate the 15-minute wait at any time and get the less preferred snack
    - Harder to wait if snacks were visible
    - Some told to think about the snacks while others not. Thinking made waiting harder
    - Individual differences in self-control
      - Age, IQ, educational level, income
  - Techniques for improving self-control
    - Pre-commitment
    - Provide immediate reward for selecting the delayed reinforcer
    - Attach punishment to the immediate option to make it less desirable
    - Distract from larger reward during delay
    - Visualize the delayed reinforcer
    - Start with no delay for larger reward and then gradually introduce the delay
- Self-control in other species
Other choice situations
- Risk taking
  - Risk prone - prefer risky alternative
  - Risk averse - prefer safer alternative
  - Juncos are risk prone when food is scarce and the riskier option might provide more food
  - Humans are risk prone when resources are scarce and risk averse when resources are plentiful
    - May be qualified by how close the person is to their goal
- Tragedy of the commons
  - Shared resources invariably are used to exhaustion
    - Benefits of using the shared resource go entirely to the individual
    - Costs of overuse the shared resource are distributed across the group
  - Ways to avoid
    - Making it difficult for individuals to act selfishly
    - Attaching punisher to small, immediate alternative and/or reinforcer to the larger, delayed alternative
    - Reminding of consequences of overuse