cpsy 0220 - final exam

“system 1” thinking

model-free/associative; operates automatically with little or no effort and no voluntary control. capabilities include innate skills and activities that become fast and automatic through prolonged practice

“system 2” thinking

model-based/rules-based; effortful activities like complex computations that require attention and are disrupted when attention is drawn away. intense focusing on one of these tasks can introduce “task blindness”

interaction of system 1 and system 2

system 1 continually generates suggestions for system 2 (impressions, intuitions, intentions, feelings); impressions can turn into beliefs, impulses can turn into voluntary actions; when all goes smoothly, system 2 adopts the suggestions of system 1.

when an event violates the model of the world that system 1 maintains, system 2 is mobilized.

system 1 falls for cognitive illusions, and system 2 can learn to recognize them; because system 1 operates automatically, it’s difficult to prevent errors and biases

sian beilock’s system 1/system 2 study

novice and experienced golfers and soccer players were asked to putt/dribble while thinking about the mechanics of their skill/doing another task at the same time; novices dribbled through the cones slightly faster when they were skill-focused than when they were dual-tasking, but experts dribbled through the cones slower when they were skill-focused and did much better when dual-tasking

demonstrates that automatic action can sometimes be impaired by “system 2” thinking

cognitive reflection test (CRT)

a brief but powerful way to measure the tendency to engage in deliberative rather than intuitive reasoning through questions that have a simple wrong answer and a more complicated right answer that takes more time to figure out

A-not-B task

a test developed by jean piaget to measure working memory and cognitive control; after habituating to one action (reaching to location A for a prize several times), the prize is moved and a different action is required (reaching to location B); this requires reversal learning

A-not-B in babies and monkeys

from 7.5-9 months old, babies have ~35% correct reaches in A-not-B actions; by 1 year, this goes up to almost 100%.

normal adult rhesus monkeys have 90-100% correct reaches, but adult rhesus monkeys with dlPFC damage have 50% correct reaches after the prize is switched.

cartesian perspective of information processing

when you hear information, you hold it in mind neutrally without deciding whether it is true; then you analyze and decide whether to accept or reject it

spinozan perspective of information processing

understanding is believing; you accept a statement as true in order to understand it, then after acceptance, you can decide whether you actually believe it

david gilbert et al.’s findings on information processing and “unbelieving”

belief and disbelief are not symmetrical; participants in their studies overweighted exacerbating evidence (regardless if it was true or false) if they were under cognitive load, leading to the conclusion that under load, people act more spinozan (automatically accept statements as true)

hebbian learning

influential early model of learning by association; held that stimuli that co-occur become associated in the brain (ex. pavlov’s dog)

blocking (challenge to hebbian learning)

agents learn little from redundant stimuli; not all stimuli that were first presented together produce associations when presented separately

rescorla-wagner model of learning

neurons that fire together unexpectedly will wire together; learning via surprise (prediction error)

prediction error

V(obs) - V(pred); observed value - predicted value

prediction errors can be positive or negative - positive is a good surprise (higher value than expected) and negative is a bad surprise (lower value than expected)

issue with original prediction error model (Vnew = Vold + PE)

beliefs are updated too quickly and cannot stabilize over time

learning rate (α)

resolves the prediction rate model by updating beliefs by using only a fraction of the prediction error. V(new) = V(old) + αPE

α = 0: didn’t learn anything

α = 1: update completely

0<α<1: update partially

reinforcement learning

one plausible model of fast and slow learning and decision-making over time in complex environments; behavior is shaped by rewards, and agents try to choose the right actions given the current state they are in. can be model-based or model-free

model-based reinforcement learning

having a goal and enacting a set of steps to reach that goal given a model of your environment (also known as goal-directed learning). action values are computed by mentally simulating the consequences of said actions within a representational map of the world that includes the states, the transition probabilities between them, and the rewards in each state.

model-free reinforcement learning

learning to associate states with actions. values for different actions are learned directly, through trial-and-error interaction with the environment every time an action is performed. this is done with prediction error; in each state, the agent observes the current state, performs an action based on the estimated values of different actions, observes the results, and updates the value of the action with prediction error. trial-by-trial, prediction errors are minimized

temporal difference learning

answers the question of how agents can learn to take actions that don’t lead to immediate rewards; by treating good options like rewards, the brain can assign credit to reliable cues to future rewards, and as a cue is made to be a reliable signal that a reward will come, a dopamine spike will occur after the cue and diminish after the reward

reinforcement learning equation

V(new) = V(old) + α(Reward + V(best option) - V(old))

how habits are learned (drummond & niv)

habits are bred through repetition, but they are not context independent or irrational; actions are habitized because they have consistently led to favorable outcomes in the past

pros and cons of model-based learning

updates can be made easily when environmental changes occur, but it is more cognitively challenging to maintain a mental representational map

pros and cons of model-free learning

it is computationally easy, but is less flexible in adapting to abrupt changes in the decision environment, as changes cannot be incorporated into learned valyes without experiencing prediction errors

arbitration between the two systems

early in learning, behavior is goal-directed (model-based); once behavior is repeated more extensively, it becomes habitual (model-free). animals can alternate between using the two systems in a given scenario, where actions proximate to the outcome are more likely to be model-based

two-step task

a task that quantifies the extent to which human behavior conforms with either model-free or model-based learning; a choice in the first stage leads to a second choice in one of two second stages with a changing probability of a reward. under model-free learning, the user repeats an action if it was rewarded, regardless of whether it was a "common" or "rare" transition. in model-based learning, the user understands the structure (map) of the task, and will not necessarily repeat the same action, but will instead select the action that generally leads to that rewarding state.

mental accounting

the set of cognitive operations used by individuals and households to organize, evaluate, and keep track of where their money is going; people do not treat money as a fungible good and attach labels to money that influences how they spend it

fungibility

fungible goods or commodities have individual units which are essentially interchangeable

impact of windfall money

in experiments, framing the same payment as a windfall makes people more likely to spend it

topical accounts

evaluating transactions compared to reference points that are determined by relevant mental accounts; the example of the jacket and calculator aligns with prospect theory, in which people are more likely to want to save money on the calculator (a lower price means a bigger change in utility with a discount) than the jacket (a higher price means a smaller change in utility with the same discount)

hedonic editing

consequence of prospect theory and mental accounting, in which people should try to segregate (separate) gains and integrate (combine) losses. separating a gain into smaller gains will result in more utility due to diminishing marginal returns. combining losses will reduce decreases in utility for the same reason

multi-attribute decision components

• the relative importance of each attribute (eg. salary > prestige?)

• cutoff values for each attribute (eg. only consider an option if it has a salary of >$75,000)

• how to trade off attributes ($10,000 per year = 10 min daily commute time?)

the “metachoice”

deciding between different methods of decision-making for multi-attribute decisions

weighted additive rule (WADD)

1. put each attribute on a common scale (like 0-100)

2. weight each attribute according to its importance and how it trades-off with other attributes (assign a value that demonstrates how you care about them in relation to each other; higher = care more)

3. for each option, multiply each value by each weight

4. the utility of the option is the sum of these values

   • calculation is the same as calculating expected utility

   • this assumes the independence of attributes!

equal-weight heuristic (EQW)

same as WADD, but in which each attribute has equal weight. often almost as accurate as WADD but way simpler.

satisficing (SAT)

1. look at options one at a time

2. for the option in front of you, compare the value of each attribute to your cutoff value

3. if any attribute is below your cutoff, reject it

4. choose the first option where all attributes satisfy all of your cutoffs

5. if no option wins, lower your cutoffs or look for other options and repeat

• the order in which options are presented matters!

• but this is useful for situations where options are presented sequentially

lexicographic ordering (LEX)

1. determine the most important attribute and sort options by just that one

2. choose the option(s) best on that attribute

3. if there is one option left, you’re done; if there are more than one left, sort by the second most important attribute and repeat

elimination-by-aspects (EBA)

similar to LEX, but with satisficing (considered more representative of human behavior).

1. probabilistically choose an attribute to focus on proportional by weight (probably focus on this attribute that you care about the most but not always)

2. drop all options which are below your cutoff for that attribute

3. repeat if more than one option is left

frequency of good and bad features (FRQ)

1. use cutoffs to separate good from bad values on each attribute

2. count the number of good and bad attributes for each alternative

3. choose the alternative with the most good attributes

• utility of an option that meets x number of cutoffs is x; the option with the highest x value is chosen

habit heuristic

choose the same option you chose last time

ex) ordering the same dish when you return to a restaurant

• advantages: super low effort, less uncertainty (picking a familiar option)

• disadvantages: you may miss good untested options (& get stuck in a “local maximum”, and options may change in quality

compensatory decision-making strategies

being great on one attribute can “overcome” being bad on another attribute.

this takes more effort compared to non-compensatory strategies because you have to keep multiple variables in mind and assess how to trade-off attributes

non-compensatory decision-making strategies

deciding between options based on a characteristic, where being deficient in this characteristic cannot be outweighed by others

selective information processing

some strategies purposefully ignore information about options, their attributes, or the decision-maker’s preferences. by contrast, non-selective processing tries to analyze all available information.

quantitative vs. qualitative reasoning

basing a decision on quantified characteristics vs. qualitative characteristics influences decision-making; qualitative reasoning is generally less computationally difficult

simplifying heuristics

useful in situations with complex/lots of options, time pressure, stress, low stakes, and no need to justify your choice; these include non-compensatory, non-selective information processing, not calculating overall utility, qualitative reasoning

reason-based choice

preferences are sensitive to how options are described (framing effects), and different frames bring forth different reasons to guide decision. incorporates comparative considerations (relative advantages, anticipated regret) that remain outside the purview of value maximization

disjunction effect

a disjunction of different reasons for a choice is less compelling than any definite reason alone (ex. students more likely to defer buying a vacation package if they didn’t know the outcome of their exams)

overplacement

believing you are better than other people when you are not (ex. driving statistics)

overprecision

excessive faith that you know the truth, despite objective uncertainties

overestimation

thinking that you are better than you actually are

overconfidence

people believe that they are better/more knowledgeable/more skilled than they actually are; people who have worse performance are more overconfident about their performance

overconfidence in children

young children are exceptionally overconfident in their abilities and predict that they will immediately learn a skill, ignoring the exponential decay skill learning function (error reduces over time); at around age 7, children begin to predict gradual improvement rather than instant success

overconfidence in adults

adults generally accurately predict the exponential shape of their learning curves, but their predictions of future performance across trials are overly optimistic. the more optimistic people are about their performance before a trial, the worse they feel after they start the trial.

effect of outside assistance on overconfidence (fisher & oppenheimer)

people often do not realize how reliant they are on outside aids; in a trivia experiemnt, people who received hints were more overconfident in their abilities than people who did not, but this effect was somewhat mitigated if the hints occurred after a delay, which made people realize how difficult the task was.

people who had to press a button to receive hints, they rated their abilities the same as those who did not receive hints.

retrieval fluency & overconfidence

overconfidence occurs when information feels easy to process (as a result of hints)

game theory

the mathematical study of strategic interactions between two or more agents, in which an actor’s best move depends on the best moves of those they interact with (other players).

nash equilibrium

the intersections of best responses among players. in other words, the best strategy (S*) for an agent given the actions of all other agents. it is not the optimal or most likely strategy

a nash equilibrium is stable (no one can increase payoff by changing their actions unilaterally) and self-fulfilling (if everyone privately believes that everyone else is playing the NE, then everyone will)

chicken game

two drivers are in cars, and stopped at opposite sides of a single-lane road. they can each decide to go straight or swerve.

what each player should do depends on what the other one is doing. eventually, you end up with a “balancing selection” where people pursue both strategies

3 nash equilibria: always straight/always swerve, always swerve/always straight, each car decides randomly whether to swerve or go straight

prisoner’s dilemma

2 people are arrested and put in solitary confinement on suspicion of robbing a bank. they face 1 year in prison if they stay quiet (cooperate). if they defect, they will go free but their partner will get 3 years. if both testify against the other, they each get 2 years.

the nash equilibrium is for both players to defect, but this is not the optimal strategy since both players will be worse off than if they had cooperated

how to encourage cooperation (2 ways)

1. punishment - every time someone defects, they are punished by a 3rd party. problem: cooperation will increase, but punishment is usually costly for the punisher. if they fail to punish the defectors, who will punish them?

2. reciprocity - be nice unless the other player isn’t; repeated interactions can increase the stability of results. repeating the last action of another player (tit-for-tat) is a very effective strategy in many games like the PD that are repeated

coordination game

a game-theoretical situation in which players coordinating with each other leads to the positive outcome, but it is difficult to do so

threshold public goods came

the desired outcome only happens if participation reaches some critical value

boosting cooperation in coordination games

• non-anonymity - cooperation can go down in anonymous environments, but some

• repeated games - often very different NEs than one-shot games

• reputations - if your actions are visible and will be remembered by others in the future (social costs)

ultimatum/dictator games

in an ultimatum game (other player can reject an offer), people reject offers of less than 20% of a sum of money about half the time, and proposers anticipate this and typically offer 40-50% of the amount of money

in the dictator game (other player has to take the offer), the proposer dictates the amount of money and still offers an average of about 20-30% of the sum

fairness equilibrium

players’ choices are affected by the “niceness” or “meanness” of the other player’s choices. player 1 has a sympathy coefficient that is greater than 0 when player 2 helps them and less than 0 when player 2 hurts them; these feelings add or subtract utility from money payoffs, but become relatively less important as money payoffs rise

effect of move order in games

simply the knowledge that another player moved first, without knowing what they picked, is enough to convey a first-mover advantage that the second mover respects and will alter their choices from their preferences in order to complement in coordination games.

iterated dominance

when predicting what other players will do, players first rule out play of dominated strategies by all players, then eliminate strategies that become dominated after the first set was eliminated, and so forth.

people tend to use 1-3 steps of iterated dominance

beauty contest game

people guess the average number that others will pick, then pick 70% of that average, while knowing that everyone else is doing the same

the nash equilibrium is picking 0. as rounds of the game progress, choices are drawn towards 0 as subjects learn

paradox of choice

while increasing the number of choices is good for a rational agent (they increase the chances of getting greater utility), in empirical situations, increased choice can decrease utility and satisfaction among decision-makers

fancy jams study (sheena iyengar)

researchers set up a display in a food store featuring a line of fancy jams (either 6 or 24 choices available) and gave customers a discount if they bought a jar; more people (~30% of buyers) bought jam from the small array, while barely anyone bought from the large array. used as an example of the effects of choice overload on decision-making

factors of choice overload

1. difficulty - influenced by time pressure, lots of options, etc.

2. set complexity - is there a dominant option, are all options good, are options alignable on attributes

3. preferences - you may not know what you want

4. decision goals - wanting to spend less time on a decision can increase choice overload

opportunity cost

the sense of loss generated by not choosing the rejected options; the more options available, the greater the opportunity cost, and the less satisfaction will be gained from the chosen alternative

anticipated regret

people predict the level of regret they may experience in the future, and make their choices based on this; regret mediates the relation between maximizing and various measures of life satisfaction, as people are maximizers because of concerns about regret, and the more options you have, the more likely you are to experience regret

expectations of quality

more choices can increase your expectations of how good your final choice will be (related to reference points). the greater the number of appealing choices, the greater the opportunity for regret.

maximizers vs. satisficers

1. maximizers are more likely to engage in social comparison and seek social comparison information in making purchases, and have heightened regret with purchases

2. maximizers are generally less satisfied with their lives and report lower subjective well-being scores

motivated reasoning

tendency to believe evidence that confirms preexisting beliefs and discount information that counters those beliefs; people reason their way to conclusions they favor and preferences influence how information is gathered, assessed, and used

motivated inference

people tend to view attributes that match themselves as more favorable, generating self-serving theories (ex. divorce study, princeton-dartmouth game)

motivated reasoning & goals

motivated reasoning occurs when directional goals (to persuade, to agree, to preserve oneself) overcome accuracy goals; ex. the the “above-average effect” results from a self-enhancement goal or a non-motivated tendency to define traits egocentrically

causes of motivated reasoning

1. prior beliefs - shape how we interpret and value new information

2. prior knowledge - some say it reduces motivated reasoning while others find the opposite

3. social identity - beliefs related to our social identity are defended very strongly

4. motives - we have motives while recruiting and evaluating evidence

motives while recruiting evidence

people look for evidence that supports a desired belief while avoiding evidence that refutes it; people are responsive to reality, but recruit subsets of relevant evidence that are biased in favor of what they want to believe

motives while evaluating evidence

because of cognitive dissonance, people are motivated to reconcile any inconsistencies between their actions, attitudes, beliefs, or values; we evaluate evidence in a way that will reduce this dissonance

confirmation bias

preferentially seeking out evidence in support of our initial beliefs; done to resolve cognitive dissonance

motivated attention

people spend more time looking at information that is in line with their beliefs, and avoid information that is against their beliefs

motivated interpretation

arguments in line with one’s beliefs are judged as stronger, and people counterargument arguments that are against their beliefs; when self-selecting evidence, people choose not to interpret evidence against their beliefs

wisdom of the crowds

a phenomenon where averaged estimates from many people can surpass the average (or even best) accuracy of any individual in the group. requires a few elements: large (enough) group, independence of judgements, diverse set of perspectives, decentralized knowledge

dialectical estimates

when participants are asked to make a second estimate after an initial estimate; these dialectical estimates are not more accurate, but the average of the dialectical and initial estimates is almost as accurate as averaging with another person

crowd influence (asch)

social pressure can cause people to say and even think things that are obviously untrue; in asch’s experiment, people go along with the answer of a crowd (of four or more) even if that answer is wrong 1/3 of the time, and 3/4 of participants gave a wrong answer at least once

having at least one other person going against the crowd can reduce the pressure of majority opinions

peak-end rule

overall judgement of experiences is disproportionately affected by its peak and ending moments; duration of the episode matters much less (duration neglect)

ex. more people chose to repeat a longer cold-water trial that had slightly warmer water near the end of the experience than a shorter cold-water trial that was uniform

affective forecasts

people’s predictions about their emotional reactions to future events, which often display an impact bias, an overestimation of the intensity or duration of the reaction

two causes of impact bias

focalism and sense-making/hedonic treadmill

focalism

the tendency to overestimate how much we will think about an event in the future and underestimate the extent to which other events will influence our thoughts and feelings

sense-making/hedonic treadmill/affective adaptation

forecasters fail to recognize how readily they will make sense of a novel or unexpected event after it happens; explanation speeds recovery from both negative and positive events

AREA model of sense-making

• attend: self-relevant, unexplained events grab attention

• react: attention to the event amplifies emotional response

• explain: people try to understand what happened and what it means

• adapt: people return to emotional equilibrium

immune neglect

people do not take into account the unconscious processes that help them recover from negative emotional events; people predict that a loss will have a larger emotional impact than a gain (loss aversion)

implications of immune neglect on decision-making

people are more motivated to rationalize a decision that is difficult to reverse and less motivated to rationalize a decision that is reversible; as a result, they avoid binding commitments despite the potential greater satisfaction that those commitments would bring

survivorship bias

the tendency to focus on cases that succeeded according to some selection metric, while neglecting those that didn’t; intertwined with heuristics and biases

hindsight bias

people think events are more likely after they have already occurred; in experiments, people rate the same events as more likely if they were told they had already happened, regardless of actual accuracy

“end of history” illusion

people generally judge that they have changed a lot in the past but predict they will change little in the future