POL 3769 - Midterm #1

Racial Resentment

Trump said wildly politically incorrect things during his campaign; why did people vote for him?

• Media consensus that Trump’s support in both elections largely came from racial resentment

  • Studies support the link between racial resentment and support for Trump

• RR: “A moral feeling that African-Americans violate traditional American values such as individualism and self-reliance”

  • Explanation for why white people tend to get more of the good things, such as money or jobs

  • Someone who is racially resentful would answer the following way

    • They would, in some capacity, agree that African-Americans need to overcome prejudice and work their way up without special favors

    • They would, in some capacity, agree that African-Americans need to work/try harder to be just as well off as whites

    • They would, in some capacity, disagree that African-Americans have gotten less than they deserve over the past few years

    • They would, in some capacity, disagree that African-Americans, through generations of slavery and discrimination, have created conditions making it difficult for African-Americans to work their way out of the lower classes

• Trump supporters are seen to be more racially resentful against African-Americans than Clinton/Harris supporters (sympathetic)

  • Used a multi-item scale in calculations

  • As one moves up the scale, towards resentful, they find that Trump support increases

    • The more racially resentful one is, the more likely they are to support Trump

• Racial resentment predicts Trump support;

• Those who resent African-Americans are more likely to favor Trump than those sympathetic towards African-Americans

  • Those who resent African-Americans are also more likely to favor Trump over Clinton/Harris

Public Opinion

Two definitions:

1) Those opinions held by private persons which the government finds prudent to heed

2) The preferences of the adult population on matters of relevance to the government

V.O. Key observes that speaking of public opinion precisely is a task “not unlike coming to grips with the Holy Ghost”

• In other words, to speak precisely of public opinion is very difficult

• Why?

  • Attitudes are invisible (latent); researchers cannot see them

  • Measuring invisible things is difficult

  • Too many people to survey (268 million voting-age adults)

  • Falling response rates

Attitudes

• Attitudes are latent and cannot be directly observed

• Opinions are verbal expressions of an attitude

• Attitudes can change over time (inconsistent); however, they do not have to, and some people hold consistent attitudes

Attitude Strength:

• Strong attitudes: durable psychological evaluations of an entity that do not fluctuate over time or across contexts

  • Consistently positive, moderate, or negative

• Weak attitudes: unstable psychological evaluations of an entity that have fluctuated over time or across contexts

  • Sometimes positive, moderate, or negative

• Constructed attitudes: No existing attitude; rather, the evaluation is created on the spot, a temporary attitude

  • Berinsky: Ask them a question, and they will give you an answer, even if their political knowledge is low

    • People will answer, even if they have little basis

    • Survey responses are seen as summary judgments over the mass of considerations (the words people encounter can change their considerations)

      • Give us a sample of the considerations on people’s minds

  • Example: A survey respondent being asked about a band they have never heard of for the first time

Political Attitudes

• A psychological tendency to evaluate a political entity favorably or unfavorably

  • Entity: The thing being judged; the object in question (i.e., abortion)

  • Evaluation: Bottom-line judgment (positive or negative) about how we feel toward the entity

  • Tendency: How consistent the evaluation is across time and contexts

    • Do one’s views change or not?

• Political attitudes can be strong, weak, or constructed

• Political attitudes vary across entities, people, and time

  • Attitudes evolve from constructed, to strong, to a bit weaker

    • (Obama 4 —> Obama 12 —> Obama 26)

  • Attitudes towards Trump are not the same as towards Obama

  • Attitudes vary in strength over time (Obama 04 v 12)

  • Attitudes vary in strength across people (Obama 26)

• Attitudes can form depending on what others say (i.e., Obama making fun of Trump at the correspondent’s dinner), based on predisposed notions (i.e., racism, sexism), or on small moments (i.e., Obama’s DNC speech)

• These attitudes cannot always be seen directly, nor can their intensity/directions

  • Need to express the latent some way

    • They are expressed in various forms (i.e., a MAGA hat, through conversation, poll questions, other behavior)

    • There are many different ways to measure one’s feelings towards an entity (their political opinion)

*Opinions on surveys do not perfectly reflect one’s political attitude (question wording matters, constructed attitudes occur, attitudes are invisible)

*Political opinion is the verbal expression of a political attitude

Survey Questions

• When preparing survey questions, always figure out the attitude that is being measured (i.e., attitudes towards Trump’s presidency)

• Pollsters like short, simple, and succinct questions with broad applicability

  • When specific issues are introduced, this can take the focus away from the main attitude (i.e., asking about Trump’s performance economically)

• Opinions should accurately reflect the latent attitude

  • Hard because attitudes are invisible; thus, they cannot be easily measured

  • There are many different questions we can ask to measure Trump’s approval rating; we have to figure out which ones will accurately reflect attitudes towards Trump

    • Question that should produce an accurate reflection of an attitude

      • “Do you approve or disapprove of how Trump is handling his job as president?

    • Question that will not accurately measure the attitude

      • “How would you rate Trump’s performance on inflation”

    • Cannot be too specific as to force people to construct attitudes, but cannot be too general to lack relation to politics

• When measuring political attitudes, either a single- or multiple-question approach can be used

• The question that is used determines the extent to which a certain attitude emerges

• Make sure questions are not worded weirdly, skewed, confusing, dated, or constrained

  • Every respondent interprets questions differently

Multi-Item Scale

• This is the preferred approach; no single best question

  • Elicits a more accurate representation of the attitude and irons out the idiosyncrasies in each question

  • Single questions viewed as imperfect, and measured with error

  • Reduces the noise in measures of public opinion and produces better estimates

• Adds up respondents’ individual responses to two or more questions and then classifies them on a 1-7 scale depending on their response average

• Does not measure the overall attitude, but one’s attitude across specific contexts (snapshots of considerations at a given moment; separate signal from noise)

  • When does one think abortion is permissible (in cases of rape? incest? when the woman does not want to marry the man?)

• However, sometimes these scales classify inappropriately across domains

  • Questions may also be too specific, forcing respondents to construct attitudes

*It is better to use multi-item scales in most cases

Used to measure opinions on abortion

• Do you approve of abortion in cases of rape? incest? (7 total questions to measure abortion attitudes)

• 0 = extreme pro-life; 7 = extreme pro-choice

• Irons out idiosyncrasies; more accurate measure of abortion attitudes

• Majority on the multi-item abortion support scale are 7s

Also used to measure opinions on LGBT support

• Two questions to measure the attitudes

• 0 = anti-LGBT; 7 = pro-LGBT

• Nearly a majority on the multi-item support scale are 7s

Can use responses from both of these issues to measure attitudes towards cultural liberalism

• Add responses together and take the average (in some cases the raw sum is taken)

• 0 = max conservative; 7 = max liberal

• Nearly 40% are 7s in their moral issue position

• However, abortion and LGBT support are only two of many cultural issues (immigration, DEI, gun control)

  • The country may be more conservative on these issues

• Furthermore, Trump has won in 206 and 2024 despite these numbers

  • Clearly, other things factor into play, such as the economy, foreign policy, etc.

The Constraints of Survey Space

• Ideally, multi-item scales would always be used to measure political attitudes, but survey space is limited

  • The survey can only be so long

• Researchers are forced to choose between two trade-offs

  • A better measurement of fewer attitudes (60 Q, 10 attitudes, 6Q per attitude)

  • Or

  • Worse measurement of more attitudes (60Q, 30 attitudes, 2Q per attitude)

Framing (Marijuana v Cannabis)

• Examining the argument that the word marijuana has been framed, specifically tainted, due to its negative connotation (criminalized/foreign)

  • There are also certain social groups associated with the word

    • Thus, using the word marijuana may dampen support for public legislation

    • Testing to see if cannabis, which has a more neutral connotation, may elicit different responses/feelings

      • Interest groups are using cannabis based on this notion

Key Findings

• The survey finds that public opinion is not drastically different when the word cannabis is used (respondents favor legalization at roughly the same proportion)

  • Support for legalization and moral acceptance, and perception of harm only change when the use is said to be medical

    • People who use it medically are viewed as older and responsible, whereas recreational users are lazy teenagers

• Framing or word choice can affect respondents’ choices; it did not, however, in this situation

  • Perhaps using other names such as devil’s lettuce or pot would elicit a more noticeable change among respondents

Populations

• Population: A collective group that one wants to learn more about

  • Most polls aim to show what voting-age adults think

    • Thus, voting-age adults are the population

• Survey results are never based on the entire population

  • Too expensive and time-consuming to interview the entire population

  • Furthermore, not everyone would answer the poll

    • Thus, samples are needed

Samples

• Samples are tiny subsets of the population (part of the population selected for sampling)

  • Pollsters typically draw samples of 500-1500 from the population

  • Pollsters want good samples; this is not guaranteed (uncertainty and the possibility of having a bad sample remain)

• Samples are used to learn what the population thinks

• Different samples yield different results

  • One could take 1,000 samples of the same size, and they would get 1,000 different approval ratings

  • Variations in survey results are often due to the idiosyncratic ways in which survey organizations conduct their polling

  • Thus, samples matter; furthermore, uncertainty always remains on whether a sample will reflect accurately (or if they will be “out of line”)

• Samples/Statistical Inference only work well under specified conditions

• **Statistical Inference: process for generalizing from the sample to the population

Good Samples

• Representative samples that reflect the characteristics of the broader population

  • As they grow, they look more and more like the real population, in observable and non-observable characteristics

    • Includes some form of probability sampling

      • The best samples use some variation of this

      • Assigns every adult an equal probability of being selected and then selects randomly

• Random samples almost always look like the broader population

  • Use statistical inference to generalize to the population

• Yet, uncertainty remains, as even representative samples can miss

Non-representative samples

• Differ from the population in ways that affect the poll’s accuracy

  • Includes both ignorable and non-ignorable non-response

    • For example, having too few non-college whites relative to the population

    • Explains why Trump’s success has routinely been underestimated

• Due to falling response rates, non-representative samples are becoming more common

Ignorable Non-response

• Happens when known factors (easily observable characteristics) affect an individual’s decision to participate in a poll and thus the sample

  • Their opinions are given too much consideration

    • Easily observable characteristics include things like age, race, gender, etc.

      • For example, men have been observed to participate less in polls than women

• This type of non-response can be ignored because it can be corrected with weighting

  • Weighting makes the sample look more like the actual population

    • Example: 0.49/0.42 = 1.67

• The sample is adjusted for known census benchmarks

Non-ignorable Non-response

• Falling response rates have forced pollsters to settle for less representative samples, specifically moving away from random sampling

• Occurs when unmeasured/unknown factors affect the decision to participate in a poll

  • Respondents differ from nonrespondents regarding the characteristic being measured (i.e., ideology)

    • Liberals may be more willing to respond, so the sample is too liberal, but this cannot be weighted for

• Can also appear when people’s willingness to participate relates to their opinion

  • For example, in a survey on gun control, those who own guns may be more likely to participate, skewing the sample

  • However, this cannot be weighted for since it’s an unknown benchmark

• Response rates are falling, and while some nonresponses can be ignored, not all can

• In 1936, Landon’s voters were more likely to be represented than FDR’s

  • Speculative reasons: social trust, enthusiasm

• The people who select into surveys bias the poll

• Bailey says that there are ways non-ignorable non-response can be countered

  • Many surveys tend to just ignore it

  • One solution that he proposes is to inform readers if the survey is vulnerable to non-ignorable non-response

  • Another solution is to recognize non-ignorable non-response in existing data and consider that in the future

    • People more interested in politics were more supportive of Biden and were more likely to answer polls

      • ANES may have suffered from non-ignorable non-response

  • He writes of tools that can measure and correct for non-ignorable non-response, used by Peress

  • Finally, survey design is important; it must create data that make it easier to diagnose and correct for non-ignorable non-response

Sampling Uncertainty & MOE

• Sample-to-population inferences are always uncertain because samples suffer from non-ignorable non-response

• Thus, this uncertainty must be taken into account

Margin of Error (MOE)

• Uses sample estimates to estimate the true population value

• Can be used to quantify uncertainty so long as the sample is random

• Tells us how confident we can be about the actual population value

  • A 95% confidence interval means that in 95/100 samples, the actual population value will be within the range of the MOE

• This is the best that can be done; uncertainty is unescapable

Literary Digest Fiasco (1936) & Ann Selzer (2024) & 2016 Election

• From 1920 to 1932, LD used large polls to correctly predict presidential winners

  • In 1936, it incorrectly predicted Landon would beat FDR

  • FDR won with 61% of the popular vote

• LD used a sample of 2.2 million people

  • The sample was drawn from telephone directories and car owner lists

    • These tended to be wealthier/affluent families, especially during this period

  • The sample was biased towards affluent voters who leaned Republican

• LD used a non-representative sample

  • Meaning this sample did not reflect the broader population

Ann Selzer & Her Big Miss (2024)

• Selzer is a very highly esteemed pollster, with “near-oracular” status

• In a poll she released in 2024, she released a poll predicting Harris winning Iowa

  • Women are driving the shift, especially the older or politically independent

    • In a previous poll she released, she showed Trump leading by 18 points over Biden

      • Iowa has also trended red over the years

• Selzer was wrong, with Trump handily winning Iowa during the 2024 election (14 points)

• Selzer’s poll exemplifies what can happen when a pollster draws on results from a non-representative sample

  • Her poll underestimated Trump’s support

  • Due to non-ignorable non-response

Polls inaccuracy in 2016

• A survey by the Princeton Election Consortium found Clinton had a projected 312 electoral votes

  • Also predicted Clinton had more than a 99% chance of winning the election

• The survey was conducted by Sam Wang, who correctly predicted 49/50 states in 2012

• The NYT gave Clinton an 85% chance of winning; FiveThirtyEight gave a 65% chance

• Of course, all these polls were wrong, with Trump defeating Clinton, with 304 electoral votes

• Trump outperformed, compared to expectations, in PA, MI & WI

• Too few non-college whites in these state samples (not represented sufficiently in the sample)

• Trump supporters were found to be less likely to participate in polls than Clinton supporters (non-ignorable non-response)

  • People with high social trust participate more in polls than people with low social trust

  • People with high social trust are more liberal than people with low social trust

  • Non-college whites in the poll vs. those not in the poll may have more social trust, leading to pro-Clinton bias

  • Pollsters also tend to have a tough time predicting who will vote or not

    • Thus, they cannot define their population properly

Polls & Large Trends

• The previous three examples all illustrate instances in which polls were way off the mark, so can polls be trusted?

  • Bailey argues that poll misfires in recent elections have led some to believe polling is in a crisis

    • Furthermore, they have been criticized as affecting voting behavior and fundraising

• Cohn argues that in 2024, the polls, while having a few big misses, identified demographic and political trends in the American electorate correctly

  • In fact, he argues that polls may have been the best they have ever been in 2024

• The following trends were identified

  • Trump gains among young and non-white voters

  • Trump advantage among low-turnout voters

  • Gains for Trump in Florida and NYC

  • Reduced gap between the popular vote and the Electoral College

• Thus, polls can be trusted, and they can be useful

• They help us learn about the country and public opinion in ways that talking to friends, special elections, and fundraising did not

  • They ascertain public preferences on a variety of policy issues and monitor the public pulse regarding indicators like party or ideological identification

  • They can teach us a lot about what is happening in American politics

  • Presidents seek updates on public attitudes using private polls

Face-to-Face era of polling (1935 - 1974)

• Resulted in highly representative samples using some variation of probability sampling

• Last up to two hours

• Use laptops for visuals and to ask sensitive questions

• Very expensive ($10+ million)

• Data collection and release take months

Telephone era of polling (1975 - 2010)

• Use Random Digit Dialing (RDD) to select respondents

  • A ten-digit phone number is composed of an area code, exchange, cluster, and the two final digits

  • The first eight digits are a “one hundred block,” and phone companies will provide these to sampling firms

    • Sampling firms then randomly sample by randomly assigning the last two digits

• Obtain good representative samples through a method of random sampling

• Last 5-20 minutes

• Cannot use visuals; harder to ask sensitive questions

• Moderately expensive (can easily spend over $100,000)

• Data collection and release takes days/weeks

• Have become more obsolete with the emergence of hand-held cellphones

Online/mixed online era of polling (2011 - present)

• Highly representative samples that use address-based sampling

  • RDD concerns due to decline in the number of landlines

• Last 10-25 minutes

• Can use visuals and ask sensitive questions

• Moderately expensive (can easily spend $100,000+)

• Data collection and release takes days/weeks

Opt-in online surveys

• Less representative samples

  • Compile lists of people who want to participate in surveys, and enter them into a database

  • Then offer them opportunities to participate in polls

  • This is not a random sampling technique

• Those who participate/opt-in are different from non-opters

  • Likely to be more partisan/hardliner

  • Polls using these samples suffer from non-ignorable non-response more often

  • Can use weighting to make the sample more representative, but this does not counter non-ignorable non-response

  • Suffers from the same self-selection problem that doomed Literary Digest

• Last 10-25 minutes

• Can use visuals and ask sensitive questions

• Relatively inexpensive (around $100,000)

• Data collection and release take days/weeks

Results

• Opt-in surveys may be more prone to bogus answers

  • However, this does not make them less useful

    • 2020 opt-in surveys were more accurate than polls using probability sampling

  • Some respondents may be more focused on the payment/reward than the survey itself

  • We should not discount methodologies, but instead be cautious about one poll finding (especially when it depends on accurately sampling a small subgroup and getting them to correctly voice their views on a controversial topic)

• This methodology brings different samples and thus different results (including outliers)

  • Pew Research Center conducted a poll that found only 3% of 18- 29-year-olds denied the Holocaust

• Context matters; people who opt in may be doing so because of world events

  • For example, Israel’s recent actions in Gaza have outraged 18- 29-year-olds

  • May impact their answers (Increased Holocaust denial)

Gold-standard surveys

• Gold standard surveys use probability sampling, which is why they are the gold standard

  • ANES - mixed modes (online and FTF) since 2012

  • GSS - mixed modes (online, RDD, FTF) since 2002

  • NY Times/Siena College - RDD phones (landline/cell)

  • Pew Research Center - address-based sampling to build an online panel

Response Rates

• Response Rates = number of completed surveys / number of survey contacts made

• Response Rates have been falling for decades

  • The Response Rate for NY Times/Siena College polls in 2018 was less than 2%

• The portion of individuals responding is becoming less and less representative

  • Those who do respond are 1%

  • By virtue of responding they are unusual

  • Inevitably, some groups will be underrepresented

  • Can correct for this with weights if we have census benchmarks, but we cannot correct for non-ignorable non-response / selection bias

• As response rates have dropped, more knowledgeable and engaged respondents are overrepresented

  • Are pollsters exaggerating the degree of polarization because of how many ideologically committed people respond

Political Knowledge

• Political knowledge is factual information about government, politics, and public affairs stored in individuals’ long term memory

• Like attitudes, political knowledge is not directly observable (latent)

  • Need to ask questions to figure out how much political knowledge Americans hold

    • These questions cannot be too easy, or too hard