Public Opinion and Polling Notes

Public Opinion

Overview

• Transitioning to studying what Americans actually think about politics and government.
• Focus on how public opinion affects government actions.
• Central question: How do we know what Americans think about politics?

What is Public Opinion?

• Definition: Opinions held by private persons that governments find prudent to heed.
• V.O. Key: Defining public opinion precisely is very challenging.

Why Is Public Opinion Important?

• Key component of a healthy democracy.
• Rooted in the foundation of the nation, e.g., "No taxation without representation."
• Questions to consider:
  • What do Americans think about current events and public servants?
  • How do we know what Americans think?
  • Does public opinion shape public policy?

How Members of Congress Learn Public Opinion

• Monitoring:
  • Social media.
  • Protests (who, what, how many).
  • Petitions.
• Correspondence from constituents (mail, email, phone calls).
• Informal conversations.
• Political conventions and assemblies.
• Town hall meetings.

Town Hall Meetings

• Citizens express opinions and activism at town halls.
• Politicians can gauge public opinion, but it may not always be representative.
• Social media offers similar opportunities and challenges.

Uses for Polling

• Polling is a common method to understand public opinion.
• Polls can provide representative samples of populations.
• Other methods are often unrepresentative.

George Gallup on Polling

• Polling is a tool to measure public opinion.
• When leaders pay attention to polls, they are paying attention to the views of the people.

Polls Can Tell Us About…

• Americans’ political knowledge (or lack thereof).
  • Civic and political knowledge.
  • Name recognition for candidates.
• Americans’ political involvement.
  • Participation in the political process.
  • Exit polls.

Ubiquity of Polls

• Polls are widespread, conducted by:
  • Candidates.
  • News media (e.g., CBS/New York Times).
  • Businesses and advocacy organizations.
  • Academics.
• Important to know who sponsors the poll and their objectives.

Drawbacks of Polling

• Small sample size can reduce accuracy.
• Exclusion of certain backgrounds.
• Poorly worded questions.
• Leading questions or political bias.
• Limited answer choices that don't capture all nuances.
• Misrepresentation of polling results.

How Does Polling Work?

• Polling data is used to make inferences about the larger population.
• Random samples are taken from the population of interest.

The Challenge of Sampling

• Ensuring the sample includes diverse people.
• Selection effects occur when the sample isn't representative.
• Example: Researching UH undergraduates’ study habits requires a representative sample.

Literary Digest and Presidential Election Straw Polls

• Straw polls were early types of polls in the early 20th century, used by magazines.
• Literary Digest's straw polls accurately predicted presidential election winners from 1916-1932.

Literary Digest’s 1936 Disaster

• Predicted Alf Landon would defeat FDR (55-41%).
• FDR won by a 61-37% margin.
• What happened?!?!?!

The Aftermath of Literary Digest Problems

• Undercoverage: Used telephone books and vehicle registrations during the Great Depression.
• Low response rate: Only 23% of 10 million people responded.
• Large samples are not necessarily better than small samples.
• Gallup, with only about 5,000 in-person interviews, got it right.
• Quota samples have come under scrutiny.
• Simple random sample (SRS) is now considered the gold standard.

Overview of Sampling Error

• Sampling error (margin of error) is the difference between sample statistics and population statistics.
• Sampling error is inevitable in every poll.

Introduction to Sampling Error

• Even with efforts to avoid selection bias, sampling error still exists.
• Sampling error: Difference between sample statistics and population statistics.
• Inevitable, not a mistake.

Example 1: Presidential Approval Ratings

• Polls ask about approval of presidents (e.g., Trump). Gallup reported a 45% approval rating in February 2025 with a margin of error of ± 4% at the 95% confidence level.

Sampling Error in Polls that Predict Election Outcomes

• Pollsters study “likely voters” instead of the general population.
• Identifying likely voters is challenging.
• Professional pollsters disagree on methods.

So We Have Identified Likely Voters. Now What?

• The population is smaller than the overall U.S. citizens (18+).
• Pollsters must work harder to obtain a good sample.
• Being willing to complete a poll doesn't guarantee they’ll vote.

Interpreting Margins of Error When Predicting Election Outcomes

• Exercise caution when gauging multiple candidates’ support.
• Multiple test statistics, each with its margin of error.
• Separate, larger margin of error for the spread between candidates.

Example 2: The Texas Senate Race

• August 2024 Texas Trends poll by UH Hobby School and Texas Southern Barbara Jordan – Mickey Leland Schools of Public Affairs.
• 1,365 likely Texas voters.
  • Cruz: 46.6%.
  • Allred: 44.5%.
• Margin of error: ± 2.65%.
• Based on this poll, it was hard to tell who would win.
• In the end, Senator Cruz was reelected, gaining 53 percent of the vote to Allred’s 45 percent

Example 3: The National Popular Vote in the Presidential Election

• Fall 2024 NPR/PBS News/Marist Poll for the popular vote in that year’s presidential election
• 1,294 registered voters.
  • Harris: 50%.
  • Trump: 48%.
  • Margin of error: ± 3.7%.
• Based on this poll, it was hard to tell who would win.
• In the end, Trump won 49.9 percent of the national popular vote; Harris won 48.4 percent

The Unique Challenge of Predicting American Presidential Elections

• Besides identifying likely voters, there are other challenges.
• Nationwide polls sample likely voters nationally.
• These polls predict only national results.

For Presidential Elections: Likely Voters Where?

• Electoral College requires state-level polls.
• Nationwide polls don't reflect state-level dynamics.
• Historically, state polls have been difficult and expensive.

We Need State Level Polls

• Due to the Electoral College, state-level polls of likely voters are needed.
• National polls don't accurately represent state-level opinions.
• State polls are historically difficult and expensive to conduct.

The Relationship between Sample Sizes and Sampling Error

• As sample size increases sampling error decreases.
  The relationship between sample sizes and margin of error is shown on the graph.

The Trouble with State Level Polls

• State-level polls require large sample sizes for a 3% margin of error.
• State polls often lack resources despite their importance.
• 2016 state polls were of low quality.
• Interpret election polls cautiously.

Example 4: The Battlegrounds

• Seven battleground states with a combined 93 electoral votes.
• Polling in each state was within the margin of error.
• Neither candidate had a clear advantage in polls.
• In the end, President Trump won all seven states.

Concerns About Question Wording

• Objective of a poll is to gauge public opinion; push polls aim to change it under the guise of a poll.

What is a Push Poll?

• A push poll attempts to change public opinion while pretending to be a legitimate poll.

Message Testing v. Push Polls

• Message testing is used by candidates to research campaign strategies.
• Push polls manipulate public opinion rather than gauge it.
• Example: 2000 South Carolina Republican Primary.

What Can Be Done About Push Polls?

• Unfortunately, despite push polls being considered unethical, the answer is…not a whole lot.
• The American Association of Public Opinion Researchers (AAPOR) prohibits members from conducting push polls.

Issues with Question Wording

• Changes in wording and answer choices can drastically affect findings.
• Examples: "Welfare" vs. "aid to the poor"; death penalty and sentencing options.
• Implications for campaigns: wording can influence responses.

Pre-Survey Information

• General Social Survey question example: Asking about spending on various problems and whether it’s too much, too little, or about right.

General Social Survey, 2006

• "Welfare" vs. "Assistance to the Poor."
• Different responses based on wording.

Capital Punishment (The Death Penalty)

• Historically most Americans have favored the death penalty as a punishment for convicted killers
• Support/opposition is not as easy to gauge as you might think
• Consider carefully the responses Gallup received to two different questions regarding the death penalty

Support for Capital Punishment - Version 1

• Are you in favor of the death penalty for a person convicted of murder?
• % in favor has varied from 90% to under 60% from 1936 to 2019.
• % opposed has varied from 10% to around 40% from 1936 to 2019.

Support for Capital Punishment - Version 2

• If you could choose between the following two approaches, which do you think is the better penalty for murder - [ROTATED: the death penalty (or) life imprisonment, with absolutely no possibility of parole]?
• % death penalty has varied from 56% to around under 50% from 1986 to 2019.
• % life imprisonment has varied from 34% to over 50% from 1986 to 2019.

Reflections on This Example

• How does question wording influence responses?
• How might pollsters use this finding for good or ill?

Tips for Writing Good Questions

• Balanced arguments: "Do you favor or oppose the death penalty?" is better than "Do you favor the death penalty?".
• Avoid double-barreled questions (two questions in one).
  • Example: "If we can now trust the Russians, do you think we should focus our attention on saving endangered species?"

Try to Avoid Negatives!

• Avoid negatives due to confusion.
• Example: Instead of "Do you think McDonald’s should not have been found not liable…" use "Do you think McDonald’s should have been found liable…or not?"

The Importance of Questions Order

• Question order can prime people to answer later questions in a certain way.

The Classic Question Order Effect

• Two versions of questions about Communist reporters: changing the order changes the responses.

Effects of Public Opinion on Public Policy

• To what extent are politicians beholden to public opinion?
• To what extent should politicians be beholden to public opinion?

Assessing the Impact of the Public on Policy Outcomes

• Political scientists struggle with this question.
• Challenges:
  • Congress's true intentions (reelection vs. genuine desire).
  • Inconsistencies in public opinion.
  • The public is operationally liberal yet symbolically conservative.

Studies of Public Mood

• Evidence of responsiveness to public opinion.
• Policy changes receive strong public support.
• Researchers can predict policy changes based on public opinion.
• Policy outputs move in the same direction as public mood (liberal or conservative).

Public Policy Mood

• (Graph from 1952-2020 showing % Liberal Responses).
  Source: James A. Stimson ( http://stimson.web.unc.edu/data/ )

Kidnapping and Murder of Polly Klaas and Three Strikes Laws

• 1993: Polly Klaas kidnapped and murdered.
• 1994: California passed a three strikes law.
• Third strike leads to a mandatory sentence of 25 years to life.

Weakening Three Strikes in California

• Proposition 36 (2012) thrown out with 69 percent of the vote:
  • Criminal conviction has to be violent to constitute Strike 3.
  • If sentenced under three strikes, you could appeal your sentence.
  • Those whose strikes include murder, rape, and/or child molestation may not do so

How Should Politicians Respond to Public Opinion?

• Disagreement among scholars on the extent of correspondence between public opinion and policy.
• Consider two approaches and apply them to crime and punishment.

Delegate Model

• Representative should vote as their constituents would like.
• Public opinion matters greatly.

Trustee Model

• Public opinion is less likely to affect a representative’s votes.
• Edmund Burke: Representatives should use their own judgment, not just follow public opinion.