Media, public opinion & survey methodology (6))

COMM111

Definition of Survey

• A statistical analysis of answers to a poll of a sample of a population, for example, to determine opinions, preferences, or knowledge

• A method of gathering information from a sample of individuals

• A powerful, scientific tool for gathering accurate and useful information

Surveys combine:

• The ancient but extremely effective obtaining of information from people by asking questions;

• And the modern ransom sampling procedures that allow a relatively small number of people to represent a much larger population (Schuman & Presser, 1996:1)

BUT

• Surveys are useless unless you have a clear grasp on what you want to understand, and from whom you want to extract information (Halperin & Heath 2012:231)

Examples of surveys

Opinion Polling - in politics

• Predicting who will win an election, measuring party support, policy support/issue polls

Market Research

• Brand recognition, purchasing preferences & trends, which media people like and why

Academic Research

• Descriptive Surveys: Gather information on what people do and think

• Analytic Surveys: Answer RQs or test hypotheses

Advantages of Surveys

1. Allow the measurement of things we don’t observe naturally (ex. attitude/behaviour/preference)

2. Can be used to collect data for a population that is too large to observe directly

3. They are convenient

4. They can reach vulnerable population groups

5. They can detect small relationships that are not usually observed

6. They are (relatively) cheap

Disadvantages of Surveys

1. Multiple points of error within the collection & design of surveys

2. Creating surveys is not an “exact science” but instead a series of best practices (Howard & Pressor, 1996)

3. Deliberate or accidental mis-representation of results

4. Challenges to survey reputation? Example: Data collection exercises by political parties

Descriptive Surveys - Types of Surveys

Used to gather information largely on what people do and think. Thus a researcher might use this type of survey to find out what young people think about drugs, what drugs they might use, and with what frequency

Analytic Surveys - Types of Surveys

Used to answer research questions or to test hypothesis. A researcher might collect data from the general population which detailed information on health habits ex. diet, exercise, smoking, and so on. This information might then be used to make predictions concerning the state of health of the population at some future date. For example, it might be possible to predict the amount of heart disease in a population ten years hence, based on this sort of data

Cross-sectional surveys - Types of Surveys

Collect information from the sample at the same point time

(Time and population survey)

Advantage

• All information collected at once

Longitudinal survey - Types of surveys

Information is collected at different points in time in order to study changes in time

• Trend study

• Cohort study

• Panel study

(Time and population)

Types of Longitudinal Surveys

• Trend Study

• Cohort Study

• Panel Study

Trend Study - Type of Longitudinal Survey

• Members of the population can change, sample many times over different points of time to see if any trends appear

• Example:

  • A researcher might be interested in the attitudes of high school principals toward the use of flexible scheduling

    • Population: head teachers in Merseyside

    • Instrument: Questionnaire

    • Number of samples: 3 in a school year

Advantage

• Members of the population can change over time without affecting the result

Disadvantage

• Not for analysing particular individuals but groups

Cohort Study - Types of Longitudinal Surveeys

Members of the population do not change, but maybe members surveyed change

Example:

• The media head department of a school wants to know the importance of teaching methodologies and languages

  • Population: History students graduated from 1990 to 1999

  • Instrument: Questionnaire

  • Number of samples: 2 in a year

Advantage

• It is possible the selection of different sample

Disadvantage

• Members of population cannot change

Panel Study - Types of Longitudinal Surveys

Always survey the same sample

Advantage

• Same sample always

Disadvantage

• Loss of individuals could happen

Kinds of Question Structure

• Open-ended

• Close-ended with ordered choices

• Close-ended with unordered response choices

• Partially closed-ended

Nominal - Question Structure

Variables which have two or more categories

Ex: Homes: Detached, Semi-detached, Flats, Bungalows…

Dichotomous - Question Structure

Only two categories

Ex: Yes/No

Ordinal - Question Structure

Ranked variables/categories

Ex: Education, class, or agreement scales (strongly like, like, neutral, dislike, strongly dislike)

Interval - Question Structure

Numerical values that can be measured on a continuum

Example: Temperature

• Continuous: Assume the number can go be theoretically infinite with a huge number of fractions of the interval (height)

• Discrete: Assumes a finite number with a finite number of possible values (marks given to an essay)

Self-administered Questionnaires

Standard set of questions on a sheet of paper, app, or website respondents fill themselves

Group-administered Questionnaires

Groups of respondents are brought together to respond to the questionnaire

Interview Survey

Face-to-Face, or over the phone. Interviewee works with respondents through the questions

Steps in survey research

1. Define the problem

2. Identify the population

3. Choose how you will collect the data (telephone? engagement with relevant communities? external polling company?)

4. Design, construct, pilot, and refine

5. Select your representative sample

6. Administer the survey

7. Analyse, interpret, and communicate results

Sources of Error (Halpering & Health 2012:232)

Sample

A small piece of something that you hope represents the whole

Requirements for Accurate Estimates

• The sample is large enough to yield the desired level of precision

• Everyone in the population has an equal (or known) chance of being selected for the sample

• Questions are asked in ways that enable the people in the sample to respond willingly and accurately

• The characteristics of people selected in the sampling process but who do not participate in the survey are similar to the characteristic of those who do

Questions should be

• Clear and understandable

• Be single-barreled

• Be specific and not vague

• Avoid leading questions

  • tendency to say yes to yes/no questions

• Consider the order of questions

  • when ranking, candidates with names that alphabetically comes higher up have an advantage over others

Responses - How to create good/bad surveys

• Knowledge: Questions which assume knowledge on behalf of the surveyed

  • Respondents might guess

  • Respondents might look for hints in the question to help them make sense of it (path of least resistance)

  • Respondents are terrible at predicting (would X influence your vote?)

• Respondents sometimes lie

  • Social Desirability Bias

  • Bradley Effect