Types of Bias in Surveys and Experimental Design

Response Bias

Response bias occurs when the behavior of either the interviewer or the respondent causes inaccurate or untruthful responses. Factors such as body language or the context in which the survey is conducted may influence answers.

Example of Response Bias

In a survey about alcohol consumption among high school students, if the survey is conducted in the presence of the students' parents, the students are likely to deny drinking alcohol, even if they had. This results in response bias.

Solution to Response Bias

Response bias can be reduced through thorough training of interviewers and careful supervision during the interview process.

Nonresponse Bias

Nonresponse bias occurs when selected participants cannot be reached for the interview or refuse to answer. This creates an unrepresentative sample.

Example of Nonresponse Bias

In informal surveys like radio call-ins or online polls, only people with strong opinions are likely to respond, making the results unreliable and unrepresentative of the general population.

Impact of Nonresponse Bias

Nonresponse bias makes it difficult to determine if the opinions of those who responded accurately reflect the beliefs of the entire population.

Undercoverage Bias

Undercoverage bias occurs when certain groups in the population are not adequately represented in the sample.

Example of Undercoverage Bias

Telephone surveys often have undercoverage bias because certain groups, such as those without telephones or those in rural areas, are left out. In the U.S., only a small percentage of households lack telephones, but in some African countries, less than 4% of households own a telephone, making telephone surveys biased.

Impact of Undercoverage Bias

Undercoverage bias leads to inaccurate results that do not represent the entire population.

Wording Effect Bias

Wording effect bias arises when survey questions are confusing or lead respondents toward a particular answer.

Example of Wording Effect Bias

A Pew Research Center survey used two questions about military action in Iraq, one of which included a mention of potential casualties. This change in wording caused responses to vary dramatically: 68% favored military action without mentioning casualties, while only 43% favored it when casualties were mentioned.

Impact of Wording Effect Bias

The phrasing of questions can heavily influence how respondents answer, making the results biased.

Parameters

The actual proportion of a population with a certain characteristic (e.g., the proportion of U.S. women who have extramarital affairs).

Statistics

The fraction of a sample with a certain characteristic, used to estimate the population parameter.

Experiments

Planned procedures designed to investigate cause-and-effect relationships between variables. The independent variable is manipulated, and the effect on the dependent variable is measured.

Observational Studies

Involve observing and measuring variables without manipulating them.

Explanatory (Independent) Variable

The variable that is manipulated to observe its effect on the dependent variable (e.g., brand of formula in a study on weight gain).

Response (Dependent) Variable

The outcome being measured in the experiment (e.g., weight gain in babies).

Factors

Variables whose effects on the response variable are of interest. Factors can be qualitative (e.g., sex of the baby) or quantitative (e.g., weight or age).

Lurking Variables

Lurking variables are unmeasured variables that might influence the results of the study.

Example of Lurking Variable

In a study on formula consumption and weight gain in babies, a lurking variable could be the height of the parents, which may affect the baby's weight gain but is not measured in the study.

Confounding Variables

Confounding variables are factors that cannot be separated from the explanatory variable and may distort the results.

Example of Confounding Variable

In a study on breastfeeding versus formula feeding, breastfeeding could be a confounding variable because it is impossible to separate its effects from the effects of formula feeding.

Control Group

A control group is a group of experimental units that does not receive the treatment but is measured for comparison.

Example of Control Group

In a study on baby formula, a control group would be babies who do not receive formula, to compare their weight gain against those who do.

Placebo Group

A placebo group receives a 'placebo,' which resembles the actual treatment but contains no active ingredients.

Placebo Effect

This group is used to measure the placebo effect.

Blinding

Blinding refers to concealing the identity of the treatment from the subjects (single-blind) or both the subjects and the experimenters (double-blind).

Blinding Example

In a bottled water preference study, subjects would not know which brand they are drinking (single-blind), and the experimenters would also be unaware of which brand they are giving to each participant (double-blind).

Purpose of Blinding

Blinding helps reduce bias in the measurement and reporting of results.

Randomization

Randomization involves assigning subjects to treatments randomly to ensure that the groups are similar and that the results are not skewed by external factors.

Randomization Methods

Each subject receives one treatment, randomly assigned. Each subject receives all treatments in a random order.

Blocking

Blocking is the process of grouping similar experimental units to control for known variables that might affect the results.

Blocking Example

If a researcher believes men and women respond differently to a treatment, they can block the experiment by sex, ensuring that both groups are studied separately.

Replication

Replication involves repeating treatments to reduce the effect of random variation and increase the reliability of the results.

Replication Example

Administering a medication to multiple patients ensures that the results are not due to chance.