Sampling Methods and Bias in Statistics

A Census involves the collection of data from every member of a population.
A representative sample is a sample where the relevant characteristics of the members are generally the same as the characteristics of the population.
In a study determining the mean height of all students, a statistics class is more likely to be a representative sample than the men's basketball team.

A statistical study suffers from bias if its design or conduct tends to favor certain results. Conclusions from biased studies are not considered trustworthy.
Bias causes include:
- The researcher having a personal stake in a particular outcome.
- Results being reported in a misleading way.
- Data being collected intentionally or unintentionally in a way that makes it unrepresentative.
- The sample differing in a specific way from the general population.
An estimate by Nielsen Media Research of late-night show viewership would likely be biased if it only surveyed households where the wage earner worked the late-night shift, as those workers are unable to watch television at that time.

A Random Sample is one in which every member of the population has an equal chance of being selected.
A Simple Random Sample (SRS) is defined as a sample of a particular size where every possible sample of that size has an equal chance of being selected.
Common techniques for generating an SRS include the Lottery method (drawing from a hat) and the Random number method (using software generators).
Selecting names from local property tax records to poll all town residents is not an SRS of the town population because it limits the sample to homeowners only.

Systematic Sampling involves choosing every $k^{th}$ member of a population, such as every $50^{th}$ member.
The National Air and Space Museum used systematic sampling to test ideas for a new solar system exhibit by interviewing a visitor exactly every $15\,minutes$ .
Systematic sampling may fail if there is a pattern in the population; for example, choosing every $10^{th}$ room in a dormitory where males are assigned to odd-numbered rooms and females to even-numbered rooms could result in a non-representative sample.
Convenience Sampling uses results that are readily available rather than sophisticated procedures. This often leads to non-representative samples, such as a supermarket salsa taste test where only people who like salsa or shop at specific times participate.

Cluster Sampling involves dividing the population into groups (clusters), randomly selecting some of those clusters, and choosing all members within those selected clusters.
An example of cluster sampling is checking the gas price at every station within a mile of rental car locations at a few randomly selected airports.
Stratified Sampling involves partitioning the population into at least two strata (groups) and drawing a sample from each group.
The U.S. Labor Department uses stratified sampling for unemployment reports by grouping cities and counties into about $2000$ geographic areas and then randomly selecting households within each area.

A successful study requires a sample that is representative of the population.
Biased samples are unlikely to be representative.
Even a well-chosen method may be unrepresentative due to bad luck in the selection process.
Samples taken by different researchers using the same method on the same population will typically result in different statistics.