Measurment and Sampling

Measurement

Describing abstract concepts in terms of specific indicators by assigning numbers or other symbols to them according to rules.

Indicator

An observation assumed to be evidence of the attributes or properties of some phenomenon.

Item

A single indicator of a variable (e.g., one question).

Index

A composite measure made by combining multiple items into one score.

Scale

A composite of multiple items forming one score on a variable; may reflect multiple dimensions or the degree to which a variable is present.

Level of measurement

The rules defining which mathematical operations can be performed on the numbers a measure produces (nominal, ordinal, interval, ratio).

Nominal measures

Observations classified into mutually exclusive and exhaustive categories; numbers are only labels (e.g., religious affiliation).

Ordinal measures

Mutually exclusive, exhaustive categories with an inherent fixed order but unknown distances between ranks (e.g., socioeconomic status).

Interval measures

Ordered categories with equal spacing between them but no true zero (e.g., temperature in degrees F).

Ratio measures

Interval measures that also have a meaningful zero point (e.g., annual income, prison sentence length).

Continuous variables

Variables that can take on an infinite number of values (e.g., age, income).

Discrete variables

Variables with a finite number of distinct, separate categories (e.g., household size, number of arrests).

Validity

The accuracy of a measure: whether it measures the variable it is intended to measure.

Face validity

Whether a measure appears, on its face, to measure what it claims to.

Content validity

Whether a measure covers the full range of meanings or dimensions of the concept.

Jury opinion

An extension of face validity in which several experts judge whether a measure is valid.

Reliability

A measure's ability to yield consistent results each time it is applied.

Random errors

Measurement errors that are neither consistent nor patterned; chance errors that tend to cancel out in the long run.

Systematic errors

Measurement errors that are consistent and patterned, so they do not cancel out and bias results.

Nominal definition (bonus)

The conceptual meaning of a concept, at the abstract theoretical level.

Operational definition (bonus)

The specific procedures or observations used to measure a concept at the concrete research level.

Stability (bonus)

A reliability property: the measure should not change from one application to the next.

Equivalence (bonus)

A reliability property: all items of an instrument should measure the same thing.

Double translation (bonus)

Translating an instrument into another language and back again to check that meaning is preserved.

Population

All possible cases of what we are interested in studying, defined by content, units, extent and time.

Sample

One or more elements or cases selected from some larger grouping or population.

Representative samples

Samples that accurately reflect the distribution of relevant variables in the target population.

Sampling frame

A listing of all the elements in a population, from which the sample is drawn.

Randomness

Selection by chance alone, so each element's probability of inclusion is known and not influenced by the researcher.

Probability samples

Samples in which every element of the population has a nonzero, known chance of inclusion.

Simple random sampling

A probability sample in which each element has an equal probability of inclusion.

Systematic sampling

A variation on simple random sampling that takes every kth element on a sampling frame.

Stratified sampling

Dividing the population into subgroups (strata), then drawing separate random or systematic samples from each.

Area sampling

Also called cluster or multistage sampling: randomly selecting areas, then smaller areas within them, then units from the final areas.

Nonprobability samples

Samples in which the investigator does not know each element's probability of inclusion.

Availability/Convenience sampling

Also called accidental sampling: selecting whoever is readily available.

Snowball sampling

Also called respondent-driven or interactive sampling: starting with a few 'seeds' who each refer others with specified characteristics.

Quota sampling

Setting a number of cases to fill from each category of the population, then filling those quotas nonrandomly.

Purposive sampling

Also called judgmental sampling: the researcher uses judgment to select cases that best serve the study's purpose.

Dimensional sampling

Specifying important dimensions and choosing at least one case for every possible combination, to make small samples more representative.

Sampling error (bonus)

An estimate of how much a sample's values differ from the population's values.

Sampling distribution (bonus)

A distribution of sample statistics from many samples of the same population.

Proportionate stratified sampling (bonus)

Each stratum's sample size matches its share of the population; aims to reduce sampling error.

Disproportionate stratified sampling (bonus)

Enough cases drawn from each stratum to allow statistical comparisons between strata.

Sampling fraction (bonus)

The number of elements in the sample relative to the number in the population.

Random-digit dialing (bonus)

Generating phone numbers at random so unlisted numbers can be reached.

Survey

A data-collection instrument or effort in which a sample of people respond to questions or statements, by questionnaire or interview.

Survey research

A data-collection technique in which information is gathered from respondents by having them respond to questions or statements, usually with large samples.

Questionnaire

A survey form containing written questions that people respond to directly.

Interview

A survey in which an interviewer reads questions to respondents and records their answers.

Interview schedule

The document an interviewer uses: specific questions in a fixed order, with instructions and transition phrases.

Closed-ended questions

Questions with a fixed set of response alternatives, all determined in advance; easy to handle as data.

Open-ended questions

Questions respondents answer in their own words; responses are unpredictable and complex to handle.

Filter questions

Questions whose answer determines which question the respondent goes to next.

Contingency questions

Questions answered only by respondents who gave a particular answer to a filter question.

Cover letter

A letter introducing the project to sample members (sponsor, purpose, selection, confidentiality, deadline, etc.) to encourage participation.

Response rate

The proportion of a sample that completes the questionnaire or interview.

Probes

Follow-up questions in an interview meant to elicit clearer and more complete responses.

Focus group

A group depth interview: a whole group interviewed at once, led by moderators following an outline of topics.

Computer-assisted interviewing

Interviewing in which questions appear on a computer that routes skip patterns and records answers directly as they are given.

Matrix question (bonus)

A format presenting several items with the same response options in a table.

Nonresponse bias (bonus)

Distortion that occurs when nonrespondents differ in important ways from those who respond.

Available data

Observations collected by someone other than the investigator, for purposes different from the researcher's.

Secondary analysis

Analyzing data originally collected by other researchers for a different study.

Data archives

Organizations or repositories that store datasets and make them available to other researchers.

Missing data

Information that is absent or incomplete for some cases or variables in a dataset.

Content analysis

Transforming the symbolic content of a document from a qualitative, unsystematic form into a quantitative, systematic form.

Coding

Assigning units of content to the categories of a coding scheme; categories must be exhaustive, mutually exclusive and theoretically relevant.

Manifest coding

Coding the more objective, surface content of a document or medium (e.g., counting characters).

Latent coding

Coding by inference from what is in a document to a more abstract category (e.g., how central a character is).

Statistical data (bonus)

Quantified observations of some element of human behavior.

Document (bonus)

Any recorded, nonquantitative form of communication: a book, letter, audio or video recording, etc.

Nonreactive (bonus)

Research in which subjects are not influenced by being studied; a key advantage of available data.

Scale

A number of items that combine to form a composite score on a variable.

Likert scale

A series of statements, each followed by response alternatives; the numbers for each response are totaled for an overall score.

Summated rating scale

A scale whose score is the sum of responses to its items; the Likert scale is the classic example.

Thurstone scale

A scale built with equal-appearing intervals: judges rate items from 1 (most unfavorable) to 11 (most favorable), and respondents pick the items they agree with.

Semantic differential

A format in which respondents rate a stimulus (person, event) on scales between pairs of polar-opposite adjectives.

Guttman scale

Items listed progressively by intensity, so only one pattern of responses produces any given score.

Response bias

The tendency for answers to be influenced by things other than people's true feelings, beliefs and behaviors.

Response set

Answering a series of items the same way (e.g., always 'agree') regardless of their content.

Response pattern anxiety

Changing answers to avoid what feels like a repetitive response pattern.

Social desirability effect

Giving answers that make oneself look good rather than answers that reflect one's true views or behavior.

Reproducibility (bonus)

In a Guttman scale, how well a person's composite score predicts which items they agreed and disagreed with.

Unidimensionality (bonus)

All items in a scale measure one variable.