ARM Exam 2 Terms (Exam Styled)

What is a composite measure? Identify its three types and explain how they differ.

A composite measure combines several questions or indicators into one variable. An index and a scale are unidimensional: they measure one general concept. A typology is multidimensional: it combines two or more concepts into categories. Example: several questions about fear of crime can form one index; parent gender and types of children can form a typology.

Why do researchers use indexes and scales as data reduction devices?

They combine several indicators into one numerical score that shows a respondent’s relative standing on a concept. This can capture a concept more fully than one question and can reduce the influence of bias in any single item. Example: combine worries about burglary, violence, and shootings into an overall crime-worry score.

What is an index? How is it usually constructed? Give an example.

An index combines items measuring one general concept, usually by adding their scores. It has weaker rules for combining items than a scale and usually gives a numerical score. Example: add responses about worry over burglary, violent crime, and mass shootings to measure overall worry about safety. Make sure higher or lower scores mean the same thing across items.

What is a scale, and how does it differ from an index?

A scale combines items measuring one concept but uses their logical relationship or intensity, not just their sum. Its rules for combining items are stronger than an index’s. Example: a Guttman scale of abortion attitudes orders circumstances from easier to harder to support.

What is a Guttman scale? Explain using the abortion example.

A Guttman scale places items in an expected order of intensity. Someone who agrees with a harder-to-accept item would generally be expected to agree with easier items too. In the class example, support for abortion when the woman’s health is endangered is easier to endorse than abortion for any reason. Researchers check if answers follow that pattern.

What is a typology? How does it differ from indexes and scales?

A typology combines two or more dimensions into distinct categories, usually nominal categories instead of a continuous score. Example: classify parents by mother/father and whether they have sons, daughters, both, or no children. Indexes and scales measure one general dimension; typologies combine multiple dimensions.

What is the difference between unidimensional and multidimensional measures?

Unidimensional measures capture one general concept, such as overall fear of crime. Multidimensional measures combine different concepts or dimensions, such as parent gender and children’s gender. Indexes and scales are unidimensional; typologies are multidimensional in the class notes.

Why are experiments especially useful for explanatory research?

Experiments help researchers test causal explanations because the researcher manipulates the independent variable and measures changes in the dependent variable. Comparing a treatment group with a control group helps identify the effect of the intervention. Example: randomly assign some areas to increased foot patrol and compare crime outcomes with control areas.

Describe the classic experimental design and explain how it helps establish causality.

The experimental and control groups both take a pretest and posttest. Only the experimental group receives the intervention. Researchers compare how much each group changes. If the treatment group changes more, the difference may be due to the intervention, assuming groups are comparable and other influences are controlled.

What is the difference between random selection and random assignment? Why does it matter?

Random selection chooses people from a population for the study and helps with generalization. Random assignment places participants into treatment and control groups and helps make groups comparable for causal inference. An experiment can use a convenience sample but still randomly assign participants. The two forms of randomness serve different purposes.

How does matching help create comparable experimental groups?

Researchers identify participants who are similar on important characteristics, such as age or education, and place matched participants into different conditions, ideally using random assignment within pairs. This helps reduce preexisting differences that could otherwise explain the outcome.

What is a double-blind experiment and why is it useful?

In a double-blind experiment, neither participants nor the people administering the intervention know who receives which condition. This helps prevent expectations and experimenter behavior from influencing results. The notes especially connect this to medical research.

Differentiate internal validity from external validity in experiments.

Internal validity asks if the intervention really caused the observed change within the study, rather than another factor. External validity asks if findings apply outside the study to other people or real-world settings. A tightly controlled laboratory experiment may have strong internal validity but weaker external validity if the setting is artificial.

What is the history threat to internal validity? Give an example.

History occurs when an outside event during the experiment affects outcomes. Example: a city launches a public safety campaign while researchers test foot patrols. A crime decrease might reflect the campaign, not just the patrol intervention.

What is the maturation threat to internal validity? Give an example.

Maturation means participants change naturally over time, independently of the treatment. Example: students become more confident as a semester progresses, even without a special confidence-building program.

What is the testing threat to internal validity? What is the Hawthorne effect?

Testing occurs when taking a pretest changes later answers or behavior. The Hawthorne effect refers to people changing behavior because they know they are being observed or studied. Example: people report safer habits after a survey simply because the questions made them think about safety.

What is the instrumentation threat to internal validity?

Instrumentation happens when the way an outcome is measured changes between the pretest and posttest. Example: a researcher uses easy questions before an intervention but harder questions afterward. The difference may come from the instrument, not the treatment.

What is statistical regression, or regression to the mean, as a threat to internal validity?

Extremely high or low scores often move closer to the average on later measurement even without treatment. Example: a school selects students with unusually low test scores for a program; their scores may improve partly because extreme scores tend to become less extreme.

What is selection bias as a threat to internal validity?

Selection bias occurs when groups differ before treatment in ways that influence the outcome. Example: people who volunteer for an exercise program may already be more motivated than those who do not. Random assignment helps reduce this threat.

What is experimental mortality (attrition), and why is it a problem?

Experimental mortality means participants drop out before the study ends. If dropouts differ across groups or are related to outcomes, the remaining groups may no longer be comparable. Example: people who dislike a treatment leave the study more often.

Explain diffusion or imitation, compensation, compensatory rivalry, and demoralization.

Diffusion or imitation: control participants learn about or copy the treatment. Compensation: researchers give extra help to the control group. Compensatory rivalry: control participants try harder because they know they missed the treatment. Demoralization: control participants give up because they feel excluded. Each can distort the treatment-control comparison.

What is the Solomon four-group design and what problem does it help detect?

The Solomon design uses four groups: (1) pretest, treatment, posttest; (2) pretest, no treatment, posttest; (3) treatment and posttest only; (4) posttest only. Comparing groups with and without pretests helps show if the pretest itself changed responses or interacted with treatment.

What is a quasi-experiment, and when would researchers use one?

A quasi-experiment studies an intervention without full experimental control, especially when random assignment is not possible for legal, ethical, or practical reasons. Example: study changes after a state raises the legal drinking age. Because groups may differ before the policy, researchers must consider alternative explanations.

Identify four quasi-experimental designs named in class.

Separate-sample pretest-posttest: compare different samples before and after an intervention. Nonequivalent control group: compare an intervention group with a nonrandomly formed comparison group. Interrupted time series: examine many observations before and after an intervention. Multiple time series: compare trends over time for intervention and comparison groups.

What is an interrupted time-series design? Give an example.

Researchers measure an outcome repeatedly before and after an intervention and examine changes in the pattern. Example: track monthly traffic deaths for several years before and after the drinking age rises to 21. A change at the intervention point may suggest an effect, but other events must be considered.

How does a multiple time-series design improve on a single interrupted time series?

It tracks outcomes over time for both an intervention group and a comparison group. This helps researchers distinguish a policy effect from broader changes affecting everyone. Example: compare monthly traffic deaths in a state that changed its drinking-age law with a state that did not.

What are statistical controls, and why are they important?

Statistical controls let researchers examine the relationship between one independent variable and an outcome while holding other variables constant. They help assess non-spurious relationships and avoid leaving out important explanations. Example: compare party identification and deportation support separately among liberals/moderates and conservatives.

Why can a bivariate relationship be misleading? Explain with a control variable.

A bivariate relationship compares only X and Y, but another variable may explain some of the pattern. For example, party identification and deportation support may be related partly because party groups differ in political ideology. Controlling for ideology helps show if party differences remain within ideology groups.

What are surveys best suited to study, and what are their limitations?

Surveys are useful for describing populations and testing hypotheses about attitudes, beliefs, values, and self-reported behavior. They can support population estimates with appropriate samples. They are less suited to understanding how people construct meanings in depth or directly measuring events such as arrests or media content. The individual is usually the unit of analysis.

What is the difference between primary and secondary survey analysis?

Primary analysis means researchers collect and analyze their own data. Secondary analysis means they analyze data collected by others. Example: designing your own immigration survey is primary analysis; using the General Social Survey to study deportation attitudes is secondary analysis.

Define response rate and self-selection. Why can they threaten survey findings?

Response rate is the percentage of the selected sample that actually participates. Self-selection occurs when people who choose to respond differ from those who do not, such as people with strong opinions being more likely to answer. This can bias estimates and weaken generalization. Researchers can explain the study, follow up, and offer reasonable, noncoercive incentives.

What do SUG and FRUG mean, and why do they matter?

SUG means selling under the guise of research; FRUG means fundraising under the guise of research. These activities pretend to be surveys but aim to sell or raise money. They can make people suspicious of legitimate surveys and reduce participation and response rates.

Compare self-administered, telephone, and in-person surveys.

Self-administered surveys (mail, web, CASI) can be inexpensive and allow privacy, but may have low response rates or access problems. Telephone surveys can be quick and build some rapport but face interviewer effects and limited length. In-person interviews allow rapport and flexible help, often with higher response rates, but cost more and still risk interviewer effects. Choose based on budget, time, sample, and sensitivity.

What is CASI, and why might it be useful for sensitive questions?

CASI means computer-assisted self-interviewing. Respondents enter their own answers on a computer, often with an assistant available for technical help. It can give people more privacy when answering sensitive questions, though computer familiarity and access can be problems.

What are CATI and CAPI?

CATI means computer-assisted telephone interviewing: the interviewer asks questions by phone and enters answers into a computer, often with built-in skip patterns. CAPI means computer-assisted personal interviewing: an in-person interviewer enters answers into a computer.

What is random digit dialing (RDD)?

Random digit dialing is a telephone sampling method in which telephone numbers are generated for potential selection instead of relying only on an existing contact list. The notes describe it as a common approach in telephone surveys. Coverage and nonresponse remain concerns.

What are interviewer effects? Explain how they can harm reliability and validity.

Interviewer effects happen when an interviewer’s words, reactions, or characteristics influence answers. Example: an interviewer gasps when someone reports having had an abortion; the person may change later answers. This can make responses less accurate and less consistent. Training, standard instructions, monitoring, and nonjudgmental behavior help reduce the problem.

What is social desirability bias, and how can researchers reduce it?

Social desirability bias occurs when respondents give answers that make them look good instead of answering truthfully. Example: someone underreports drug use because a parent is nearby. Researchers can use private response methods, neutral wording, nonjudgmental interviewers, and careful question placement.

What factors should researchers consider when choosing a survey mode?

Consider the study’s purpose, response rates, cost, sampling frame, characteristics of the population, available time and facilities, and sensitivity of questions. Example: a low-cost web survey is quick but misses people without internet access; an in-person survey costs more but can build rapport.

Why are questionnaire design and pretesting important for valid survey results?

Survey answers depend on the questions asked, their wording, and the response choices. Poor questions can confuse respondents, push them toward answers, or leave out meaningful options. Pretesting checks comprehension, clarity, discomfort, length, order, and response categories so researchers can revise before collecting final data.

What is a leading or loaded survey question? Give an example and fix it.

A leading or loaded question uses wording that pushes respondents toward an answer. Example: “Should we stop the terrible policy that hurts families?” A more neutral question is “Do you support or oppose this policy?” Neutral wording reduces bias.

What is a double-negative survey question, and why should it be avoided?

A double-negative question contains multiple negatives that make the meaning hard to follow. Example: “Do you disagree that schools should not ban phones?” A clearer version is “Should schools ban phones?” Confusing wording can reduce accuracy.

What are acquiescence bias and forced-choice questions?

Acquiescence bias is the tendency to agree with statements regardless of actual beliefs. A forced-choice question presents alternatives directly instead of asking for agreement with only one statement. Example: ask whether military strength or diplomacy is better for peace, instead of asking respondents to agree that military strength is best.

What are context effects and priming in surveys?

Context effects happen when earlier questions influence later answers. Priming makes certain issues more noticeable in respondents’ minds. Example: asking many questions about health care before asking for presidential approval may change how people rate the president. A common approach is to ask general questions before specific ones.

What are response sets, and why might researchers reverse question polarity?

Response sets are patterns of answering that do not fully reflect the question’s content, such as repeatedly selecting “agree.” Reversing the wording direction of some items can help detect automatic agreement, although reversed questions must remain clear.

How should researchers ask sensitive survey questions?

Use neutral, nonthreatening language; provide face-saving opportunities; make behaviors sound possible without judging them; consider lettered rather than numbered choices; and place sensitive questions after rapport has developed, often in the middle or near the end. Example: ask about drug use privately and without showing surprise.

What is a survey wishbook, and what are the steps in designing questions?

A wishbook is an initial collection of possible survey questions developed through brainstorming, theory and literature, experts, colleagues, and people in the target population. Researchers decide what information is needed, draft questions, consider existing items, organize the survey, pretest, revise, and repeat as needed.

What exactly should be checked during questionnaire pretesting?

Check each question for comprehension, clarity, appropriateness, response choices, discomfort, and influence on answers. Check the whole survey for length, organization, and response sets. Pretesting is repeated: test, revise, test again, and revise again.

What is evaluation research, and how does it differ from basic social research?

Evaluation research assesses programs or policies, often for sponsors. Basic research seeks theoretical understanding and emphasizes internal validity; evaluation research emphasizes practical outcomes and external validity.

What are the six stages of policymaking?

Identify the problem; plan policy options; develop interventions; implement interventions; evaluate implemented options; decide their future. Evaluation research is typically associated with stage five.

What is a needs assessment?

During policy planning, determine priorities of the affected population, the scope of the problem, and which programs could address it and for whom.

What is an impact assessment?

Assessment of the potential impact of a proposed intervention or program during policy planning.

What is formative evaluation?

Pretesting an intervention before implementation to see how its components work and whether they are consistent with program objectives.

What is program monitoring (process evaluation)?

During implementation, assess program coverage, whether delivery occurs as intended, and resources spent.

What is the difference between program coverage and program delivery?

Coverage asks whether the program reaches the target population; delivery asks whether it is implemented as intended.

What is the difference between effect assessment and efficacy assessment?

Effect assessment asks whether the program produced intended results; efficacy assessment evaluates benefits in relation to costs.

What is the difference between cost-benefit and cost-effectiveness analysis?

Cost-benefit assesses monetary benefits relative to costs; cost-effectiveness assesses substantive program impact relative to costs.

Why is external validity especially important in evaluation research?

Researchers want to generalize intervention results to other settings and populations, but sampling and social context can limit generalizability.

How does theory help evaluation research? What is the treatment black box?

Theory explains why policies might work and provides insight into processes within the treatment black box that connect intervention to outcomes.

How can stakeholders and external parties influence evaluation research?

Funders, staff, policymakers, and interest groups may have competing goals and influence implementation, reporting, or use of findings.

What is systematic social observation (SSO)?

Collection and coding of observational data using standardized procedures, often with random samples of places or times, to quantify social interactions.

How are observations coded in SSO?

Researchers apply a standardized coding guide; units of analysis can be interactions or neighborhoods. Google Street View can supply observational data.

What is meta-analysis?

A statistical synthesis of findings from multiple studies addressing related questions. The lecture lists this approach but does not define it.

What is content analysis?

Systematic, objective quantitative analysis of existing messages by coding and categorizing words or other features.

What are the main steps in quantitative content analysis?

Identify document population; determine units of analysis; sample documents; design coding procedures; test and refine coding; analyze occurrences.

What is inter-rater reliability in content analysis?

Consistency across coders applying codes to the same content. Use shared example texts, training, explicit instructions, and resolution of disagreements.

What are spatial data and Geographic Information Systems (GIS)?

Spatial data have geographic identifiers. GIS links cases to locations such as census tracts to study relationships between area characteristics and outcomes.

What are spatial effects and hot spot analysis?

Spatial effects include spillover across geographic areas. Hot spot analysis identifies concentrations of events such as crime.

What is mixed-methods research?

Intentional use of both quantitative and qualitative approaches to answer research questions and take advantage of the strengths of each.

What is a convergent parallel mixed-methods design?

Quantitative and qualitative approaches occur at the same time; their findings are integrated and interpreted together.

What is an exploratory sequential mixed-methods design?

Qualitative research comes first and is followed by quantitative research, such as interviews informing a survey.

What are embedded, multiphase, and transformative mixed-methods designs?

Embedded adds a secondary method to a primary one; multiphase links several quantitative and qualitative phases; transformative uses a social-justice-focused theoretical perspective.

What is big data, and how is it used in social research?

Very large computer-readable datasets used to reveal patterns, trends, and associations, such as social media activity, search trends, and crime prediction.