Survey Research Overview

Population Variations

Differences in attitudes within a population

Quantitative Research

Emphasizes generalization and statistical inferences

Qualitative Research

Focuses on understanding reasons behind behaviors

Probability Sampling

Selecting cases randomly to represent a population

Nonprobability Sampling

Selecting cases without randomization, focusing on patterns

Survey Quality

Dependent on effective, unbiased survey questions

Close-ended Items

Survey questions with fixed response options

Double Barrel Questions

Asking multiple questions in one, leading to confusion

Demographic Questions

Questions about respondent characteristics, placed at the end

Pretesting Survey

Assessing the survey instrument before data collection

Experimental Group

Receives the experimental stimulus in an experiment

Control Group

Does not receive the experimental stimulus in an experiment

Random Assignment

Assigning participants randomly to experimental conditions

Within Subject Design

Participants experience all conditions in an experiment

True Experimental Design

Includes posttest only, pretest/posttest, and Solomon four-group designs

Laboratory Experiment

Conducted in a controlled environment for internal validity

Field Experiment

Conducted in a natural setting for external validity

Audit Study

Investigates discrimination by keeping all factors constant except the variable of interest

External Validity

Extent to which study findings can be generalized to real-world settings

Internal Validity

Extent to which an experiment's design allows for causal inferences

Informed Consent

Participants' agreement to take part in a study with full understanding

Randomization

Assigning participants to conditions by chance to avoid bias

Deception

Withholding information about the study's true purpose from participants

Association

Relationship between dependent and independent variables

Direction of influence

Impact in a chronological order (independent variable before dependent variable)

Nonspurious

Controlling environment and participant selection to limit external factors

Selection bias

Bias in selecting participants affecting internal validity

Experimental mortality

Participants leaving the experiment, including differential attrition

Historical events

External events influencing participant responses

Maturation

Personal changes in participants during the experiment

Testing

Effect of taking a test on subsequent test results

Instrumentation

Flaws in the instruments used to measure outcomes

Statistical regression to the mean

Responses moving towards the mean when measured multiple times

Experimenter bias

Ignoring evidence against desired outcomes

Empirical generalizations

Drawing conclusions about a population from a representative sample

Statistical inferences

Making predictions about a population based on sample data

Sampling bias

Non-representative sample affecting generalizations

Probability

Chances of an event occurring

Random selection

Each case having an equal chance of being chosen

Sampling distribution

Probability distribution of a sample statistic

Standard error

Standard deviation of a statistic's sampling distribution

Confidence interval

Range around a sample mean indicating level of certainty

Target population

Population for which sample results are generalized

Sampling frame

Basis for sampling, similar to an operational definition

Sampling designs

Methods like simple random, systematic, stratified, or cluster sampling

Simple random sampling

Equal chance for every case to be selected

Systematic sampling

Selecting every kth element from a list

Stratified random sampling

Independent random sampling within variable categories

Cluster sampling

Independent samples drawn from natural groupings

Sampling error

Error in probability sampling due to sample variability

Coverage error

Error due to mismatch between target population and sampling frame

Nonresponse error

Error due to incomplete data collection

Item nonresponse error

Leaving some answers blank while answering most

Convenience and accessibility

Factors guiding selection of cases based on ease of access

Quota sampling

Non-probability sampling with pre-specified characteristics

Purposive sampling

Sampling based on theoretical reasons

Snowball sampling

Chain referral sampling method for hard-to-reach populations

External validity

Generalization of findings beyond the sample

Internal validity

Relates to causal relationships within the study

validity

relates to accuracy

Sampling

Process of selecting cases from a population to study

Debriefing

Explaining the study's purpose and procedures to participants after completion

Recruiting Participants

Process of inviting individuals to take part in a study

Random sampling error

Error from random selection in sampling

Opinion polls

Short-term, non-probability sampling for opinions

Causality

Confidence in a causal relationship between concepts

Generalization

Extending findings to a larger population

Saturation

Point where new data stop yielding insights

Structured interviews

Questioning with fixed response options

Open-ended items

Questions allowing free-form responses

Unit of analysis

Individuals analyzed in the study

Nomothetic

Providing limited understanding compared to qualitative methods

Reactive measurement effects

Respondents altering responses due to awareness of being studied

Social Scientific Surveys

Involving probability samples and testing hypotheses

what are the three types of measurement?

1. observational terms
2. indirect observables
3. constructs

what are teh threee sources fo measuresed operations?

1. self reports
2. observations
3. archival records

reliability

the consistency of measurement

what is the relationship between validity and reliability?

reliable measure may or may not be valid
but an UNreliable measurement can NOT be valid

concept

A general idea or thought about something

observational term

things we can see by looking at them
(type of measurement)

indirect observables

Require relatively more subtle, complex, or indirect observations for things that cannot be observed directly (type of measurement)

construct

abstract concepts that are made up of things that are observable
(type of measurement)

mutually exclusive

Events that cannot occur at the same time

sampling in qualitative research

make conclusions by asking and figuring out why people did what they did

sampling in Quantitative research

takes a while to figure out sampling because it impacts the conclusions they come up with
-the samepl eneds to resprent the majorty of the population

advantages of face to face interviews

-Interviewers can clarify and restate questions, ask for more complete answers
-Response rate tends to be higher than other modes
-Best choice for lengthy interviewees

disadvantage of face to face interviews

-Cost considerably more than other modes
-Interviewer effects/reactivity

Advantages of telephone interviews

-Substantial savings in time and cost in comparison with FTF
-Allows for centralized administration and control

disadvantages of telephone interviews

-Questions must be simpler, with fewer response options
-More nonresponse to questions than FTF
-Decreasingly lower response rates than FTF
-Increasingly difficult to draw random sample

advantages of paper-pencil

-Less expensive
-Samples can be large and geographically dispersed
-Respondents may be more accessible
-Anonymity protects privacy and allows for more sensitive questions

Limitations of paper-pencil

-Response rates tend to be much lower than interview surveys
-No opportunity to clarify questions or probe answers
-More unanswered questions

advantages of computer interviews

-reduced cost
-software creates design and delivery
-requires less time to do
-can increase sample size at low cost
-flexibility in questionnaire design

disadvantages of computer interviews

-response rate is lower
-sampling frames high in coverage error