Exam 2: Research Design/Analysis

Operationalization

translating a concept into its real world counterpart

ex: nightly activites

Mutually Exclusive

must be able to classify every observation into one and only one attribute

Exhaustive

must be able to classify every observation

Measurement Reliability

a measurement technique, applied repeatedly to the same thing, will yield the same result each time (scale example)

Techniques for Measurement Reliability

test-retest method; interrater reliability

Test-Retest Method

making the same measurement more than once

Interrater Reliability

if a researcher suspect that unreliability is generated by research workers, they can compare the measurements from different workers

Measurement Validity

a measure accurately reflects what it is intended to measure

Technique for Measurement Validity

multiple measures

Multiple Measures

• use multiple (different) measures to measure a concept

• include other measures of adolescent violent behaviors

Level of Measurement

nominal; ordinal; metric

Nominal

attributes are mutually exclusive and exhaustive

Ordinal

attributes can be logically rank ordered (in addition to being mutually exclusive and exhaustive)

Metric

attributes are numerical and represent quantities

What are the 3 Components of Classical Experiment?

independent & dependent variable; pretesting & posttesting on the DV; randomized experimental & control groups

Independent and Dependent Variable

examines the influence of an independent (cause) on a dependent variable (effect)

Pretesting and Posttesting on the DV

• subjects are measured on a dependent variable (pretest)

• subjects are then exposed to a stimulus/treatment that represent the independent variable

• subjects are then remeasured on the dependent variable(posttest)

Randomized Experimental and Control Groups

• subjects are randomly divided into 2 groups; one group receives the treatment, other group does not

• randomization allows the researcher to be certain that an changes in the dependent variable are attributable to the independent variable (treatment)

How does randomization reduce threats to internal validity?

if the experimental & control groups are statistically equivalent prior to treatment, then any difference between them after treatment is most likely due to treatment, rather than some pre-existing difference between the groups

Randomization in Criminal Justice

• it produces experimental & control groups that are statistically equivalent before treatment

• this reduces possible sources of systematic bias in assigning subject to groups

Quasi-Experiments

nonequivalent-groups design; interrupted time-series design

Nonequivalent Groups Design

• CJ researchers often cannot randomly assign subjects to experimental & control groups

• when this happens, it is important to make the 2 groups as comparable as possible

Interrupted Time-Series Design

• the independent variable is the intervention, while the dependent variable is measured during all the "observation"

Threats to Internal Validity

history; maturation; testing; instrumentation; statistical regression; selection bias; experimental mortality; causal time order

History

historical events may occur during the course of the experiments and confound the experimental results

Maturation

the subjects of a study naturally change over time in ways that are unrelated to the treatment (IV)

Testing

the process of testing and retesting influences behavior and confounds the results

Instrumentation

changes in the measurement process itself (different interviewer, new survey) may change how subjects score on the DV

Statistical Regression

subjects with extreme scores tend to return tomore normal scores on their own regardless of intervention

Selection Bias

subjects are selected based on some criteria that makes them atypical of the target population which they are supposed to represent

Experimental Mortality

subjects drop out of the study, thus producing inaccurate results of the treatment

Causal Time Order

the IV and DV are measured at the same time, making it difficult to determine which is the cause and which is the effect

Sampling

selecting part of a population

Why do we sample?

we select samples to represent some larger population of people or things

How do I know what to sample?

• the subjects or cases in your study will be consistent with your units of analysis derived from your hypothesis

• if you're looking at the effect of street lighting along city blocks, you would select a sample of city blocks from the larger "population" of city

Probability Sampling - Definition

enables us to make relative few observations and then generalize to a much wider population

Probability Sampling - What does it enable?

• probability that a unit will be selected to be in the sample is known

• make predictions that out sample accurately represents a larger population

• generalize from observed cases (ex: sample) to unobserved cases (ex: population)

Population Parameters

summary description of a given variable in a population

Sample Statistics

summary description of a given variable in a sample

What is the goal of Sampling?

to select a set of elements from a population in such a way that the sample statistics accurately portrays the population parameters

Random Selection

each element has an equal chance of being selected independently of any other event in the selection process

Types of Probability Sampling Designs

multistage cluster sample; stratification

Multistage Cluster Sample

a multistage sample in which natural groups (clusters) are sampled initially, with the members of each selected group being subsampled afterward

Stratification

• group the units of a population into homogeneous groups (or strata) before sampling

• improves the representativeness of a sample, at lease in terms of the stratification variable

Types of Non-Probability Sampling Designs

purposive; available subjects; snowball

Purpose Sampling

• units are selected based on the researcher's judgement about which ones will best suited for the research purpose

Reliance on Available Subjects

• aka convenience sampling

• easy & low cost

Snowball Sampling

• identify a single subject or a small # of subjects and then ask them to identify similar others who might be willing to participate

When is Non-probability Sample most appropriate?

most appropriate when its impossible to determine the probability that any given element will be selected in a sample

3 Sources of Data

asking questions; making direct observations; examining written records

Asking Questions

surveys, individual interviews, focus groups

Making Direct Observations

counting the # of participants at a meeting, administering a drug test, using radar to detect speed of traffic

Examining Written Records

• police records, court records, census data, UCR

• data originally collected by making direct observations or asking people questions

• important to know how the data were collected