SOC 200 Research Methods Exam 2

Final exam (non-cumulative) for SOC 200

Purpose (experiments)

Experiments involve → taking action and observing consequences of that action

• The purpose of experimental research is to discover causal relationships between variables

Goal - determine whether changes in an independent variable produces changes in a dependent variable

• E.g. Does studying for a test lead to better test performance?

Components (experiments)

1. IV & DV

2. Pre-testing and post-testing

3. Experimental and control groups

Independent variable - takes the form of a stimulus (present or absent) → cause

Dependent variable - effect

• IV is the suspect. DV is the victim. (or criminal act)

Pretest

the measurement of a dependent variable among subjects [before doing it]

Post-test

the measurement of a dependent variable among subjects after they have been exposed to an independent variable

Experimental Group

a group of subjects to whom an experimental stimulus is administered to

Control Group

a group of subjects to whom no experimental stimulus is administered and who should resemble the experimental group in all other respects

• [subjects in control should be of same approximate demographic as experimental group] + [control also takes pretest & posttest]

Basic Experimental Design

T1 (before)		T2 (after)
R O1	X	O1’ Experimental
R O2		O2’ Control

R = randomization/Random Assignment (make sure that the two groups are homogenous)

O = Pretest (performance as T1)

O’ = Post Test (Performance at T2)

X = Treatment

Double-Blind Experiment

An experimental design where neither the subjects nor the experimenters know which is the experimental and which is the control

Causation

There are 3 requirements necessary for establishing causal relationship between IV and DV

• IV must precede DV

• IV and DV must be shown to covary (go together/relationship)

• Changes observed in DV must be result of changes in the IV (and not some other unknown variable)

Control

the central characteristic of an experiment

• when a researcher tries systematically to rule out variables that are possible causes of the effects she is studying other than the variable that she has hypothesized to be causes

• The Degree of Control is contingent upon…

  • The manipulation of the IV, the creation of equivalent experimental groups, and the control of extraneous variables influences

    • These 3 issues determine whether full, quasi, or pre-experimental designs are used

Pre-experimental Designs

Indicates that they do not meet scientific standards of experimental designs, and frequently used because conditions for true experiments difficult to meet. Often loose in structure, could be biased

• One shot case study/one-group posttest-only design

• One-group pretest-posttest design

• Post-test-only nonequivalent groups design/static group comparison

Quasi-Experimental Designs

designs which contains some of the controls and manipulations of experimental design, but lacks random assignment of subjects

• single-group interrupted time series design/time series design

• Pre-test-post-test quasi-equivalent groups design

• Interrupted time series quasi-equivalent groups/multiple time series design

True Experimental Designs

greater control and refinement, greater control of validity

• pretest-posttest Equivalent groups design

• Post-test only equivalent groups design

• Solomon Four-Group design

uses random assignment to protect against sources of invalidity

One-shot case study

a single group of subjects is measured on a dependent variable following an experimental stimulus

• A training program is implemented and participants are given a posttest at the conclusion of training (X → O)

One-group pretest posttest

one group, not randomly selected nor randomly assigned, is given a pretest, followed by a treatment/intervention, and finally a posttest. There is no comparison group. Generally done with intact groups.

• A classroom teacher gives her students a pretest then implements an instructional strategy followed by a posttest (O1 → X → O2)

Static Group Comparison

one group which has experienced a treatment/intervention (X) is compared to another group that has not had the intervention. The groups are not randomly selected and are generally pre-existing groups. There is no pre-observation/pretest.

• Comparison of GRE scores for students who attend a rural high school versus those who attended an urban high school

Non-equivalent control group

randomization is absent (O1 → X → O1’ AND O2 → O2’).

Interrupted Times Series Design

This design uses several waves of observation before and after the introduction of the independent (treatment) Variable of X

• repeated measures on the same group over time (O1 O2 O3 O4 O5 → X → O5 O6 O7 O8) with no control or comparison group

  • A study designed to test whether the implementation of a crackdown on speeding in a given state reduces the traffic fatality rate in that state

Interrupted Time Series Design With Comparison

the addition of a second time series for a comparison groups helps to provide a check on some of the threats to validity of the single interrupted Time Series Design, especially history

• The design uses several waves of observations in both groups (treatment and comparison) before and after the introduction of the independent variable X in the treatment group

  • A study to assess the effect of a crackdown on drunk driving on automobile fatalities in one state, compared to automobile fatalities in another state without a similar crackdown

Solomon Four-Groups Design

R O1	X	O1’
R O2		O2’
R	X	O3’
R		O4’

Internal Validity

the extent to which extraneous variables have been controlled (changed in the outcome might not be because of the experiment)

• Sources → history, maturation, testing, instrumentation, statistical regression, selection bias, mortality, causal time order, contamination, etc.

External Validity

the possibility that conclusions drawn from experimental results may not be generalizable to the ”real” world

• [low external validity in experiments, high internal validity though → cannot achieve both]

History

impact of specific historical events in the environment

• E.g. Effect of curriculum change on students’ performance

Maturation

biological and cognitive change of the subjects

• e.g. subjects become tired/bored

• e.g. children become mature within a reasonable time span

Sensitizing

effect of pretest

• E.g. subjects will know what the researchers are trying to test

• E.g. subject become wise on the test

Instrumentation

(instrument decay) - process of measuring becomes unreliable

• E.g. Researchers’ experience will improve the testing procedures and consequent results

Statistical Regression

second scores (after the treatment) are likely to regress toward the mean. When subjects demonstrate extreme scores, their second scores are likely to be less extreme

• E.g. The first score was their ability or chance?

Selection Bias

differential recruitment of subjects

• E.g. Diversity/multicultural workshop (preaching the choir)

Mortality (attrition)

differential drop-out of subjects because of illness or resentment

Contamination

communication between members of two groups

Interaction of pretest and treatment

If pretest sensitized the subjects, it might have increased the educational effect of X.

T1 (before)		T2 (after)
O1	X	O1’

Interaction of artificiality and experiment

hawthorn effect/laboratory setting facilitated the efficacy of treatment

• E.g. watching an educational film in laboratory or on TV at home

Strengths (experiments)

• isolation of experimental variable’s impact over time

• replication of experiment

Weaknesses (experiments)

artificiality of laboratory settings

Field experiment

examines an intervention in the real world rather than in the lab

• in field, IVs cannot be manipulated as easily as in laboratory

• Experiment in the field can not be isolated as laboratory

• Field experiments have more external validity as their settings are natural

• Often combined with survey method

  • E.g. studying natural disasters

Epistemology

Philosophy that studies the process through which knowledge is acquired

Positivism

Social reality is given to us, human beings a passive agents, natural law governs behaviors

• emphasizes value-free research (objectivity) and studies only observable/measurable objects

Constructionist

Social reality is constructed by individuals, constructions through interpretations, multiple realities exist, and realities vary depending upon time and place

Characteristics (Qualitative studies)

Emphasize subjective reality

• Exploratory study rather than confirmatory

• cultural study using ethnographic methods

• theory development rather than theory testing

• study of subtle nuances of a phenomenon, study of subjective meanings

Studies subjective concepts (trust, alienation, empowerment), possible to achieve high validity but does not have the purpose of generalizing findings, cannot achieve reliability in terms of statistics, and is very labor intensive

Iterative Process

From problem specification to writing up the results (gradual)

• includes… participant observation, face-to-face interviews, focus groups, archival/document analysis, Open-ended responses in surveys

Grounded Theory Method

An inductive approach to the study of social life that attempts to generate a theory from the constant comparing of unfolding observations

• Purpose → to develop theory about phenomena of interest → the theory needs to be grounded/rooted in observations

Phenomenology

Concerned with the study of experience from the perspective of the individual

• emphasizes the importance of personal perspective and interpretation

• A school of thought that emphasized focus on people’s subjective experiences and interpretations of the world

  • The phenomenologist wants to understand how the world appears to others

Historical analysis

A method that seeks to make sense of the past though the disciplined and systematic analysis of the “traces it leaves behind”

• E.g. artifacts, written documents

Ethnography

Emphasis on studying an entire culture

• ethnography is an extremely broad area with a great variety of practitioners and methods

• The most common ethnographic approach is participant observation as a part of field research

  • Ethnographers become immersed in the culture as an active participant and records extensive field notes

  • As in grounded theory, there is no preset limiting of what will be observed and no real ending point in an ethnographic study

Participant Observation

The use of observation to understand attitudes and behaviors in natural settings, to study subtle cultural aspects within communities and organizations, when subjects cannot fill out questionnaires, to study processes and dynamics, and phenomena which are not measurable but observable.

Observer effect

Also known as Hawthorn Effect, participants change behavior to be more prosocial/productive when they are being observed

Cautions (Participant Observation)

Personal bias in interpretation, observer effect, previous knowledge about the topic

• literature can skew knowledge and opinions, creating bias for the researcher

Observer (participant observation)

Focuses on emergent (not planned) and contextual phenomena

• Stages of observation…

  • Descriptive (unfocused), focused (narrower range), selective (focus on emergent themes)

Cultural study

looking at norms and practices

case study

events (divorce, drop out)

• the in depth examination of a single instance of some social phenomenon

Organizational Study

studies roles, titles, cliques, etc.

Complete Participant

Investigator’s identity as researcher is unknown, investigator fully participates

Participant as observer

Investigator’s identity as researcher is known, investigator fully participates

• most frequently employed

Observer as Participant

Investigator’s identity as researcher is known, investigator interacts with participants but does not fully participate

Complete Observer

Investigator’s identity as researcher is unknown, investigator has no interaction with subjects, does not participate in behavior under study

• E.g. Observation in shopping mall

Focus Groups

A group of subjects interviewed together, prompting discussion

• creates group dynamics, one group consists of 5-8 people, there is a moderator/interviewer, times varies between 3-minutes and 2 hours, audiotape the interviews, if the target population is heterogeneous create subgroups

  • Advantages - real life data, flexible, fast, inexpensive

  • Disadvantages - not representative, little interviewer control, difficult analysis, interviewer/moderator skills, difficult logistically

Face to Face Interviews

Conduct an interview individually, gaining entry into the community (build trust), utilization of informants, use of semi-structured interview schedule, audiotape interviews

• loss of anonymity → can only be confidential

Theoretical Sampling

Themes that emerge through the interviews will lead to next sampling

• E.g. hearing about mental health issues → doing an interview on mental health issues.

Theoretical Saturation

The point where new data will not lead to new findings

• Hearing the same thing signals a stopping point

Grounded Theory

Development of logical/theoretical explanation

Criterion Sampling

Sample subjects which meet certain criterion/criteria

Unique Cases

Selecting subjects who are extremely different (case study)

Data Analysis (face-to-face interviews)

Transcribe the data verbatim (word for word), cleaning the data, read through the transcripts (while taking notes), mark meaningful quotations

Coding

Assigning numbers (colors, underlines, marks, etc.) to segment

Segment

Analysis unit which is comprehensible when read outside of the text

Category (variable)

A group of segments which tap similar phenomenon or meaning

Core Category

Category which is importance and or meaningful to the study

Theme

Construct to be used for theory contruction

Grounded Theory Method (face-to-face interviews)

Open coding - reading through each transcript while assigning numbers

Axial Coding - intense analysis around one category

Selective Coding - connecting several themes

Descriptive

Description of phenomenon

Interpretive

Interpretation of “other’s” experience

Theoretical

Explanation of phenomenon

Content Analysis

A research tool used to determine the presence of certain words or concepts within texts or sets of texts

• Study of recorded human communication…

  • Researchers quantify and analyze the presence, meanings, and relationships of words and concepts, then make inferences about the messages within the texts, the writer(s), the audience, and even the culture and time of which these are apart

  • Appropriate topics include → books, websites, paintings, and laws