Research Methods Exam 10/30

Category Variable: Quantitative

Focus: Numbers, hard facts, and statistical data

Category Variable: Qualitative

Focus: Words, ideas, images, and human experiences

Category Variable: Participant (subject) variable

Particular to the subject; characteristic or trait that a person brings with them to the study

Category Variable: Independent Variable

Manipulate

Category Variable: Dependent Variable

Measures

Category Variable: Extraneous

any factor in a research study that is not the main focus of the experiment but can still affect the dependent variable

Operation al Definitions

operations/procedures to measure/manipulate variables

Ex: name of scale; number of questions

Measurement Scale: Nominal

distinct, unordered categories using names, labels, or arbitrary numbers

Ex: gender, marital status, blood type, geographic location, brand names

Measurement Scale: Ordinal

learn, logical order or rank

Ex: education level, income status, race placement

Measurement Scale: Interval

quantitative level of measurement where data is ordered, the difference between values are equal and meaningful, and the zero point is arbitrary

Ex: temperature, IQ scores, calendar dates and years

Measurement Scale: Ratio

order values, equal intervals, and a true zero point

Ex: time and duration, financial metrics, length, height, mass, weight

Experimental Studies

Manipulate the IV, eliminate/control extraneous variables, random assignment, determine cause and effect

Experimental Studies: Random Assignment

any participant has a equal probability of being assigned to an condition

no systemic bais

random selection/sample: take whole list and pick randomly from population

Experimental Studies: Determine Cause and Effect

1. temporal precedence - hypothesized cause must occur before its measured effect (A > B)

2. Covariation - two or more variables to vary or change together in a related way

3. elimination other causes (extraneous variables)

Non-Experimental Studies

observe/measure, only DVs (measured variables), no casual conclusions

Non-Experimental Studies: No Causal Conclusion

Bidirectionally (A>B) (B>A) (Ex: anxiety, lack of sleep)

Confound (third variable)

Relationships

Postitive

Negative

no correlation - no pattern, no connection between variables

Curvilinear

Choosing a Method: Experimental Method

strengths - control/eliminate extraneous

weakness - time, unrealistic

Choosing a Method: Field Experiment

strength - outside/natural setting

weakness - hard to control extraneous variables

Choosing a Method: Quasi-Experiments

strengths - look at variables can’t control

weakness - no random assignment, no manipulation of IV

Choosing a Method: Multiple Methods

matching their specific research question, ethical constraints, and goal to the strengths of different study design

Validity: Construct validity

does the operational definition of a variable match the theoretical construct/meaning of the variable?

the extent to which a test, survey, or measurement tool accurately represents and measures the unobservable, theoretical concepts it claims to measure

Validity: Internal Validity

the ability to draw casual conclusions

a study can prove that changes in the dependent variable (the outcome) are caused directly by the independent variable (the intervention) rather than by outside, uncontrolled factors

Validity: External Validity

generalize results beyond the sample, situation, people, setting, times

Reliability

the quality of being dependable, trustworthy, or capable of performing consistency well over time.

is the measure stable?, consistency/stability, true score, measurement error

Assessing Reliability: Pearson’s r

time and 2nd time, -1 to 1, strength (absolute value)/ direction (+/-)

Assessing Reliability: test-retest

measure the same participants on two occasions

+0.8

problems - different study techniques before 2nd SAT, seasons/weather, remember how they respond

Assessing Reliability: internal consistency

correlations among items on the same measure

split-half - split measure in half and compare the first half to the second half

Chronbach alpha - correlates with every item

Assessing Reliability: Interrater (IRR) or (IOR)

individuals observe behaviors and compare observations (at least 2)

+0.8

Assessing Validity: Construct validity

does the operational definition match the true theoretical meaning of the variable?

Assessing Validity: Face validity

does the measure appear to measure what its supposed to measure?

Assessing Validity: Criterion-oriented validity

assessing relationship between scores and some criterion

Assessing Validity: Criterion-oriented validity: Predictive

use measure to predict some future behavior

Assessing Validity: Criterion-oriented validity: Concurrent

relationship between measure and behavior both measured simultaneously (at the same time)

Assessing Validity: Criterion-oriented validity: Convergent

score on measure relate to other measures of the same construct

Assessing Validity: Criterion-oriented validity: Discriminant

ensure should not be related to some variables

Ex: depression can not equal neuroticism

Measurement Problem: Reactivity

participant is aware of whats being measured so they might change behavior

Observational Research: Naturalistic

spontaneous behavior in natural environment

don’t get involved

qualitative data - write down what people are doing > open to interpret

problems: observer bias, wait for situations to naturally occur, trying to stay unbiased (not tell hypothesis), ethical approval (video taping)

Observational Research: Non-Naturalistic

participant observation

• undisguised > they know your observing, could change

• disguised > not know your observing, more natural

systematic observation

• observe few specific behaviors in a particular setting, set up for it to occur

Observational Research: Coding

could set up ahead of time > systematic

observe as you can them code > naturalistic

observer bias

• blind - don’t know hypothesis

• inter-rater reliability > different observers interpret similar way

  • IRR - agree/disagree (Ex: 8/10 =0.8 0.8 = 80%)

Sampling: Specimen record

observer records as much about behavior as possible

Sampling: Event sampling

identify target behavior and record when it occurs and ignore others

Sampling: Time Sampling

choose time intervals for observations, observe behavior in that time

Ex: 1 min time periods

Sampling: Time-point sampling

record behavior only at a certain time point ahead of time, also target behaviors ahead of time

Sampling: Situation sampling

look for a specific behavior in a certain location

Other Non-Experimental Research: Case Studies

one individual/group/phenomenon in depth

Other Non-Experimental Research: Archival Research

looking at archives to explore data

Ex: FBI database

Other Non-Experimental Research: Content Analysis

study some type of media

Survey Research

questionnaires and interviews

attitudes, behaviors, demographics

• political attitudes, poll, votes

• depression/anxiety are not survey’s, they’re instruments

Constructing Questions: open-ended

what is the most important reason you decided to attend ECSU?

Constructing Questions: closed

considering the choices below, what is the most important reason you decided to attend ECSU

• the school is close to home

• the school is the size I preferred

• the school has a beautiful campus

• the school has a liberal art focus

problems: answers don’t represent why, bad response choices

Constructing Questions: Scales

bipolar scale label extremes, five or seven alternatives

• strongly disagree - - - - - strongly agree

• (not at all confident) 1 2 3 4 5 6 7 (extremely confident)

Constructing Questions: Order

demographics first or last?

• don’t need some information ( school year, political affirmation)

  • gender, age, ethnic background

• general to specific

• reactivity concerns > change in behavior, know what study is, change responses

  • scramble questions

Constructing Questions: Problems

too complicated, too long, addresses more than one issue, too vague, leading question, negative wording, response set (some participants will agree with everything or disagree = bias), social desirability (answer in social acceptable ways)

Constructing Questions: Pretest Survey

testing before giving to actual participants to make sure there are no errors

Questionnaires

written

strengths - easy to code, less time consuming, cost less, anonymous

weaknesses - response set, social desirability

Interviews

strengths - follow-ups

weaknesses - consistency, harder to code

Direct mail survey

response rate is low, don’t know the characteristics of non-respondents that make them different

Email, Internet

representativeness

Smartphone research

reach at anytime

Focus group interview

group of people talk in depth about topic, interviewer bias

Sampling: Population

all individuals of interest

Sampling: Sample

some subset of population

Sampling: Sampling frame

list of all individuals in population one draws sample from

Sampling: Sample Size

large sample size - the better, sample representative of population

small sample size - smaller margin of error,, sample value is to be closer to population value

Sampling: Probability Sampling

random sampling - equal chance to be selected

stratified random sampling - split population to sub groups that reflect characteristics then random select from groups

cluster sampling - identify groups of individuals then pick random from other groups

Sampling: Non-Probability Sampling

probability to be chosen is not known

haphazard sampling - ask anyone to participate (convenience)

purposive sampling - looking for a sample to meet certain criteria

quota sampling - choose sample that reflects important population

Sampling: Representative Sampling

a small subset of a larger population that accurately mirrors the key characteristics, demographics, and traits of the entire group