research methods exam 2

Correlation

• statistical relationship between two quantitative (numerical and express quantities, amounts or counts) variables

  • as variable X changes what happens to variable Y

  • relationship involves strength and direction

Pearson’s r

statistic that quantifies the strength of a correlation between quantitative variables

• good for measuring linear relationships, not nonlinear (curvilinear) relationships

• -1 to 1

Directionality of Pearson’s r

• signified by (+ / -) of pearson’s r

• positive correlation - as x increases, y increases

• negative correlation - as x increases, y decreases

strength of pearson’s r

• how closely two variables “move” together

• strong correlation = closer to -1 or 1

• weak correlation = closer to 0

null relation

r = 0 and there is no pattern or relation

Correlational research

• non-experimental research - no manipulation of IV or causal conclusions

• can test many different research questions

  • is x (associated, linked, related) to y

why might correlational research be conducted?

• bc researchers don’t think relationship is causal or not interested in causality

• can’t manipulate IV (unethical/not possible or practical)

• establish reliability + validity of measures

• can be higher in external validity then controlled experiments

• can provide additional, converging evidence following experiments

restriction of range

it is important to ensure variables in your study are not limited/restricted in range

• need full coverage of entire variable

• range restriction can lead to bias correlation coefficients that results in misleading statistical findings

correlation is NOT

NOT causation

the bi-directionality problem

correlational research cannot tell us the causal directionality of a relationship

• x could cause y, but y could also cause x

third-variable problem

• two variables could be related because of a third confound/unmeasured variable

spurious correlations

correlations that are affected by confound/third variables

causal relationships - could conduct control trial

higher levels of exercise reduce peoples stress

Correlational relationship

people who consume more caffeine tend to report increased levels of anxiety

causal relationship - not possible to manipulate IV (age)

increased age results in worse memory

correlational - not able to test IV

lower conscientious is linked to higher rates of aggressive behavior

observational research

• refers to the several different types of non-experimental studies in which behavior is systematically observed + recorded

• goal is to obtain snapshot of specific characteristics

• nothing is manipulated or controlled

naturalistic observation

• observer = passive recorder

• occurs in natural setting

• low internal validity but high external validity

natural setting

place where behavior would occur normally

structured observation

• laboratory setting - artificial/consistent setting

• higher internal validity

Observations

• can be better then self reports - more valid/reliable

• better for finding “uncomfortable” data

Observer bias

when observers see what they expect to see

observer effects

when participants confirm observer expectations (expectancy effects)

• participants do the action they know is being tested of them

what prevents observer bias and observer effects

1. well-trained observers

2. inter-rater reliability measure

• Chronbach’s alpha!

3. masked research design (observers are blind to hypothesis)

Reactivity

• when participants react to being watached

two solutions

1. blend in (unobtrusive observations)

2. wait it out (participant habituation)

participant observation

when researchers become active participants in group/situation they are studying

• can be undisguised or disguised (need IRB approval)

possible drawbacks for participant observation

• researchers may change social dynamics and/or influence behavior of the people they are studying

• researchers may become less objective and influence part of study

Structured observation

researches make careful observations of one or more specific behaviors in particular setting that is more structured then the settings used in naturalistic or participant observation

• gather quantitative rather then qualitative data

• researchers are interested in limited set of behaviors which allows them to quantify these behaviors

coding

• requires clearly defining a set of target behaviors

• participants are categorized based on which behaviors they’ve engaged in and number of times they engaged in each behavior

case study

• in depth examination of individual

• tends to be more qualitative

• provides higher level of analysis

• only way to study rare conditions

• can’t be used for evidence for theories - but can help develop theories due to small sample size

survey methodology

you need to identify the means of collecting data

• telephone interviews

• in-person surveys

• mail surveys

• internet surveys

two main survey research assumptions

1. People are cognitive misers - we liked to take shortcuts

• less thinking in surveys = better

• participants “life bars” decrease thru surveys

2. people are willing to respond truthful + accurate

• good survey questions increase these odds

open ended research question

people can respond however they want

• ex. what is your gender

close ended research questions

people have limited number of response questions

• ex. what is your gender - man, women, non-binary, not listed, prefer not to say

guidelines for writing survey questions

• make questions simple

• know your audience

• be specific

• avoid double-barreled questions

• avoid leading and loaded questions

• avoid neg wording

• consider people’s response set tendencies

double-barreled questions

• most common issue in surveys

• avoid questions that ask about two or more ideas simultaneously

ex. have you tried cigarettes or vaping in past month?

Leading questions

sway or lead people to respond in a particular way

• researchers bias comes into play

ex. how often do you engage in immoral behaviors like theft??

Loaded questions

questions that assume something about the respondent (ex. knowledge, behavior, identity)

• where do you enjoy drinking beer??

negative wording

• difficult to interpret (double negatives)

• lowers understanding of the question and taxes our participants “life bars”

ex. How much do you agree or disagree w the following?? “i dont buy much online, i nver binge wath netflix, i oppose online classes.”

response sets

People tend to stick to a response style

• social desirability (“I should respond this way”)

• extreme wording (“definitely yes! or definitly no!”)

this is another reason why we often have a mix of questions to get a certain construct

response rate

number of people who participated in a survey divided by the number of people selected for the sample

• ideal = higher then 70%

• acceptable for validity = 50%

low response = bad because results are representable of entire group

shorten survey time could help improve response rate

accuracy of survey results

survey researchers have refined their techniques so that surveys are generally pretty accurate

• accuracy depends on how well the samples are chosen and on sample size

population

the entire group of type of people that you are studying - you want it to be generalizable

sample

smaller part or subset of the population - study participants

census

data collection that involves every member of the population of interest

• one out of every 10 years → determines voting lines

• difficult to gather data

survey

data collection that involves a subset of the population (sample)

what is census data used for

• state population determines how the 435 seats in house of reps are divided among the 50 states

• rewdrawing congressional, state legislative and school district boundaries

• dictates funding for medicated and SNAP programs

types of probability sampling

• simple random sample

• stratified sample

types of non-probability sampling

• self-selected sample

• convince sample (cianna)

• purposive sample

• snowball sample

sampling

adequate samples are necessary for valid research

two types: probability and non-probability sampling

probability sampling

each person in the population has an equal probability of being selected for the study

non-probability sampling

the probability of someone being selected to participate is unknown

simple random sampling

• everybody has equal chance of being selected

  • need to have full list of everyone - define population

• goal: sample is representative of population

• often the most preferred method

Stratified sampling

• break population into subgroups (strata) and then sample from every relevant sub-population

• need to have full list of everyone

• focuses on proportions - 75% from X and 25% from Y

• very common in national polling

self-selected sample

• sample in which people actively seek out the chance to participate in research

• considered bias - results are not generalizable to the population

  • participants differ from general population due to reasons causing them to want to participate

Convenience sample

• recruit those who are easily accessible

• goal: sample quickly and efficiently

  • ex. college students/online panels

• limits generalizability

NO - convenience samples are easy to obtain and how representative a sample is affects how broadly we can generalize findings

are college students actually representative of everyone?

advantages to using convenience samples

• fast

• inexpensive

• easy to recruit participants

• useful for certain research questions

limitations to using convenience samples

• selection bias

• less representative

• limited generalizability

• can be overrepresented

purposive sample

• recruit those who fit specific criteria + intentionally seek them out

• goal: target unique, special, or hard-to-reach populations

  • ex. LGBTQ+ unhoused youth, transgender service members

snowball sample

• have subjects recruit others to participate in your study

• goal: target networks of hard-to-reach or hidden populations

  • those who might be difficult to study bc the people may be engaged in secretive activities, so they don’t want to be recognized as members of that population

ideal sample techniques

• use probability sampling technique

• obtain large sample size

• get complete responses from all participants

difficult due to time, money, logistics, feasibility - response rates vary

experiment

type of study designed specifically to answer the question of whether there is a causal relationship between two variables

• AKA randomized controlled trials (RCTs)

fundamental features of an experiment

• researchers manipulate the level of the IV (conditions)

• the researchers exert control over variables other than the IV and DV

control variables

any variable that an experimenter holds constant between groups

control group

group that does not get the IV or gets some standard value of the IV

• serves as source of comparison

random assignment

• control technique to equate groups of participants

• accomplished by ensuring every person has an equal chance of being assigned to any group

• increases the chance that the groups are equal (balanced) prior to start of experiment

random assignment

• Eliminates systematic differences between groups

• distributes influence of extraneous variables among groups

• most important + powerful control technique

• random assignment = method for assigning participants in sample to different conditions

• simple random sampling = method for selecting sample from population

differences of random assignment and simple random sampling

one way design

• experimental design in which only one IV is manipulated

  • ex. two group experimental design

• experiment requires at least 2 levels of IV to compare one level on the DV relative to the other

between subjects design

• different people are compared to each other

• each participant is only in one experimental condition

posttest only design

• participants undergo manipulation then complete measures

• participants assigned randomly to groups → IV introduced → DV is measured

pretest pottest design

• participants first complete the measures, then manipulation, then measures again

• participants are randomly assigned to groups → DV measured → IV introduced → DV measured

advantages to pretest posttest design

• rules out selection bias - occurs when random assignment fails

  • we can use pretest scores to ensure that the two groups are the same

• use change scores is more powerful bc it eliminates noise

  • accounting for possible pre-existing differences within participants

within subjects designs

repeated measures within groups

• sample ppl are compared to themselves

• each participant is in ALL experimental conditions

within subjects design pro vs con

benefits

• each participant serves as their own control

• no possibility of selection effect

• noise is reduced

• doubles sample size

drawbacks

• order effects

order effects

1. fatigue effects

2. practice effects

3. carryover effects

fatigue effects

participants get worse at the measure (eg. boredom, distraction)

practice effects

participants get better at the measured (learning effect; ex. solving a complex puzzle)

carryover effects

effects of one condition contaminate subsequent responses

• biased bc of earlier experiments

counterbalanced design

in within-subjects design, presenting the levels of IV to participants in different sequences to control for order effects

conducting the experiment

1. set the stage

2. manipulate the IV

3. measure the DV

4. debrief participants

informed consent

• discuss purpose of study

• general procedure + amount of time

• risks + benefits of research

• compensation

• how identity + privacy will be protected

• voluntary participation and permission to withdraw

• contact info for questions

demand characteristics

features of a study that influence participants to become aware of (or think they know) the purpose of the study

• this can lead to changes in behaviors or responses (reactivity)

how to avoid demand characteristics

• be general abt what will happen, what participants will do, avoid explaining true/specific purpose

• make it difficult to figure out what study is abt - use deception + filler tasks

• ask them their thoughts during debriefing - may need to remove participant from analyses if their guess is correct

manipulating IV

• straightforward manipulations

  • manipulate w written instructions, verbal materials, and/or visual materials

• staged manipulations

  • manipulate an entire setting ; event manipulations

  • often uses confederates

    • a person who appears to be another participant (or random person) but who is part of the experiment

selecting a manipulation strength and type

1. ethical considerations

2. realism of the experiment - concerns external validity

3. money and resources

measuring/checking the DV

• beware of floor effect - all low scores on measure

• beware of ceiling effect - all high scores on measure

debriefing participants

• explain true purpose of study

• mitigate any neg consequences of manipulation

• can only keep secret when deception (staged manipulation) is involved