research methods midterm

uw madison, spring 2023

measured variable

variable observed or recorded

manipulated variable

variable that is changed

operationalize

to make a construct measurable by giving it observable characteristics (measured/manipulated variable)

frequency claim

how often/how many times a single variable is observed, the rate/degree

association claim

when a variable is correlated with another in certain situations

causal claims

a variable changes another variable

What are the 4 big vaildities

1. internal
2. external
3. construct
4. statistical

internal validity

the possibility of a 3rd variable explaining the relationship is ruled out

external vaildity

the findings are generalized to different populations and settings

construct vaildity

the operationalized measurements are measuring what they say they are

statistical vaildity

the data supports the hypothesis/findings, findings are precise, reasonable, and replicable

temporal precedence

you are able to establish that one variable occurred before the other variable

random assignment

assigning participants to the control/experimental groups randomly

what form of validity does random assignment support

internal validity

empiricism

using the sense and observing behaviors to determine facts and answer questions

theory-data cycle

revising and adjusting theories by collecting new data

theory

a set of statements that explain the relationship between variables, supported by data

hypothesis

an educated guess as to what the results of an experiment will be

data

the information gathered from observations

preregistered hypothesis

coming up with a hypothesis before you being testing and collecting data

replication

the process of redoing the test to determine if you will get the same results or not

weight the evidence

the more data continues to show support for/is against the hypothesis, the stronger the likelihood of the hypothesis being correct

do theories prove anything? do results prove theories?

no, results support theories and theories may show consistency in explaining situations

flasifability

its possible to prove a theory wrong by testing a new hypothesis

merton’s scientific norms

1. universalism: claims evaluated by merit
2. communality: knowledge is created and shared with the community
3. disinterestedness: your personal beliefs should not interfere with the results
4. organized skepticism: question everything

applied research

doing research to address a specific problem in the real-world

basic research

collecting data and doing research for the sake of adding to our knowledge base

translational research

taking data that has been collected in basic research and applying it to create new policies, programs, treatments, etc.

scientific journals

peer re-veiwed articles all in one place, typically all concerning the same field of study

science journalism

taking peer reviewed articles and making them easier to read for the public to understand what researchers are finding, typically isn’t accurate descriptions of research

confounds

3rd variable that messes up the relationship between two variables, you don’t know its messing it up though

probalilistic

research findings dont explain all cases all the time

biased reasoning

assuming something seems true, but in reality it isn’t actually true

availability heuristic

certain thoughts are more likely to pop up in your head because they are easier to remember

present/present bias

you are more likely to remember something that happened, as opposed to when something didnt happen

confirmation bais

You cherry pick information so that you only pay attention to the information that supports your views

Bias blind spot

You assume that you aren’t capable of having biases, only other people have those

Authority

Just because someone is in a position of authority doesn’t mean that they’re always right

scientific sources

Scholarly articles, peer reviewed journals, data collected in research, full-length books, chapters in edited books

review journal articles

An article that collects a bunch of peer reviewed articles in the same field and summarizes/integrates them

abstract

the general summary of your study

intro

Outlines previous research, how your research connects to that, what your research question is, and your hypotheses

Method

Contains your participants (demographics, compensation), materials (surveys, instruments), and procedure (detailed description of how the study was conducted)

results

Gives the final data collected, as well as the statistical analyses

Discussion

Lays out what the findings mean, where there were flaws, and what research could be done moving forward

references

all sources that you cited in your article

***3 Types of Operationalized Measures***

1. Physiological
2. Observational
3. Self-report

measured variable

Variable observed and recorded

manipulated variable

variables that change

frequency claim

how often/how many times a single variable is observed

association claim

when a variable is correlated with another variable in certain situations

causal claim

a variable that changes another

***4 Big Validities***

1. internal
2. external
3. construct
4. statistical

internal validity

The possibility of a 3rd variable explaining the relationship is ruled out

external validity

the findings are generalizable to different populations and settings

construct vaildity

the operationalized measurements are measuring what they say they are

statistical validity

the data supports the hypothesis/findings, findings are precise, reasonable, and replicable

temporal precedence

you are able to establish that one variable occurred before the other

random assignement

assigning participants to the control/experimental groups randomly

what validity does random assignment support?

internal validity

self report

When the participant records their responses through a survey, poll, or questionnaire

categorical variables

Variables about participants that are qualitative and need to be coded (sex, SES, race)

quantitative variables

Variables about participants that are coded with meaningful numbers (height, weight)

three types of quantitative variables

1. ordinal: ranking, not set value
2. interval: intervals are meaningful, no true zero
3. ratio: intervals are meaningful, and there is a true zero

***Test-retest reliability***

When you do the test again, you still get the same results

***Forced-choice question***

Participants are only given 2+ choices to answer from, or yes/no question

***Internal reliability***

when do you use a bar graph

categorical variables

how do you determine the validity?

content validity

The questions seems to be appropriately operationalized to measure what they’re looking for; Fully captures all parts of a construct

criterion validity

The measurement appropriately relates to the behaviors associated with the construct; V important for stat-data

known-groups paradigm

When you have a group of people who have already been established as being part of a certain group, they should still get the same scores on other measurements testing that same group

convergent validity

When comparing a measurement to another measurement that tests the same thing, you should get the same scores

***Discriminant validity***

When comparing a measurement to a measurement that tests something similar, you should get different results

***Open-ended question***

Participants are given the option to respond however they’d like, takes a lot of time to code responses though

likert scale

A sliding scale from Strongly Disagree (1) to Strongly Agree (5) which participants choose where they land when reading a certain statement

***Semantic Differential Format***

Likert scale, but instead using adjectives - Very Attractive to Not Attractive at All

leading questions

Guide participants to respond in a certain way

***Double-Barreled Question***

2 questions in one

***Negatively worded questions***

May use double negatives in the question, or use the words “don’t, can’t, never, etc” which may confuse participants

***Response sets/Nondifferentiation***

Individuals answer the exact same way for every question (all positive, all negative, all neutral)

***Acquiescence***

Only respond to questions with “Strongly Agree/Agree”

***Reverse-worded items***

Used to combat acquiescence, just flip the question’s format

fence sitting

always choosing a neutral option

***Socially desirable responding***

Chose the option that they think they should choose instead of how they really feel because it’s controversial/they’re embarrassed

what does S-data describe?

It describes what people are feeling, NOT __why__ people do certain things

observer bias

Person observing has their own biases that impact how they perceive people’s behaviors

observer effects

Observers subconsciously impact participants responses by subtle tone/movements

reactivity

Participants will act differently when they know they’re being watched

census

survey of entire population

***Biased Sample***

When certain participants have a higher chance of being included in the study

***Probability Sampling***

Using simple, systematic, or cluster sampling to find an unbiased sample of participants

***Simple Sample***

Pick participants at random using a random number generator

***Systematic Sample***

Pick every X number of a participant as your sample

cluster sample

Pick your sample out of predetermined groups (athletes, high school students)

***Multistage Sample***

Get a cluster and then pick a random sample from that cluster

***Stratified Random Sample***

Randomly select participants from a demographic category (strata)