EXAM #1

empiricism

an approach in the science of basing decisions on data and using insights from data to develop, support, and/or challenge a theory

the theory-data cycle

theory: a statement or set of statements that describes general principles about how variables relate to one another

prediction/hypothesis: a way of stating the specific outcome in the data that the researcher expects to observe if the theory is accurate

data: a set of observations representing the values of some variable, collected from one or more research studies (the backbone of how theories become stronger or weaker after observation)

qualities of a good theory

supported by data: supporting data from multiple studies/labs

falsifiable: a theory must be testable, such that some imaginable pattern of data can prove it wrong

parsimony: when two theories both explain data equally well, the simpler theory is preferred

biases of intuition

a feeling of knowing without direct evidence or experience, such that the information feels like it is known instinctively

biases of experience

availability heuristic: mistaking the ease and frequency of recall with the probability or likelihood of the phenomenon

confirmation bias: tendency to consider only information or interpretations that agree with what we already believe/only accepting evidence that already aligns with our belies even when alternative evidence is available

bias blind spot: the belief that we are unlikely, or less likely, to fall prey to biases in decision-making than other people

importance of comparison groups

for evaluating the true effects of an intervention by comparing outcomes between treated and untreated groups

enhancing internal validity and controlling for confounding variables

identifying confounds in experience

recognize alternative factors that may influence the outcome of results in a scientific study

forms of dissemination for scientific theories and results

methods used to share and communicate scientific theories and findings with various audiences

what is an empirical journal article; what are its parts

presents original research findings

1. introduction

2. method

3. results

what are common sources for scientific versus popular audience

scientific: peer-reviewed journal articles, academic books, and conference papers

popular: magazines, newspapers, and online blogs

what is a variable

any characteristic, trait, or condition that can take on different values or levels in research

how are variable types represented in jamovi

nominal: three circles

ordinal: line graph

continuous: ruler

types of variables

nominal

ordinal

continuous

quantitative

nominal vs. quantitative variables

nominal: categories without order (gender, color, etc)

quantitative: numerical and can be measured

quantitative variables: ordinal, interval and ratio scales

ordinal: ranks with no defined differences

interval: equal intervals without true zero

ratio scales: equal intervals with a meaningful zero

what is the difference between independent and dependent variables

independent: manipulated

dependent: measured

what is the difference between measured and manipulated variables

measured: observed and recorded without interference

manipulated: intentionally changed by researchers to asses impact

different types of claims

frequency, association, causal

frequency claim

single variable

statement of how often something occurs

often used to draw attention to prevalence

association claim

assert that two variables are related to each other

the frequency of one variable is tired to or linked with the frequency of another

cannot assert strongly why the relationship exists

causal claim

argue that two variables are related because one variable causes another variable

directionality in the relationship is asserted: “variable x causes change in variable y”

look for directed terminology: leads to, may lead to, affects, causes, increases, decreases, changes, etc.

independent variables: manipulated

dependent variables: measured

mental constructs

behaviors that can not be physically observed but are affecting the minds of every participant (intelligence, motivation, and personality)

operational definition

how a concept is explicitly measured

has clear rules

does not overly depend on interpretation of another concept that is not explained or observable

results in a variable with multiple values for each person in your study sample

results in a valid measurement of the construct

types of associations

positive, negative, no association

positive associations

variables increase and decrease together

negative associations

one increases while the other decreases (opposite)

no associations

no pattern

four main validities

construct

external

internal

statistical

construct validity

the extent to which a measure provides an accurate estimate of a concept

what are question we ask for construct validity

does the measure provide an accurate estimate of the theoretical construct?

is the operational definition appropriate?

ways of evaluating construct validity

evaluate construct validity of frequency claim by addressing measurement of the single variable in the claim

evaluating construct validity for an association claim: address each variable

statistical validity

the extent to which the study analyses were accurate and the strength of the effect

what are questions we ask for statistical validity

did they do the right statistical test?

how big is the effect?

what is the margin for error in the outcome variable?

likelihood of occurring by chance- can the effect be replicated?

internal validity

the extent to which confounds in the relationship between two variables have been minimized

role of confounds in internal validity

if yes, you have a confound- something other than the intended manipulated variable also varies between conditions or levels of the independent variable

confounds weaken internal validity

what are questions we ask for internal validity

is there an alternative explanation for change in the dependent variable?

were potential confounds in the manipulation controlled?

external validity

the extent to which the results of a study generalize to a larger population and/or to other situations

what are questions we ask for external validity

about people- who does the result creating the claim most apply to?

if I conducted this study with different participants, would I expect the same results?

about situation- is this finding representative of other circumstances?

people- who are participants and who does the result apply to?

situation- what is the situation and would results apply to other contexts, times, or places?

why is it hard to maximize both internal and external validity in one study

decisions that help you maximize internal validity often hurt external validity because they limit the population and situations studied

differences between reliability and validity

validity is accuracy

reliability is consistency

reliability

describes the consistency of scores across measurements

relationship between reliability and validity

a measure must be reliable to be valid BUT a reliable measure is not necessarily valid

types of reliability

test-retest

interrater

internal

test-retest reliability

consistency of scores across multiple testing occasions

does the test give similar results when given multiple times to the same person?

interrater reliability

similarity between raters

how similar are two different people’s recorded observations of a behavior?

internal reliability

applies to measures with multiple items (surveys, achievement tests, skills tests)

quantifies how similar responses are to questions designed to measure the same construct