2024-25 Research Methods & Data Interpretation

hindsight bias

the tendency to believe, after learning an outcome, that one would have foreseen it. (I-knew-it-all-along phenomenon)

theory

an explanations using an integrated set of principles that organizes and predicts observations.

hypothesis

a testable prediction, often implied by a theory; to be useful, this must be a falsifiable claim, meaning it must be possible to gather evidence that would refute the claim (if it exists)

operational definition

a statement of procedures (operations) used to define research variables. (ex: memory may be defined as "number of words correctly recalled from a list").

replication

repeating the essence of a research study, usually with different participants in different situations, to see whether the basic finding extends to other participants and circumstances.

case study

an observation technique in which one person is studied in depth in the hope of revealing universal principles.

survey

a technique for ascertaining the self-reported attitudes or behaviors of people, usually by questioning a representative, random sample of them.

false consensus effect

the tendency to overestimate the extent to which others share our beliefs and behaviors.

population

all the cases in a group, from which samples may be drawn for a study.

random sample

sample that fairly represents a population because each member has an equal chance of inclusion.

naturalistic observation

observing and recording behavior in naturally occurring situations without trying to manipulate and control the situation.

correlation coefficient

a statistical measure of the extent to which two factors vary together, and thus of how well either factor predicts the other

scatterplot

a graphed cluster of dots, each of which represents the values of two variables. The slope of the points suggests the direction of the relationship between the two variables. The amount of scatter suggests the strength of the correlation.

illusory correlation

the perception of a relationship where none exists; the basis for many superstitions

experiment

a research method in which an investigator manipulates one of more factors (independent variables) to observe the effect on some behavior or mental process (dependent variable). By random assignment of participants, the experiment controls other relevant factors.

double-blind procedure

an experimental procedure in which both the research staff are ignorant (blind) about whether the research participants have received the treatment or placebo.

placebo effect

experimental results caused by expectations alone; any effect on behavior caused by the administration of an inert substance of condition, which is assumed to be an active agent.

experimental group

A subject or group of subjects in an experiment that is exposed to the factor or condition being tested.

control group

the condition of an experiment that contrasts with the experimental condition and serves as a comparison for evaluating the effect of the treatment., In an experiment, the group that is not exposed to the treatment; contrasts with the experimental group and serves as a comparison for evaluating the effect of the treatment.

random assignment

assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups.

independent variable

the experimental factor that is manipulated; the variable whose effect is being studied.

dependent variable

the experimental factor - in psychology, the behavior or mental process - that is being measured; the variable that may change in response to manipulations of the independent variable.

descriptive statistics

Statistics used to describe only the observed group or sample from which they were derived; summary statistics such as percent, averages, and measures of variability that are computed on a particular group of individuals.

inferential statistics

numerical methods used to determine whether research data support a hypothesis or whether results were due to chance (e.g. p-value)

mode

the most frequently occurring score in a distribution.

mean

the arithmetic average of a distribution, obtained by adding the scores and then dividing by the number of scores.

median

the middle score in a distribution; half the scores are above it and half are below it.

range

the difference between the highest and the lowest scores in a distribution.

standard deviation

a computed measure of how much scores vary around the mean score.

statistical significance

a statistical statement of how likely it is that an obtained result occurred by chance.

APA Ethical Guidelines

these rules specify that researchers avoid procedures that might cause serious physical or mental harm to human subjects, protect confidentiality of the data, respect a subject's right to refuse at any time during the study; includes Informed Consent, Freedom to Withdraw, Debriefing, No Harm, and Confidentiality

overconfidence

the tendency to be more confident than correct—to overestimate the accuracy of one's beliefs and judgments.

confirmation bias

a tendency to search for information that supports our preconceptions and to ignore or distort contradictory evidence

wording effects

when a specific word used in a question affects how respondents answer the question or the order of the questions

p-value

The probability level which forms basis for deciding if results are statistically significant (not due to chance).

sample

items (often people) selected at random from a population and used to test hypotheses about the population

confounding variable

in an experiment, a variable, other than the independent variable, that could influence the dependent variable

debriefing

giving participants in a research study a complete explanation of the study after the study is completed; required by APA ethics guidelines

informed consent

agreement to participate in psychology research, after being informed of the dangers and benefits of the research

generalizability

the extent to which a study's findings can be reasonably assumed to apply to the study population (not just the sample); enhanced by having larger, random samples and large differences between (experimental and control) groups

social desirability bias

A tendency to give socially approved answers to questions about oneself; a potential challenge in surveys involving self-report

sampling bias

A problem that occurs when a sample is not representative of the population from which it is drawn.

correlation

A measure of the extent to which two factors vary together, and thus of how well either factor predicts the other.

skewed distribution

a representation of scores that lack symmetry around their average value

histogram

A graph of vertical bars representing the frequency distribution of a set of data.

normal curve

the symmetrical bell-shaped curve that describes the distribution of many physical and psychological attributes. Most scores fall near the average, and fewer and fewer scores lie near the extremes.

institutional review board (IRB)

A committee organized by a university or other research institution that approves, monitors, and reviews all research that involves human subjects. Its main purpose is to ensure compliance with ethics standards.

deception

Researchers sometimes need to keep details of a research design hidden from participants (or intentionally mislead them about the study's true purpose). Note: must be corrected during debriefing

3rd variable problem

A problem that occurs when the researcher cannot directly manipulate variables; another variable not measured in the study may explain the relationship between the two measured variables

Peer review

A process by which the procedures and results of an experiment are evaluated by other scientists who are in the same field or who are conducting similar research.

quantitative research

a research that relies on numerical data (e.g., counts, percentages, Likert scales, ratings)

qualitative research

research that relies on in-depth, narrative data that are not translated into numbers (e.g., structured interviews)

directionality problem

a problem encountered in correlational studies; the researchers find a relationship between two variables, but they cannot determine which variable may have caused changes in the other variable (e.g., Does A cause B? Or does B cause A?)

Likert scale

a numerical scale used to assess attitudes; includes a set of possible answers with labeled anchors on each extreme (e.g., "strongly disagree" to "strongly agree")

structured interview

a research procedure in which all participants are asked to answer the same open-ended questions

self-report bias

bias that occurs in survey data when people don't accurately report or remember their behaviors

Convenience sampling

using a sample of people who are readily available to participate

meta-analysis

a "study of studies" that statistically combines the findings of multiple studies to arrive at a conclusion about the trends in the research about a particular topic/question

Effect size

the standardized mean difference or magnitude of difference between two group's averages; often used to determine the "practical significance" of a research finding (as opposed to statistical significance).

Informed assent

participant's agreement to participate in the absence of full understanding; commonly applies to individuals who have not attained legal majority and/or capacity

confidentiality

the act of holding information in confidence, not to be released to unauthorized individuals; this is an ethical obligation of psychological researchers

anonymity

participants in research have right to remain unknown; unless they specifically grant permission otherwise, researchers must refrain from including details that would personally identify their participants

confederates

an "actor" in psychological and social research, this is a person who is working with the researcher and posing as a part of the study, but the subjects are not aware the individual is coordinating with the researchers

single-blind procedure

an experimental procedure in which only the research participants are ignorant (blind) about whether they have received the treatment or a placebo

regression toward the mean

the tendency for extremes of unusual scores to fall back (regress) toward their average

Within-subjects design

a research design that uses each subject as his or her own control, because each subject participates in all study conditions; like random assignment, this produces equivalent "groups" or conditions that can be used for comparison

percentile rank

the percentage of scores below a specific score in a distribution of scores

Cohen's d

a measure of effect size that assesses the difference between two means in terms of standard deviation (below 0.2 is considered small, medium (0.5) and any thing greater than 0.8 is large).