Research Methods: No chapter & Chapter 14

Hypothesis Testing (no chapter):

  • Hypothesis→a specific prediction derived from a theory & a specific research design.

  • Ho (null hypothesis) → groups/conditions will NOT differ

  • Ha (alternative hypothesis) →groups and conditions WILL differ.

  • Descriptive Statistics→differnet for our 2 conditions

  • Just because the means are different DOES NOT mean that groups are TRULY different

  • This could be a result of random “chance” variations in individuals or because of measurement error

  • Error Variance→Difference due to chance or miscalculations

  • Inferential Statistics→we can specify the probability that those differences are due to error variance 

  • “Reject” or “fail to reject”

  • “Rejecting” the Null →high probability that the null is false or low probability that these groups differ due to error variance 

  • “Fail to reject” the null→ low probability that the null hypothesis is false or high probability that these groups differ due to error variance.

  • Type I Error→concluding there’s a difference when there isn’t one at all

  • Alpha Level (p-value) →the probability of making a type I error (incorrectly rejecting the null); the smaller you set the alpha, the less likely you will make a type I error. 

Set alpha @ p < .05

  • Type II→ fails to find differences between groups thus failing to reject the null when there is a true difference

  • Why would we make type II errors…

  • Weak manipulations

  • Insensitive measurements

  • Ceiling or Floor Effects→a substantial portion of the sample has a score approaching the maximum (ceiling effect) or minimum (floor effect)

  • Noisy data or random error: too much unsystematic variability; could be a result of individual differences or situational noise.

  • Individual Differences→can be an issue w/ between-subjects designs

  • Situation Noise→unrelated distraction in the external environment that creates systematic variability.

Ex: experimenting outside, literal outside noises might distract the subjects

  • Preveting type II error…

  • Employ a strong manipulation

  • Carefully control the environmental situation 

  • Add more participant 

  • Use a within-subjects design

All of these things increase our Power!

Power:

  • Power→the ability to detect the “true” differences between groups; power is the ability of an experiment to correctly reject the null when the hypothesis is incorrect/false.

  • Control the environmental situation: standardized environment, minimizing any potential external influences, and be consistent among all participants

  • Add more participants: the more people you measure, the less impact any single person will have on a group's average; Helps reduce the influence of individual differences. 

  • Larger samples = more precise estimates

  • Use Within-subjects design: not concerned w/ individual differences because participants are compared w/ themselves.



Experimental Design:

  • Simplest Experimental Design→one IV w/ 2 levels; Use a t-test; all of the other designs do not use a t-test

  • T-test→tells us whether there is a statistical difference between 2 groups

  • Unpaired T-test→one-way w/ 2 levels; IV is between-subjects using random assignment

  • Question: how do you know how much error is in your data?

t = -2.75, p = .013 →null can be rejected!   .013 < .05



Replication, Transparency, and Real-World Importance: Chapter 14

  • Replication→finding the same results when we conduct the study again 

  • 3 Major Types of Replication Studies:

  1. Direct

  2. Conceptual

  3. Replication-plus-extension 

  • Direct replication→when researchers repeat an original study as closely as they can 

  • Participants from the same population 

  • Use the same materials 

  • Follow the same procedures
    In direct replication, we are interested in: testing the reliability of previous findings 

  • Testing the reliability of previous findings!

  • May be differences in: population, materials, procedures

  • Conceptual Replication→ interested in the generalizability of previous findings

Ex: do the findings generalize children?

  • Replication-plus-extension→add variables to test additional questions 

  • Other populations

  • Other settings

 Ex: Note-taking via computer V.S. handwritten

  • The facial feedback hypothesis→our emotional experience can be influenced by our facial experiences. Smiling = happy     frowning = sad 

  • When we are conducting a direct replication we are interested in finding the internal validity!

  • Conceptual replication or a replication-plus extension is focused on external validity!

The more settings and populations in which a study is conducted, the better you can assess generalizability

  • Cultural Psychology→subdiscipline in psychology focusing on how cultural contexts shape the way a person thinks, feels, and behaves. 

  • A failed replication doesn't mean the study is useless

  • However, failed replications do suggest the original findings may be more limited than initially thought



Questionable Research Practices:

  • Underreporting null findings→research may include several different variables, & levels within those variables

  • Underreporting null findings occurs when researchers mislead by only reporting the conditions or measures that support their hypothesis.

  • HARKing Meaning: researchers create a hypothesis after the results of found

  • Hypothesizing

  • After

  • Results are

  • Known

  • p-Hacking→analyzing data in multiple ways until the effect becomes statically significant 

  • It’s not always intentional! However, when researchers aren’t transparent about how they run their data…it does get a bit fishy



Transparent Research Practices:

  • Open Science→the practice of sharing one’s data & materials freely so others can collaborate, use and verify the results 

  • Preregistration→when scientists publish their study’s method, hypotheses, and plan for statistical analyses BEFORE they collect data

  • Underreporting becomes a lot harder with these steps put in place before hand^^

  • People state their data beforehand thus you can’t make your hypothesis after.

  • For p-hacking other scientists can re-run and confirm statistical analyses.