Research 2 Statistical Significance and Effect Size

Understanding Statistical Significance vs Effect Size

Statistical Significance

Statistical significance is introduced as the likelihood that observed findings in data are attributed to chance. This requires careful consideration of the sample selection used in studies, emphasizing that depending on the sample randomness, results can differ dramatically.

  • Key Concept: In research, observed differences between groups (e.g., Group A and Group B) might be misconstrued due to sampling error, which refers to errors resulting from a non-representative sample.

  • Statistical significance is quantified through calculations yielding a p-value, which signifies the probability that the finding reflects true effect versus random chance. A common threshold for statistical significance is a p-value of less than 0.05, indicating at least a 95% confidence level that the observed effect is not due to chance.

Effect Size

While statistical significance indicates the reliability of findings, it does not reveal the magnitude of the effect, subsequently leading to discussions on effect size.

  • Effect Size Definition: A measure of the strength of the relationship between two variables; in simpler terms, it illustrates how impactful a finding is in practical terms.

  • There are multiple methods to assess effect size, but the speaker highlights four prevalent methods used in psychology.

  • Pearson's R: A correlation coefficient ranging from -1 to +1 indicating the strength and direction of a linear relationship between two variables. An R value near 0 suggests a weak correlation, while values near -1 or +1 indicate strong negative and positive correlations, respectively.

  • The speaker notes that while Pearson’s R offers insight into relationships, it does not clarify how much variability in one variable can be explained by another, necessitating the calculation of R-squared (r²):

    • When calculated from Pearson's R, r² provides a proportion of variance explained in the dependent variable by the independent variable.

    • Although straightforward for simple correlations, r² calculations for more complex models may involve different methodologies.

Practical Example: Cilantro

The speaker uses cilantro as an anecdotal example of variability in perceptions; they mention that people have different taste experiences with cilantro due to genetic differences, which can relate back to statistical analyses of variance in qualitative research.

Types of Errors in Research

  • Type I Error (False Positive): The occurrence of concluding a difference exists when there is none, described with the analogy of incorrectly thinking data is valid under the 5% significance chance.

  • Type II Error (False Negative): Conversely, this error occurs when the study fails to identify an effect that does actually exist, utilizing an example of misreading statistical power and the implications of missing findings.

Hypothesis Testing

The conversation transitions into hypothesis testing using mindfulness intervention as a case study.

  • Null Hypothesis: There is no difference between the new intervention and the standard intervention.

  • Alternative Hypothesis: The new intervention is superior.
    The speaker engages the audience by prompting them to discuss the implications of Type I and Type II errors within the context of this study.

Validity Concerns

  • Internal Validity: The ability of a study to rule out alternative explanations and establish a causal relationship, represented through potential biases, sampling errors, and measurement inaccuracies.

  • External Validity: Refers to the generalization of research findings across different settings and populations.

Assignment and Methodology Discussion

As part of the course structure, students will critique a study on the quality and methodology used. The speaker states that many methodological weaknesses should be self-identified rather than repeating those specified by the authors being critiqued.

Qualitative Research Overview

Though the assignment primarily focuses on quantitative research, the conversation briefly touches upon qualitative research methodologies, delineating approaches like phenomenology, ethnography, and grounded theory, which guide research based on specific questions.

  • Trustworthiness in Qualitative Research: As outlined, trustworthiness includes strategies for establishing credibility, dependability, transferability, and confirmability. The speaker emphasizes the significance of documenting bias and participant representation through rigorous methodology and rich descriptions of findings.

Concluding Thoughts

  • The speaker expresses the importance of understanding both types of errors and validity issues while conducting research, particularly within the realm of psychological studies involving interventions. Encouragement is given towards practicing critiques to refine skills in analyses and apply theoretical knowledge effectively.

  • There is also a final nod towards reinforcing the need for clear assignments related to methodologies, highlighting the continuous learning curve in mastering research evaluations.