P-Values, Statistical Significance, and Correlation Interpretation

Fundamentals of P-Values and Statistical Significance

• P-Value Definition: A $p$-value serves as a direct quantitative measure of error in hypothesis testing and experimental analysis.
• Acceptable Error Threshold: The maximum acceptable level of error to establish statistical significance is $0.05$, which corresponds to a $5\%$ error rate.
  • A $p$-value of $0.05$ indicates that there is a statistically significant difference between two comparison groups.
  • Any error rate greater than $5\%$ ($p > 0.05$) exceeds the maximum tolerable threshold for declaring a meaningful effect or difference.
• Directionality Rule for P-Values: Unlike standard mathematical metrics where larger values are preferred, $p$-values operate on an inverse rule where smaller values indicate better statistical outcome:
  • Smaller $p$-values represent lower probability of error and stronger evidence of a statistically significant difference.
  • Larger $p$-values represent higher error rates and a lack of statistical significance.
• Values Lacking Statistical Significance ($p > 0.05$):
  • $0.06$
  • $0.07$
  • $0.08$
  • $0.09$
  • $0.10$
  • $0.20$
  • $0.30$
  • $0.40$
  • $0.50$
  • $0.60$
  • $0.80$
  • $0.90$
• Values Demonstrating Statistical Significance ($p \le 0.05$):
  • $0.05$
  • $0.04$
  • $0.03$
  • $0.02$
  • $0.01$
  • $0.005$
  • $0.001$

Interpreting Correlations and Significance Cutoffs

• Standard Decision Convention: By universal academic and statistical convention, an experimental outcome or correlation is statistically significant if and only if $p \le 0.05$.
• Step-by-Step Correlation Analysis Workflow:
  1. Interpret the relationship direction (positive or negative) and strength (weak, moderate, or strong) using the correlation coefficient ($r$).
  2. Evaluate the accompanying probability value ($p$-value) against the $0.05$ threshold to determine if the relationship is statistically significant.
• Case Study 1: Bipolar No More:
  • Observed Relationship: There is a strong, negative correlation between administration of the treatment Bipolar No More and a decrease in symptoms of bipolar disorder.
  • Significance Assessment: Statistically significant.
  • Justification: The probability value falls directly at the threshold where $p \le 0.05$.
• Case Study 2: Lithium:
  • Observed Relationship: Data yields a correlation coefficient of $r = -0.60$, indicating a moderate, negative relationship between taking lithium and a decrease in bipolar disorder symptoms.
  • Significance Assessment: Not statistically significant.
  • Justification: The probability value is greater than $0.05$ ($p > 0.05$).
• Testing and Coursework Guidelines:
  • Although $p > 0.05$ represents an error rate greater than $5\%$, formal evaluations (quizzes, assignments, and activities) require explicitly writing that the $p$-value is "greater than $0.05$" (or $p > 0.05$) rather than stating a percentage.

Application and Presentation of Probability Values

• Analytical Expectations:
  • Computation or hand calculation of probability values is not required in standard introductory interpretation workflows; probability values are provided directly for evaluation.
  • The primary task is accurate conceptual interpretation of the provided $p$-value.
• Representation in Scientific Literature:
  • In academic research publications and empirical papers, $p$-values are commonly presented in one of two formats:
  1. As an exact equality statement (e.g., $p = 0.04$).
  2. As an inequality relative to alpha thresholds (e.g., $p < 0.05$ or $p \le 0.05$).

Questions & Discussion

• Inquiry Regarding Scope (Todd):
  • Question: Is the material covered up to this point inclusive of all $t$-test procedures?
  • Response: Coverage of $t$-tests is deferred to the beginning of the next class session because detailed introduction and analysis of $t$-tests cannot be adequately completed in a brief 3-minute window remaining at the end of class (at 2:16 PM).