lecture recording on 14 February 2025 at 08.51.53 AM

Statistical Testing in Research

• Statistical testing is a method used to determine relationships between variables.

• Essential for supporting the inclusion of specific features in statistical or machine learning models to predict responses.

Research Questions and Hypotheses

• Definition: A research question guides the investigation into a dataset.

• Research questions define what the researcher is interested in answering.

• Examples of Good Research Questions:

  • Are there patterns in MRI data that can specify prostate cancer?

  • How does parental education level impact childhood obesity rates in Phoenix?

  • What is the relationship between physical activity levels and childhood obesity?

• Characteristics of Good Research Questions:

  • Clarity: Clear enough to understand its purpose.

  • Focus: Narrow enough to be thoroughly answered.

  • Specificity: Identifies variables of interest.

Hypothesis Development

• Well-constructed hypotheses are predictions about the relationship between variables, formatted as "If-Then" statements.

• A hypothesis should be specific and testable.

• Example of a hypothesis: As parental education increases, childhood obesity rates decrease.

• Emphasis on falsification: The aim of research is to attempt to disprove the hypothesis rather than confirm it.

Null Hypothesis vs. Alternative Hypothesis

• Definitions:

  • Null Hypothesis (H0): There is no effect or relationship.

  • Alternative Hypothesis (H1): There is an effect or relationship.

• The testing framework typically assumes the null hypothesis is true unless evidence suggests otherwise.

• Example of null hypothesis: There is no relationship between parental education and childhood obesity rates.

• Example of alternative hypothesis: Higher parental education correlates with lower childhood obesity rates.

Importance of Falsification

• The testing paradigm aims to reject the null hypothesis rather than prove the alternative.

• It is critical to never claim acceptance of the null hypothesis but to either reject or fail to reject it.

• Common Misunderstandings: Absence of evidence does not confirm absence of the phenomenon.

Error Types in Hypothesis Testing

• Type I Error: Rejecting the null hypothesis when it is actually true (false positive).

• Significance Level (Alpha): Determines the probability threshold for rejecting H0.

• Common alpha levels: 0.05 (5% chance of a Type I error).

Understanding P-Values

• Definition: The probability of obtaining results as extreme as the observed data, given that the null hypothesis is true.

• • Lower p-values suggest less likelihood of a Type I error.

• The process involves applying the p-value to the significance level to decide on rejecting H0.

Application of Statistical Concepts

• Example: Investigating loot boxes in gaming:

  • Collect data on obtained items from loot boxes and compare it against expected outcomes (e.g., a 50% drop rate).

  • Determine if the observed drop supports or rejects the claim of fairness in loot distributions.

Conclusion

• Statistical testing holds significant value in validating research questions and hypotheses.

• The aim is not to confirm preconceived notions, but to rigorously test and gather evidence.

Statistical Testing in Research

Statistical testing is a critical method in research that helps to determine the existence and nature of relationships between multiple variables. This process is essential for supporting the inclusion of specific features in statistical or machine learning models, which aim to predict responses based on data. The ability to conduct thorough statistical testing underpins the validity of research outcomes and helps in making data-driven decisions.

Research Questions and Hypotheses

Definition:

A research question serves as a guide for the investigation into a dataset, honing in on specific areas of interest that require exploration. These questions are foundational, allowing researchers to formulate objectives and methodologies for their studies.

Examples of Good Research Questions:

• Are there patterns in MRI data that can specify the presence of prostate cancer?

• How does parental education level impact childhood obesity rates in urban areas like Phoenix?

• What is the quantitative relationship between physical activity levels and the prevalence of childhood obesity among school-aged children?

Characteristics of Good Research Questions:

• Clarity: The purpose of the question should be easily understandable to ensure that the research focus is maintained.

• Focus: The question should be narrow and specific enough to allow for thorough investigation and conclusive answers.

• Specificity: The question must clearly identify the variables of interest in the study to facilitate a targeted analysis.

Hypothesis Development

Hypotheses are well-constructed predictions about the expected relationship between variables, usually formatted as "If-Then" statements. This structure allows researchers to make clear predictions based on theoretical or empirical foundations. An example of a strong hypothesis is: "As parental education increases, childhood obesity rates decrease."

Emphasis on Falsification:

The ultimate goal in research is not to prove a hypothesis correct but rather to attempt to disprove it. This approach ensures that the research process is rigorous and objective.

Null Hypothesis vs. Alternative Hypothesis

Definitions:

• Null Hypothesis (H0): Proposes that there is no effect or relationship between the variables in question.

• Alternative Hypothesis (H1): Suggests that there is a significant effect or relationship present.

The testing framework typically operates under the assumption that the null hypothesis is true, unless strong evidence to the contrary emerges. An example of a null hypothesis might be: "There is no statistically significant relationship between parental education and childhood obesity rates in a community." In contrast, the alternative hypothesis could state: "Higher parental education correlates with lower childhood obesity rates among children."

Importance of Falsification

In statistical testing, the aim is to reject the null hypothesis rather than to confirm the alternative hypothesis. It is crucial to avoid claiming the acceptance of the null hypothesis; instead, outcomes can lead to either a rejection or a failure to reject it. One common misconception is that the absence of evidence in support of the null hypothesis confirms its validity, which is not the case.

Error Types in Hypothesis Testing

• Type I Error: This occurs when the null hypothesis is incorrectly rejected, leading to a false positive result. The significance level (alpha) indicates the probability of making a Type I error, with common thresholds set at 0.05, representing a 5% chance of incorrect rejection.

Understanding P-Values

Definition:

A p-value is defined as the probability of obtaining results as extreme as those observed in the sample, assuming that the null hypothesis is true.

Lower p-values indicate a reduced likelihood of making a Type I error, guiding researchers in their decision-making process regarding the rejection of the null hypothesis. The interaction between the p-value and the significance level determines the outcome of the hypothesis test.

Application of Statistical Concepts

Example:

Consider the investigation into loot boxes in gaming.

• Researchers would collect data on items obtained from loot boxes and compare it to the expected outcomes, such as a predefined 50% drop rate for certain items.

• The goal would be to ascertain whether the observed outcomes support or reject the claims of fairness in the loot distributions, which could have significant implications for regulatory practices in the gaming industry.

Conclusion

Statistical testing is indispensable in validating research questions and hypotheses. Rather than merely confirming preconceived notions, its aim is to rigorously test and gather empirical evidence. Effective statistical testing not only enhances the credibility of research findings but also contributes profoundly to the scientific body of knowledge.