Understanding Experimental Analyses and Statistical Tests

2.4 Experimental Analyses

Overview of Experimental Analyses

  • Objective of Experimental Analysis

    • To understand how the results of experiments are analyzed.

    • Unlike correlations, true experiments provide solid evidence about cause and effect.

True Experiments

  • True experiments rely on random assignment to groups:

    • Ensures each experimental condition is equal before the experiment begins.

    • The independent variable is the only possible difference in outcomes between competing groups.

Example Experiment: Multitasking Ability

  • Scenario of Experiment:

    • The hypothesis is that spending more time on social media during a lecture (independent variable) influences exam scores (dependent variable).

  • Design:

    • Recruit 100 students.

    • Randomly assign:

    • 50 students browse social media for 30 minutes during the lecture.

    • 50 students browse social media for 10 minutes during the lecture.

  • Each student is tested on the lecture content at the end of the class.

Data Analysis

  • Data Analysis Process:

    • Upon gathering data, statistical analysis is necessary.

    • The analysis compares two groups using the t test statistic.

The t Test

  • Definition:

    • The t test is a statistical analysis that compares averages and ranges of two groups to determine if they differ significantly.

  • Historical Context:

    • Developed by William Sealy Gossett, a chemist and statistician, while working as the head brewer at the Guinness Brewery in Dublin, Ireland.

    • Created to monitor quality between morning and afternoon batches of Guinness without quality sampling all kegs.

  • Key Components of the t Test:

    • Random Samples: Must avoid bias through random selection.

    • Sample Size: The number of samples must be sufficient to represent all beer casks produced during a run.

  • Publication Under Anonymous Pen Name:

    • Gossett published his findings under the pseudonym Student, which is why the t test is sometimes referred to as Student's t test.

  • Alternative Naming:

    • Some suggest it would be more appropriate to refer to it as Gossett's t test.

Application to Multitasking Experiment

  • In the multitasking experiment:

    • The scores of participants are displayed in visual form (e.g., Figure 2.5).

    • Group comparisons:

    • Green Group (10 minutes): Performed better than the Blue Group (30 minutes).

Comparing Three or More Groups: Analysis of Variance (ANOVA)

  • Context for Use:

    • If there are more than two groups to compare, such as in an extended version of the multitasking experiment.

  • Example Setup:

    • Five groups:

    • Group 1: No social media (control group).

    • Group 2: 10 minutes on social media.

    • Group 3: 20 minutes on social media.

    • Group 4: 30 minutes on social media.

    • Group 5: Entire class duration on social media.

  • Analysis of Variance (ANOVA):

    • ANOVA compares three or more groups, determining if one group significantly differs from the others.

    • If differences are found, further investigation is needed to identify which groups are significant outliers.