Null Hypothesis Significance Testing

What is your research question?

Looking for a relationship:

    What type of data?

        Continuous:

            Parametric assumptions met?

                Yes: Pearson’s r correlation

                            Example: Relationship between hours studies and exam score (%)

                            Test ran: Pearson’s r correlation between hours studies and exam score

                No: Spearman’s Rank correlation

                            Example: Relationship between class rank and self-rated stress                                             level (ranked 1-10).

                             Test ran: spearman’s rank correlation between class rank                                             position and stress ranking

        Categorical:

                Chi-squared test for association

                            Example: Association between smoking status (smoker or non)                                             and presence of chronic cough (yes/no)

                            Test ran: Chi-squared test of association between smoking status                                             and cough                         

Looking for a difference:

    What type of data? - always continuous

        Continuous:

            Number of groups / samples?

                Two:

                    Independent measures: Independent samples t-test

                                                                Ex.: difference in reaction time                                                                 (milliseconds) between caffeine group vs.                                                                 no-caffeine group

                                                                Test Ran: independent samples t-test                                                                 comparing reaction times between groups

                    Repeated measures: Paired samples t-test

                                                            Paired is measuring the same people twice,                                                             or deliberately matched pairs based on age,                                                             gender, twins, left vs right eye of same                                                             person, etc.

                                                            Ex.: difference in test scores (%) for the                                                             same students before and after tutoring

                                                            Test ran: paired samples t test comparing pre                                                             and post tutoring scores

                Three or more:

                    How many independent variables?

                        1 IV:

                         Independent measures: One-way independent ANOVA

                                                                   Ex.: difference in exam scores (%)                                                                     across 3 teaching methods; lecture,                                                                     video, interactive.     

                                                                   Test ran: one-way independent ANOVA                                                                     on teaching method                    

                         Repeated measures: One-way repeated measures ANOVA

                                                                Ex.: difference in reaction time (ms) for the                                                                 same participants under 3 noise levels;                                                                 quiet, moderate, loud.

                                                                Test ran: one-way repeated measures                                                                  ANOVA on noise level

                        2 IVs:

                        Independent measures: Two-way independent ANOVA

                                                                Ex: effect of teaching methods (lecture vs                                                                 interactive) and study environment (home                                                                 vs library) on exam score (%)

                                                                Test ran: two-way independent ANOVA                                                                 with teaching method x study environment

                        Repeated measures: Two-way repeated measures ANOVA

                                                            Ex: effect of drug dose (low vs high) and time                                                             (morning vs evening) on pain rating (0-10)                                                             for the same participants

                                                            Test ran: two-way repeated ANOVA on dose                                                             x time

                        2+ IVs:

                            Factorial ANOVA

                                Ex: effects of sleep amount (low vs high), caffeine (yes vs no)                                        and study method (notes vs flashcards) on exam score (%)

                                test ran: factorial ANOVA with sleep x caffeine x study method

Looking to predict outcome:

    What type of data? - always continuous

        Continuous:

            How many independent variables:

                1 IV: Regression

                            Ex: predicting exam score (%) from hours studied

                            Test ran: simple linear regression on hours studied predicting                              exam score

                2 IVs: Multiple Regression

                            Ex: predicting exam score (%) from hours studied and sleep                             duration

                            Test ran: multiple regression with hours studied and sleep                             predicting exam score