Psych 300B: Final Exam Review (Violated Assumptions)

What are the 5 assumptions of the random sampling model of hypothesis testing

1. Data are scores

2. Participants randomly sampled from population

3. Dependent variable is normally distributed in the population

4. Homogeneity of variance is met

5. Each group, condition or cell has nk ≥ 7

How robust is the t-test to violation of assumptions

Moderately robust except for when homogeneity is violated

Especially apparent when larger variance associated with smaller sample size

How robust is the F-test to violation of assumptions

Less robust to violations of assumptions, need homogeneity of variance and n within a 1:1.5 ratio (smaller for factorial design)

How well is the assumption of score data of hypothesis testing met

Not that well, many researchers use t and F tests to analyze ordinal data

How well is the assumption of sampling of hypothesis testing met

Poorly, as true random sampling is rarely used as it is technically illieagle

How well is the assumption of DV’s being normally distributed of hypothesis testing met

Hardly ever, Micceri demonstrated that majority of research’s population distributions are not normal

How well is the assumption of sampling distribution being normal of hypothesis testing met

Cannot say, as we cannot prove the sampling distribution to be any specific shape

Know it is kurtotic when nk ≥ 25-30, but not always practical/realistic to have so many participants

How do we fix the violation of no true random sampling

Use one of the other random sampling methods, and randomly assign participants to each condition

If violated cannot generalize outcome from sample to population

How do we fix the violation of homogeneity of variance

HINT: different for t and F tests

Test for violations using Levene’s

t-test - use Welch’s to reconfigure sampling distribution

F-test - use Welch’s or Brown-Forsyth to re-run anova with reconfigure sampling distributions (use Dunnett’s for probe)

How do we fix the violation of nk ≥ 7

No fix, we must use a different model of hypothesis testing

How do we fix the violation of normality of the population

Do data transformations, do this by applying transformation to all scores in dataset to make distribution more kurtotic but maintain ordinal relationship

What are the two major reasons we have non-normality in a population

Ceiling/floor effect - leads to skew in sampling distribution

Outliers in data - produces large values for variance and estimated standard error (so small observed test statistic)

How do you combat outliers in data

Typically trim the data, but it is important to be transparent and objective as to why this occurs

What are the two ways to fix skewed data without outliers

1. Mathematical procedures applied to all scores (data remain scores)

2. Transforms scores to lower scale of measurement (apply non-parametric test)

What are the mathematical fixes that can be used if we have a moderate, strong or extreme skew in our data

Moderate - square-root transformation

Strong - log linear transformation

Extreme - inverse transformation

Note - if data is negative for the first two, must add constant to all terms to remove negativity

How do we transform data to a lower scale of measurement

Convert scores to ranks (preferred) or tallies

This method is less preferred over mathematical fix as we lose quantitative information

What are the 4 tests used when doing rank transformations

Related samples - Wilcoxon signed-rank test

Independent samples (k = 2) - Wilcoxon rank-sum test, also called Mann-Whitney U test

Independent samples (k > 2) - Kruskal-Wallis H test

Correlation - Friedman’s rank test

What are the 2 tests used when doing tallie transformations

Related samples - Sign test

Independent samples (k ≥ 2) - Median split

What are the 3 steps to converting to ranks

1. List all values low to high and assign a rank based on score

2. Reorganize data back into groups

3. Apply appropriate rank test

Mann-Whitney U test

Mostly applied when the outcome is not normally distributed and samples are small

3 steps to compute the U statistic for a Mann-Whitney U test

1. Convert scores to ranks and compute summed ranks (R0 and R1)

2. Compute U statistic for each group (U0 = n0n1 + (n0(n0 + 1) / 2) - R0 and U1 = n0n1 + (n1(n1 + 1) / 2) - R1)

3. Smaller U statistic is compared to critical values of the Mann-Whitney U within the appropriate table

What is the Random assignment model

Alternative form of hypothesis testing that can be used when extreme violations of parametric assumptions occur

Can also be used whenever RS model could be used

4 major highlights of the RA model

1. Sample is not required to represent the population

2. Data are not required to meet parametric assumptions

3. Data are analyzed using computer -insensitive randomization tests

4. The statistical outcome gives an exact probability value for the statistical test

What is the major difference between the RS and RA model of hypothesis testing

The sampling distribution is empirical in the RA model, and theoretical in the RS model, so we get an exact p-value for RA model

What are the 3 major assumptions of the RA model of hypothesis testing

1. Data are scores, or appropriate to statistical test applied

2. n ≥ 3 for each group

3. Independence in scores achieved by random assignment

Sampling distribution in the RA model of hypothesis testing

Null distribution of all possible outcomes given actual values that occurred in dataset

Generate a posteriori from dataset and is unique to each research study

What do the x and y axis of the sampling distribution represent in the RA model of hypothesis testing

x-axis - difference of means

y-axis - frequency of each difference of means

What is the shape of the sampling distribution in the RA model of hypothesis testing

Varies with each distribution, and is not relevant to analysis

How do we build our sampling distribution in the RA model of hypothesis testing

List all possible outcomes of our data and then build a frequency distribution table from said outcomes

The crf column is the actual area under the curve (exact probability value)

4 problems with t-tests (RS model of hypothesis testing)

1. Increased type 1 and 2 error with small n

2. Not robust to severe violations

3. Populations rarely normal

4. rarely meet all assumptions

4 problem with randomization tests

1. Need more software for these tests

2. Not widely understood by psychologists and other researchers

3. Not applicable to complex designs

4. Hard to publish research (due to lack of acceptance/understanding)

What is the bottom line regarding randomization and t-tests

When assumptions are not severely violated or sample has large n:

Either test gives about same outcome

Differences occur when severe violations occur or when experiment has a small n