Stats: ANCOVA

What are the basics mechanics of ANCOVA?

What are the basics mechanics of ANCOVA?

1. We run an ANOVA (any kind)

2. We include another variable that accounts for some of the variance in our DV (covariate)

What type of sampling is best with ANCOVA?

Random sampling and assignment

What are the three primary uses of ANCOVA?

-To increase power by reducing error variance in experimental studies (w random assignment) <- good one.

-To adjust for mismatch on nuisance variable(s) in non-experimental studies (w/o random assignment)<- bad one.

-Step-down analyses following MANOVA

How are the affects of IVs assessed using ANCOVA?

The effects of IVs are assessed holding covariates (CVs) constant

What are the basic requirements for ANCOVA?

1: CV should correlate with DV (significant DV-CV association)

2: Groups should not differ on CV (non-significant IV-CV association)

3: Groups should not differ on CV-DV association (non-significant IV x CV interaction)

What is a potential covariate?

Any variable that is significantly correlated with the DV (linear)

What should the CV be independent of in ANCOVA?

The CV should be independent of the IV

What affects your choice of CV's?

- Ideal is small number of orthogonal (independent) CVs

- only include CVs that will explain a significant amount of unique variance

- CV should be reliable

What affect does CSs have on the error term?

Each covariate reduces error df by 1.

Why should we only include CVs that will explain a significant amount of unique variance in the DV?

there is a tradeoff between increased statistical power and lower error df, which will reduce the F.

If CV is measured post-treatment, what is the problem?

If CV is measured post-treatment, it might be affected by the treatment:

- CV scores might correlate with change in the DV

- CV adjustment would remove a part of the treatment effect (& lower F-statistic for the IV)

When do you measure the CV?`

CV must be measured before manipulation/treatment to ensure no “contamination” (independence of IV and CV)

How many CVs can we have?

For small group sizes, maximum 2-3 CVs

What is the equation for CVs?

CV <.1 * (N - J +1)

(N = total sample size)

(J = number of groups)

What does ANCOVA do?

ANCOVA removes this DV variation explained by the covariate from the error term (variance unexplained by IV)

What does ANCOVA do with means?

ANCOVA adjusts DV means to what they would be if all participants scored the same on the CV

What should we keep in mind regarding means and ANCOVA?

Adjusted means should always be treated with caution

What are the assumptions of ANCOVA?

- Absence of outliers

- Homogeneity of variance for DV and CVs

- Eliminate highly correlated covariates (multicollinearity)

- Relationships between DVs and CVs, and between CVs, should be linear

- CV is independent of IV

Which violations means that you won't be able to use ANCOVA?

- Significant CV-IV association (group difference on CV)

- Significant CV x IV interaction (heterogeneity of regression)

What are the three alternatives to ANCOVA?

- PSM

- blocking

- multiple regression

What do you do in blocking?

Turn CV into another IV

What are the advantages of blocking?

- None of the assumptions of ANCOVA or within-groups ANOVA

- The relationship between CV and DV need not be linear

- Can be expanded to multiple CVs

What are the disadvantages of blocking?

Converting a continuous variable to a categorical variable results in loss of information

Why is ANCOVA best to use for random assignment?

Any group differences in the CV will be due to chance

Why is ANCOVA not best to use for non-random assignment?

Group differences in the CV are likely to be meaningful

If a sample size is small, does it matter if any of homogeneity results are significant?

“If sample sizes are relatively equal (within a ratio of 4:1 or less for largest to smallest sample size) Fmax [the ratio of largest cell variance to the smallest] as great as 10 is acceptable.”

What would be the problem if the CV is not reliable?

The more unreliable a measure is, the less well it will correlate with the DV, hence the less variance will probably be removed from the error term

What aspects of an analysis would give us confidence to run ANCOVA?

• random assignment

• measurement of the CV before intervention

 If homogeneity of regression holds, what pattern would we expect to see in the data?

If homogeneity of regression holds, we would expect that the regression slopes (least squares lines) denoting the association between each DV and CV would be roughly similar in all the groups

What does it mean when the F-statistic goes up?

This reflects the increase in statistical power afforded by the inclusion of the CV.

What are the changes to look forward between an ANOVA and ANCOVA stats?

• The degrees of freedom (dfs)

• the F-statistic

Why should CVs be reliable?

The more unreliable a measure is, the less well it will correlate with the DV, hence the less variance will probably be removed from the error term

What important extra assumption needs to be met if we want to carry out ANCOVAs ?

Homogeneity of regression slopes. This is the idea that the relationship between the CV and the DV (as quantified by the regression slope) is relatively similar (homogeneous) in the different groups (drug and placebo).

If homogeneity of regression holds, what pattern would we expect to see in these data?

If homogeneity of regression holds, we would expect that the regression slopes (least squares lines) denoting the association between each memtest and verbiq would be roughly similar in the drug and placebo groups.

How do we statistically evaluate whether these data meet this assumption?

This assumption can be tested by determining whether there is an interaction between the IV (treatment) and CV (verbiq) on the DVs (the three memory tests).