Week 6 Factor analysis and independent component analysis.

What is the goal of factor analysis

The goal of Factor analysis is dimension reduction by identifying a smaller number of unobserved (latent) factors that explain the patterns of correlations between observed variables.

What are factors

Latent variables that explain correlation between observed variables.

explain the approach of factor analysis

1 center the data

2 construct the factor model

3 assuming all factors both common and unique are uncorrelated, we can derive the covariance forumla which seperates variance into shared and unique variance.

4 assuming the data is a multivariate normal distribution, we estimate the factor loadings and uniqueness using a maximimum likelihood function.

5 rotate teh factor loadings to a simpler more interpretable structure.

6 get factor scores (show how strongly each individual expresses a factor, new coordinates in the factor space for each individual).

Write down the factor model in its simple form and for the entire dataset. And write down the factor analysis covariance formula.

What are common factors

latent variables that influence two or more observed variables, explaining the correlations among them

what are factor loadings

show how strongly a factor influences an observed variable or if itis in matrix form how strongly every factor influencesevery observed variable. 

What are unique factors 

unique variable specific variance. in matrix form its diagonal. 

What is the key assumption necessary to derive teh covariance formula for FA

That all factors both common and unique are uncorrelated 

How do we estimate the factor loadings and uniqueness

maximum likelihood function

What does this maximium likelihood function do?

It finds the factor loadings and unique variances that make the observed covariance matrix most probable under the assumed factor model. 

What key assumption is needed for this to work?

The original data is a multivariate normally (gaussian) distributed.

Why do we rotate the factor loadings?

To get a simpler, more interpretable structure, where each variable loads strongly on one factor and weakly on others.

What are factor scores? What is the differene with factor loadings?

Factor loadings show how strongly each variable is influenced by each factor, while factor scores indicate how strongly each individual expresses each factor (new coordinates in the factor space for each individual).

three common ways to calculate factor scores

Bartlett Method

Thurstone’s factor scores

McDonald’s factor scores

How can the fit of the final model be tested?

The fit of the model to data can be tested using a large sample chi-square goodness of fit test. 

Fit indices can be calculated (RMSEA, where Lower values mean better fit.)

What is a problem that can arise when checking the fit of a modelwith the first method? 

the power to reject a model close to the true data generating process, increases with sample size. 

How does the large sample chisquare goodness of fit test work?

It compares the observed values to the values predicted by the model, and if the differences are too large (given sampling variability), the model is considered a poor fit.

what does it mean if uniqueness is close to zero or one

if close to zero, the factor explains most of the variance, if close to one the variable is mostly specific variance or noise.

Should factor scores be uncorrlelated?

yes or at least close to zero like 0.12

What is ica? 

ICA is a statistical method used to separate a set of mixed signals into underlying, original source signals(factors).

What is the key assumption of ICA?

The source signals (factors) are statistically independent not just uncorrelated.

What is the goal?

To look for a transformation of your observed data so that the resulting components are as independent as possible.

explain the differences between ICA, PCA, and FA

• FA: A latent variable model that explains shared covariance and includes unique error terms.

• PCA: A linear decomposition that finds orthogonal components capturing maximum variance.

• ICA: A signal separation method that finds components that are statistically independent.

How is rotation used for FA and for ICA?

FA: rotation is used to improve interpretability (e.g., varimax, oblimin).

ICA: Rotation is essential — ICA solves for a rotation that maximizes independence.