Module 8: Longitudinal Designs

Explain the three conditions to test for causality using survey data*

1. The constructs need to covary

2. There must be temporal precedence

3. No confounding factors

What is meant by ‘constructs need to covary’?

The constructs and variables of interest must be associated in some way.

Think: There is no point in running an experiment or survey on the correlation between the number of barbecues sold and the internal temperature of my drink bottle.

Temporal Precedence

Assumes the thing we believe causes an outcome must precede the effect it is supposed to cause.

• The cause must come before the effect

• Often described as a process within a time frame

Think: If pushing a button makes disco lights and music turn on, this is temporal precedence. If the music and lights turned on before pressing the button, then there would be some other cause that is not the button.

Confounding Factors

Alternative explanations that are not considered in experiment or survey design.

Think: Illusory correlations - pirates are not decreasing because of climate change despite an association being observed

What type of research can never directly empirically test temporal precedence and why

Cross-sectional

• Cross sectional designs test at one time point, therefore there is no ‘before and after’ element that shows the direction of associations.

• Instead the direction is inferred by allocating a DV (criterion) and IV (predictor).

• The experiment itself cannot determine the direction of the association, but theory and previous findings are used to justify the direction.

Think: You do a cross-sectional survey to determine if people with dogs are happier. This means they fill out a questionnaire and you get data on their self-reported happiness and whether they own a dog (y/n). From this info alone, you cannot tell if they are happy because they have a dog, or if they got a dog because they are happy. You would have to look at other research to justify which direction you choose to report your findings from.

What is the ‘gold standard’ design in longitudinal research?

Panel designs

Think: Talking the panel of highly esteemed people

Panel Designs

When the same participants complete the same questionnaire over multiple time points.

• Ensures both the predictor and criterion variable are measured at multiple time points.

Stability

The degree of consistency in scores, means or rank orders from one time point to another.

Change

The degree of fluctuation in scores, means or rank orders from on time point to another.

What are measures of stability and change based on in longitudinal research?

The initial measurement - baseline

What do measurement of change and stability in longitudinal research help us determine?

In what ways and how much do things stay the sane over time vs how much do they change?

2 directions of associations

• Uni-directional

• Bi-directional

Uni-directional Relationship

• There is a clear direction in the relationship between the predictor and criterion variable

• A uni-directional relationship in a well-designed longitudinal study provides support for temporal precedence.

Bi-directional Relationship

• Occurs when the predictor variable is related to the criterion variable, and the criterion variable is related to the predictor variable.

What type of relationship determines temporal precedence in longitudinal designs?

Uni-directional relationships

• Bi-directional suggests that both variables influence one another, so a cause cannot be determined because they are BOTH causal

What are the 4 statistical designs are used in longitudinal survey research?

• Simplex models

• Longitudinal Correlations

• Residualised longitudinal regression

• Cross-lagged models

Simplex Models

Involve regressing a variable on itself across time

• AKA Autoregressive designs

• The measurement of a variable at time 1 should predict time 2 (stability)

• Allows researchers to explore the stability and change in one construct

Think: If I have recorded an anxiety score of 10/10 every month for 6 months in longitudinal study, the model would suggest that next month I will also record a score of 10.

Why are simplex designs also called autoregressive designs?

Because the values of a scale are automatically regressed onto the same scale.

• Means that we use data from each time point to predict scores on the same variable at the next time point using a regression model.

Think: If I have recorded an anxiety score of 10/10 every month for 6 months in longitudinal study, the model would suggest that next month I will also record a score of 10.

What does a perfect beta score of 1 between measures of a DV between 2 time points suggest?

There is a perfect association between the two time points and the individual’s relative standings on the construct have not changed.

• The rank-order of a participant has not changed (still coming first/last)

• HIGH STABILITY

Think: I was a depressed little bitch last month, I’m still a depressed little bitch this month. Things are not getting beta

What is the autoregressive coefficient?

Beta weight

What beta weight indicates a perfect association?

1

What does a large autoregressive coefficient indicate?

• Individuals do not change over time

• Individuals uniformly increase or decrease over time

What is meant by the explanation of a large autoregressive coefficient demonstrating that individuals do not change over time?

Every participant’s scores are relatively stable between two time points.

What is meant by the explanation of a large autoregressive coefficient demonstrating that individuals uniformly increase or decrease over time?

There is change in individual scores between time points, however this change pattern is observed across ALL individuals.

Think: You’re measuring depression. At time 1, everyone is happy, At time 2, everyone is still happy hooray. Between time 2 and 3, a tsunami hits. At time 3, everyone’s happiness scores plummet. HUGE change in scores between time points, but it is consistent across the board.

(all scores increased by 2 in the example)

Intra-individual stability

A person’s scores do not change between time periods.

Inter-individual stability

The rank order of participants in a sample do not change between time periods.

Think: There is huge variability between individual scores between time points for IQ, but the person who was the dumbest at the beginning remains the dumbest in comparison to the others over time, even though their scores are changing.

In a simplex design, what tells us we have stability?

Rank order (not individual stability)

What does a beta score of 0 between two time points indicate?

An individuals relative standings on the construct have changed dramatically.

• Indicates a rank-order change in participants score between T1 and T2

Think: When I always go from first to last in kahoot 😔

Simplex Design: What is this indicating?

High stability, low change

• Construct has strong relationship with itself and demonstrates temporal stability

Circles representing time 1 and 2

Simplex Design: What is this indicating?

Low stability, high change

• The construct has a weak relationship with itself and high temporal change.

Circles representing time 1 and 2

What do these results suggest?

The children’s vocabulary scores demonstrated high temporal stability, meaning the relative standing (ranking) of children’s vocabulary scores stayed the same over time.

However you can’t conclude the reasoning for this. Explanations could be either:

• Children’s vocabulary doesn’t change across primary school.

• Every child’s vocabulary increases uniformly

How can you determine whether a high autoregressive coefficient is due to stability of scores across time (no change) or uniform changes across the group?

Compare group averages across time periods.

• An increase/decrease in averages suggests there are uniform changes across the group over time.

• If averages are stable across time, it means the scores are not changing over time - indicating no growth/decline

Think: With the child vocab score example, the autoregressive coefficient was very strong, but we couldn’t tell whether that was because every person’s vocab store stayed the same from year to year, or if it was because the whole sample shows the same patterns in growth/decline. By comparing the group avg at each time period (image), we can see that there were uniform changes in scores, in that the avg vocab score increased.

Simplex Designs: Limitations of using average scores across time periods to evaluate the autoregressive coefficient.

Cannot distinguish whether changes are due to uniform growth (i.e. everyone’s scores are changing at a constant rate) or if the average change is due to intra-individual differences (i.e. one person’s scores increased dramatically which brought the averages up)

Longitudinal Correlations

Examine the relationship between the IV at time 1 and the DV at time 2

• Cannot determine/measure temporal precedence alone

How can you determine temporal precedence in a longitudinal design?

Run two separate longitudinal correlations

• If the relationship between IV at time 1 and DV at time 2 is significant, and the relationship between DV at time 1 with IV at time 2 is not significant, it can be argued that temporal precedence has been found.

• Not advocated for though!!! Bit of a mehhhh approach

If you run two separate longitudinal correlations on the same variables and both are found to be significant, what is the relationship between the IV and DV?

Bi-directional

Limitations of Longitudinal Correlation Designs

• Does not account for correlations between variables at each time point

  • don’t know the correlation between the IV and DV at time 1 vs time 2; maybe there’s a really strong correlation between times 1 and 2 but no correlation between 2 and 3

• Does not account for stability/change in a construct (unlike simplex)

What is the consequence of the limitations of longitudinal correlation designs?

You cannot rule out that the relationship between T1 and T2 is simply due to a cross-sectional relationship.

Longitudinal designs have been used to try and establish _______, but not _____ nor ______.

Temporal precedence, stability, change

Residualised Longitudinal Regressions

Combination of longitudinal correlation and simplex design

• Residualises the DV by entering the score of the DV at time 1 into the analysis

• Means the correlation between the DV across two time points is considered and then statistically removed.

Residualised Longitudinal Regression designs allow researchers to…

Predict change

Residualised Longitudinal Regressions: How does entering the DV score at time 1 allow researchers to predict change?

The unique variance remaining in the DV at time 2 reflects the change in the construct over time.

• The variance that is not explained by the same construct measured earlier

Think: If eating ice cream (IV) is found to make you fatter (DV), then we also need to consider whether being fat at time 1 predicts being fat at time 2. If the regression of the DV shows that being fat at time 1 predicts being fat at time 2 without the influence of icecream, then we can statistically remove this unique variance to see the isolated effect of icecream on body fat in a longitudinal design. This is a residualised longitudinal regression.

What has been highlighted and what design is this?

Residualised longitudinal regression

• Shows the variance explained in the DV at time 2 that is explained by the independent variable only - not the variance explained by the DV at time 1.

• Purely the change in DV at time 2 accounted for by the IV at time 1.

Which design should be used to answer this RQ:

How do we know that feeling connected with the community predicts change in wellbeing and not stability?

Residualised Longitudinal Regression

Strengths of residualised longitudinal regressions

• The correlations between variables at T! is statistically controlled for

• The stability of the DV is accounted for

• Allows researchers to predict change and find a longitudinal effect

Weakness of Residualised Longitudinal Designs

• Temporal precedence is still not identifiable because there is no test for bi-directional relationships.

Which research design determines the correlation between the same variable over time?

Simplex design

Which research design shows the correlation between an IV and DV over time?

Longitudinal design

Cross-Lagged Designs

The two measures serve as both the independent and dependent variable.

• Therefore, there is always more than one DV

• Basically combine two residualised longitudinal regressions into the same analysis

• Allows investigation of bi-directional effects

• Statistically removes the stability of each construct, isolating the effects of each variable on the other.

How many parameters to a two-wave cross-lagged model?

6

What does “two-waved” cross lagged model mean?

There are 2 time points.

State the 6 parameters in a two-waved cross-lagged model

1. Correlation between Var1 and Var 2 at time 1

2. Correlation between Var1 and Var2 at time 2

3. Stability of Var1 between time 1 and time 2

4. Stability of Var2 between time 1 and time 2

5. Cross-lag path: Var 2 at time 1 predicting change in Var1

6. Cross-lag path: Var 1 at time 1 predicting change in Var2

Think:

Cross means cross-lag

Flat means stable

Circle means correlation

If both cross-lags are significant in a two-waved cross-lagged model, what can we conclude?

There is a bi-directional relationship and temporal precedence cannot be inferred.

Which longitudinal designs can explore bidirectional relationships?

Cross-lagged model

Which longitudinal design only explore uni-directional relationships without consideration of change or stability?

Longitudinal correlation

Which design can predict change but can’t explore bi-directional relationships?

Residualised regression

Which designs only estimate stability and change?

Simplex

Which designs account for cross-sectional relationships?

• Residualised regression

• Cross-lagged model

Disadvantage of cross-lagged model

• Expensive to run

• Requires additional software as SPSS is too dumb to run it

***Table not a question***

***This is the answer***

What are the assumptions of longitudinal analyses?

• Inter-individual stability

• Consistent measurement (same measures; don’t change the questionnaire from time point to time point)

• Synchronicity

• Timeframe needs to be appropriate

• Other variables

The greater the temporal (time) distance between measurements…

The weaker the relationship

What happens if a change has to be made to a questionnaire at time point 3/5 due to validity problems?

You can only use results from the items that were included in all time points.

• Highlights why it is important to research and know the validity of your scales before you start

Synchronicity

The administration of questionnaires occur with the same interval between time periods for all participants.

When is assumption of inter-individual stability violated?

When one group of participants changes faster or slower relative to other participants.

Think: You’re measuring wealth growth over 10 years in a group of 18 year olds but 1 person’s parents just buy them everything and they have no expenses, so their wealth grows faster.

What is the relationship between time spent on social networking sites and wellbeing, and what assumption does this relationship speak to?

Greater social media is associated with higher sleep disturbances, which reduces wellbeing

• Other variable (3rd variable effect)

Describe the assumptions and other considerations in longitudinal research*

Demonstrate how to make decisions regarding the appropriate analysis to use to test various research questions*