Validity

What is validity?

The extent to which conclusions can be deemed accurate and trustworthy

What questions do you ask in a critical review?

• Is there a clear hypothesis?

• Is the methodology appropriate?

• Has the data been analysed correctly?

• Have the findings been interpreted correctly?

• Are there any other explanations?

How can validity be compromised?

• Biased sampling

• History effects - external events influences the results

• Maturation effects - participants naturally changing

• Sample attrition - participants dropping out

• Testing effects - participants do a test that affects a future test

How can you avoid validity being compromised?

• Randomisation and suitable sampling - reduces bias

• Sample size - needs to represent the population and a larger sample size will dilute the impact of outliers.

• Blind or double-blind testing - can’t try to predict what should be happening

• Follow standardised methods where possible

What is internal validity?

Extent that you can demonstrate a casual relationship between the independent and dependent variables

• Control groups often used

What is external validity?

Extent that findings can be generalised to the wider population

• Can results be replicated in diff settings

• Population validity - a study has high PV if you can generalise the findings from the participants to the wider pop

• Ecological validity - a study has high EV if you can generalise the finding from the study setting to every day life.

What is cross sectional design?

• Data collected from each individual at one time point.

• Used to examine relationships between two or more variables

• Establish the presence and extent of a single variable

What are the advantages and disadvantages of cross sectional design?

What is longitudinal design?

• Data collected from same individual at more than one time point.

• Determine if a variable at one point in time can predict a different variable occurring at a later point in time

• Examine change in one variable at different time points

What are the advantages and disadvantages of longitudinal design?

What is successive independent samples design?

• Combination of cross-sectional and longitudinal designs.

• Examines changes in the variable of interest over time within a population

What are the advantages and disadvantages of successive independent samples design?

What are the advantages and disadvantages of postal survey, face-face interview, phone and online survey as a survey method?

What are the 2 sampling methods?

• Probability based - more representative

• Non-probability based - less chance of minimising bias

What are examples of probability based sampling?

• Simple random sampling - larger sample sizes preferred

• Systematic sampling - every 4th person

• Stratified sampling - split into groups then randomised

• Cluster sampling - groups of the sample

What are examples of non-probability based sampling?

• Quota sampling

• Snowball sampling

• Convenience sampling

What is quota sampling?

• Sample until quota is full

• Quota is determined by the population and distribution at each strata

• Large risk of hidden bias

What is snowball sampling?

• Snowball sampling is very useful when your population is rare.

• Start with one individual. Hopefully they can identify others in the same population.

• The process builds until you have sufficient numbers.

• Assumptions and bias will be present in this type of study

What is convenience sampling?

• Uses the most easily accessible participants

• Limitations should be identified and comparisons with other, similar work made to add confidence to any conclusions

• Avoid if possible

What is the diff types of test validity?

• Face validity

• Construct validity

• Content validity

• Criterion validity

What is face validity?

How accurately a test measures what it should

What is construct validity?

If a test has face validity and measures the intended parameters and variables need to be defined

• Divergent Validity - correlations are not seen where they would not expect to be seen.

• Development or experimental changes - measurements may change over time as expected

• Factor Structure* - All factors which are expected to be present are.

What is content validity?

It measures every aspect of the construct it claims to measure

• Can only be applied to narrow constructs

What is criterion validity?

Test can be related to a theoretically predicted outcome

• Concurrent or predictive validiity

What is reliability?

How repeatable findings are

• A reliable test is not always valid but a valid test is always reliable

What is test-retest reliability?

Repeated testing with a requirement for consistent results.

• Test the same participant on two separate occasions and correlate the scores.

• If the correlation is over 0.8 then this shows acceptable test-retest reliability.