Statistics

Validity

The right quantity being measured

• Accuracy

Reliability

The extent to which measurements are repeatable.

• Precision

Validity is more fundamental than reliability (what’s the point in measuring a lot of things the same when that said thing is incorrect to begin with?)

It is always better to measure the right quantity somewhat reliably than to measure the wrong quantity very reliably.

Accurate measurements are free from systematic errors (unbiased), while inaccurate measurements are biased.

An unbiased measurement consists of

• A systemic component (the true value of the quantity)

• A random component due to imperfections in the measurement process (measurement error)

Face validity

Criterion validity

Hypothetical construct

Construct validity

Specificity

Discriminant validity

Latent variables

Cronbach’s alpha

A type of ICC that determines the validity of multi-item tests.
Ranges between 0 and 1, with 0.9 considered being the optimal (near perfect consistency between items)

Alpha increases with the number of items, so a score more than 0.9 suggests that the test contains too many items, or redundant items, and could therefore be made more efficient.

Descriptive statistics

e.g. mean values, standard deviation etc

Always do these: they are useful for revealing errors and outliers (e.g. do male macaques really have a lower mean body mass than females or was this a typing error?)

If an unexpected pattern turns out to be real, it is useful to know that the sample may be unrepresentative in some respects.

If a histogram reveals extreme skew, a lot of zeroes or data bounded between two values, this should raise a red flag.

If the dataset includes several continuous variables, it may be useful to construct a correlation matrix to explore how each pair of variables is related.