Probability and significance

Define probability

a measure of the likelihood that a particular event will occur where 0 indicates statistical impossibility and 1 is statistical certainty

Define significance

a statistical term that tells us how sure we are that a difference or correlation exists. A ‘significant’ result means that the researcher can reject the null hypothesis

What is an alternative hypothesis (H1)?

one that can be directional - state the direction of the difference or relationship or non-directional - does not state the direction or difference

What is a null hypothesis (H0)?

when we state that there is no difference or relationship - nothing will happen

What is another word for directional?

one-tailed

What is another word for non-directional?

two-tailed

What is the conventional significance level in psychology?

p ≤ 0.05 (5% chance results due to luck).

When might a psychologist choose to use a significance level of 0.01%?

• when the research can’t be replicated

• when there is human cost involved i.e effectiveness of medicine

• when there is a huge gap between the observed/calculated value and the critical value, allowing you to move down the table

What does it mean if a result is statistically significant?

It’s unlikely to have occurred by chance

What do you compare your test statistic to when checking significance?

The critical value from a statistical table. The critical value has been obtained from the statistical test

What are the 3 factors to consider when comparing calculated values and critical values?

1. is it a one-tailed or two-tailed test?

2. how many ppts have taken part? - this usually appears as the N value on the table. For some tests, degrees of freedom (df) are calculated instead

3. what level of significance is trying to be achieved? - p value is most commonly 0.05

How do you know if your result is significant using a table?

Depends on the test — calculated value must be ≤ or ≥ the critical value at chosen p.

What should you do if your result is significant?

Reject the null hypothesis and accept the alternative hypothesis.

What should you do if your result is not significant?

accept the null hypothesis - insufficient evidence for effect

What is a Type I error?

False positive — rejecting a true null hypothesis and accepting the alternative hypothesis (thinking there’s an effect when there isn’t) even though it should be the other way round as the null hypothesis is correct

What is a Type II error?

False negative — accepting a false null hypothesis (missing a real effect) and rejecting the alternative hypothesis, even though the alternative hypothesis is correct

How does changing the significance level affect Type I and Type II errors?

Lower p (stricter) → fewer Type I but more Type II errors; higher p (lenient) → more Type I but fewer Type II errors.

Why is the significance level often set at 0.05?

It balances the risk of making Type I and Type II errors.