Paper 4: Further Probability and Statistics

Probability density function

• A probability density function describes the probability of a continuous random variable falling within a particular interval of values, with the area between the function and the x-axis defining the probability over an interval. The probability over an interval therefore can be found by integrating the probability density function using the limits of the interval.

• Since probability cannot take a negative value, the function cannot be negative or go below the x-axis.

• Since the total probability for a continuous random variable is 1, the total area under the curve must be equal to 1.

• Since the probability of a continuous random variable taking a single value is 0 (P(X = a) = 0), P(X < a) and P(X ≤ a) have the same value.

• Probability density functions can be represented by a combination of different functions, each corresponding to part of the domain. These are known as piecewise probability density functions:

Cumulative distribution function

• A cumulative distribution function calculates the accumulated probability of a continuous random variable taking a value less than or equal to a specific value of x.

• The cumulative distribution function is obtained by integrating each part of the piecewise probability density function using the lowest value of the domain (L) as the lower limit and x as the upper limit, adding the cumulative probability of the previous parts to the next part. A dummy variable (t) is used for the integration. The cumulative distribution function is also defined piecewise.

• The cumulative distribution function can be used to directly find P(X ≤ x).

Obtaining a probability density function (f(x)) from a cumulative distribution function (F(x))

• Differentiate each part of the piecewise cumulative distribution function, then define each part: f(x) = d/dx(F(x))

• For parts of the cumulative distribution function that differentiate to 0, define the probability density function as 0 for ‘otherwise’.

Percentiles

• The nth percentile of a continuous random variable, X, is the value of x where P(X ≤ x) = n/100.

• The median is the 50th percentile, the lower quartile is the 25th percentile, and the upper quartile is the 75th percentile.

• To find the nth percentile (x) from a cumulative distribution function, identify the correct interval of the piecewise cumulative distribution function, and set up the equation F(x) = n/100, then rearrange to find x.

• To find the nth percentile from a probability distribution function, find the cumulative distribution function first.

Mode of a continuous random variable

• The mode of a continuous random variable is the highest point (y-value) of a probability density function.

• This is either at one of the end-points of the domain or at a maximum point.

• Differentiate the probability density function, and compare with the y-values obtained from the end-points of the domain to find the highest.

Calculating E(X) of a continuous random variable X

• Where f(x) is the probability density function of the continuous random variable X:

• For continuous random variables defined with a piecewise probability density function, integrate each part of the function with their respective intervals as the limits, adding them together.

Calculating E(g(X)) of a continuous random variable X

• Where f(x) is the probability density function of the continuous random variable X, and g(X) is a function of X:

• For continuous random variables defined with a piecewise probability density function, integrate each part of the function with their respective intervals as the limits, adding them together.

Calculating Var(X) of a continuous random variable X

Var(X) = E(X²) - (E(X))²

Consider the continuous random variable X with the cumulative distribution function

Find the cumulative distribution function of Y = X² / 4

• Define G(y), the cumulative distribution function of y:

  • G(y) = P(Y ≤ y)

  • Substitute the definition of Y: G(y) = P(X² / 4 ≤ y)

  • Rearrange to make X the subject: G(y) = P(X ≤ 2√y)

  • G(y) = F(2√y)

• Form G(y) by substituting each x of F(x) with 2√y and transforming the intervals using Y = X² / 4:

Consider the continuous random variable X with the cumulative distribution function

Find the cumulative distribution function of Y = 1/X

• Define G(y), the cumulative distribution function of y:

  • G(y) = P(Y ≤ y)

  • Substitute the definition of Y: G(y) = P(1/X ≤ y)

  • Rearrange to make X the subject: G(y) = P(X ≥ 1/y)

  • G(y) = 1 - F(1/y)

• Form G(y) by substituting each x of F(x) with 1/y and subtracting the result from 1 and transforming the intervals using Y = 1/X and swapping the top and bottom parts of the function:

S_xx

• Sum of squared deviations:

Unbiased estimator of the variance

• A population’s variance can be estimated by taking a random sample from it and calculating the unbiased estimator of the variance, s²:

• To ensure the estimator is unbiased, the formula involves division by n - 1, which is the number of values in the calculation that are free to vary, known as the degrees of freedom. One degree of freedom is lost when calculating x̄, the sample mean, as the last data point is not free to vary and must have a value that achieves the value of the mean calculated.

t-distribution

• The t-distribution is a family of curves showing the probability distribution of the test statistic.

• The y-axis is the probability density, and the x-axis is the test statistic.

• Features of the curves include the following:

  • Mean = median = mode = 0, meaning the curve is centred at a t-value of 0.

  • It is bell-shaped.

  • It is symmetrical about the mean, which is at the centre peak.

  • The tails of the curve are asymptotic towards the x-axis.

• The curves in the family vary in structure due to their varying degrees of freedom:

  • When the sample size and degrees of freedom are small, there is higher probability density at the tails.

  • When the sample size and degrees of freedom are large, there is less probability density at the tails.

  • When the degrees of freedom approaches infinity, the t-distribution curve converges to the shape of the z-distribution.

One-tailed and two-tailed tests

• A one-tailed test tests whether the sample mean has a directional increase or decrease.

• A two-tailed test tests whether the sample mean has an non-directional difference.

Critical value

The critical value is the boundary between the region of acceptance and rejection on a probability distribution: if the calculated test statistic is more extreme than the critical value, the null hypothesis H₀ is rejected.

Significance level

The significance level is the area at the extremities of the curve that are rejected.

Single-sample t-test

• A single-sample t-test is a hypothesis test that determines whether there is a significant difference between the mean of the sample group and the hypothesised mean of the population.

• It is used when the population is normally distributed and when the population variance, σ², is unknown.

• State whether the test is a one-tailed or two-tailed test.

• Two hypotheses are made:

  • Null hypothesis, H₀: μ = k, where k is the assumed value of the mean that is being tested.

    • Assumes that any difference between the mean of the sample group and the hypothesised mean of the population has negligible meaning and is due to random sampling fluctuation.

  • Alternative hypothesis, H₁: μ ≠ k / μ > k / μ < k, where k is the assumed value of the mean that is being tested.

    • A one-tailed test uses μ > k / μ < k.

    • A two-tailed test uses μ ≠ k.

    • Assumes the mean of the sample group and the hypothesised mean of the population have a statistically significant difference.

• The test statistic / t-value is calculated: test statistic = (x̄ - μ) / (s / √n), where x̄ is the sample mean, s is the unbiased estimator of the standard deviation, μ is the hypothesised mean of the population, and n is the sample size.

  • The test statistic shows the number of standard errors the sample mean is away from the hypothesised mean of the population.

  • The numerator calculates the difference of the sample mean from the hypothesised mean of the population.

  • The denominator is the standard error of the mean. A high standard error of the mean indicates that the sample mean is unstable and is unreliable as a representation of the population mean. A low standard error of the mean indicates that the sample mean is stable and is a reliable representation of the population mean.

• The critical value is found:

  • Critical value = t_{p, n-1}

    • One-tailed test: p = 1 - significance level

    • Two-tailed test: p = 1 - (significance level / 2)

    • n-1 is the degree of freedom.

  • The critical value is found by using a t-distribution table and locating the value of p and the degrees of freedom. We make the critical value positive for a right-tailed test, negative for a left-tailed test, or matching the sign of the test statistic for a two-tailed test.

• The null hypothesis H₀ is then not rejected if the calculated test statistic is less extreme than the critical value and is rejected if the calculated test statistic is more extreme than the critical value.

• The conclusion is stated:

  • If H₀ is not rejected: ‘there is insufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

  • If H₀ is rejected: ‘there is sufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

2-sample Z-test

• A 2-sample Z-test is a hypothesis test that determines if there is a significant difference between two population means.

• It is used when the sample sizes are small, the true population variances are known, the two groups of data are independent from each other, and the populations are normally distributed.

• It is used when the sample sizes are large: n₁ ≥ 30, n₂ ≥ 30, the true population variances are unknown, and the two groups of data are independent from each other.

• State whether the test is a one-tailed or two-tailed test.

• Two hypotheses are made:

  • Null hypothesis, H₀: μ₁ - μ₂ = 0

    • Assumes that any difference between the means of the two samples has negligible meaning and is due to random sampling fluctuation.

  • Alternative hypothesis, H₁: μ₁ - μ₂ ≠ 0, μ₁ - μ₂ > 0, μ₁ - μ₂ < 0

    • A one-tailed test uses μ₁ - μ₂ > 0 / μ₁ - μ₂ < 0

    • A two-tailed test uses μ₁ - μ₂ ≠ 0

    • Assumes the two populations have means that have a statistically significant difference.

• The test statistic / z-value is calculated, where x̄ is the sample mean, σ² is the population variance, and n is the sample size:

  

  • If the population variances are unknown and the sample sizes are large, the variances of the samples are used as estimates.

  • The test statistic shows how many standard errors the difference in sample means is away from the hypothesised difference of 0.

  • The numerator calculates the difference between the two sample means.

  • The denominator is the standard error of the difference between means. A high standard error indicates that the calculated difference is sample means is unstable and unreliable as a representation of the difference between population means. A low standard error indicates that the difference between the sample means is stable and a reliable representation of the difference between population means.

• The critical value is found using the normal distribution table:

  • Find the area of rejection in each tail, which is equal to the significance level for a one-tailed test and is equal to the significance level / 2 for a two-tailed test.

  • 1 - area of rejection = target area, which is located in the table body to find the critical value.

  • We make the critical value positive for a right-tailed test, negative for a left-tailed test, or matching the sign of the test statistic for a two-tailed test.

• The null hypothesis H₀ is then not rejected if the calculated test statistic is less extreme than the critical value and is rejected if the calculated test statistic is more extreme than the critical value.

• The conclusion is stated:

  • If H₀ is not rejected: ‘there is insufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

  • If H₀ is rejected: ‘there is sufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

Pooled estimator of common variance

• A pooled estimator of a common variance is used when two populations have unknown but equal variances to estimate this variance.

• It is used for 2-sample t-tests when the sample sizes are too small for a 2-sample Z-test (n₁ ≤ 30, n₂ ≤ 30).

• When you are not given unbiased estimators of the variance for each sample, the following formula is used:

  

• When you are given the unbiased estimators of the variance for each sample, the following formula can be derived from the previous:

2-sample t-test

• A 2-sample t-test is a hypothesis test that determines if there is a significant difference between two population means.

• It is used when the two groups of data are independent from each other, are normally distributed, and the true variances of the populations are unknown but the same.

  • Sample sizes can be small.

• State whether the test is a one-tailed or two-tailed test.

• Two hypotheses are made:

  • Null hypothesis, H₀: μ₁ - μ₂ = k

    • Assumes that any difference between this difference between the means of the two samples has negligible meaning and is due to random sampling fluctuation.

  • Alternative hypothesis, H₁: μ₁ - μ₂ ≠ k, μ₁ - μ₂ > k, μ₁ - μ₂ < k

    • A one-tailed test uses μ₁ - μ₂ > k / μ₁ - μ₂ < k

    • A two-tailed test uses μ₁ - μ₂ ≠ k

    • Assumes the two populations have a difference in means that is different from the hypothesised difference that is statistically significant.

• The pooled estimator of the common variance is calculated.

• The test statistic / t-value is calculated, where x̄ is the sample mean, μ is the hypothesised mean of the population, s² is the pooled estimator of common variance, and n is the sample size:

  

  • The test statistic shows how many standard errors the difference in sample means is away from the hypothesised difference.

  • The numerator calculates the difference of the difference between the two sample means from the hypothesised difference between the two population means.

  • The denominator is the standard error of the difference between means. A high standard error indicates that the calculated difference in sample means is unstable and unreliable as a representation of the difference between population means. A low standard error indicates that the difference between the sample means is stable and a reliable representation of the difference between population means.

• The critical value is found:

  • Critical value = t_{p, n1 + n2 -2}

    • One-tailed test: p = 1 - significance level

    • Two-tailed test: p = 1 - (significance level / 2)

    • n₁ + n₂ - 2 is the degree of freedom.

  • The critical value is found by using a t-distribution table and locating the value of p and the degrees of freedom. We make the critical value positive for a right-tailed test, negative for a left-tailed test, or matching the sign of the test statistic for a two-tailed test.

• The null hypothesis H₀ is then not rejected if the calculated test statistic is less extreme than the critical value and is rejected if the calculated test statistic is more extreme than the critical value.

• The conclusion is stated:

  • If H₀ is not rejected: ‘there is insufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

  • If H₀ is rejected: ‘there is sufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

Paired sample t-test

• A paired sample t-test is used to measure the effect of a variable on a set of data that is measured twice: before the change in variable and after. The mean of the differences between each pair of data is calculated.

• It is used when the two groups of data contain corresponding pairs and the differences between the pairs are normally distributed.

• State whether the test is a one-tailed or two-tailed test.

• Two hypotheses are made:

  • Null hypothesis, H₀: μ_d = k

    • Assumes that any mean difference observed between the paired measurements that is different to the hypothesised mean difference has negligible meaning and is due to random sampling fluctuation.

  • Alternative hypothesis, H₁: μ_d ≠ k, μ_d > k, μ_d < k

    • A one-tailed test uses μ_d > k / μ_d < k

    • A two-tailed test uses μ_d ≠ k

    • Assumes that any mean difference observed between the paired measurements that is different from the hypothesised mean difference is statistically significant.

• Calculate the unbiased estimator of the standard deviation of the differences, where n is the number of data pairs:

• The test statistic / t-value is calculated, where d̄ is the sample mean difference between the pairs of data, μ_d is the hypothesised mean difference between the pairs of data, s_d is the unbiased estimator of the standard deviation of the differences, and n is the number of data pairs:

  

  • The test statistic shows how many standard errors the sample’s mean of the differences between the pairs of data is away from the hypothesised mean of the differences.

  • The numerator is the sample mean of the differences between the pairs of data.

  • The denominator is the standard error of the mean difference. A high standard error indicates that the calculated mean of the differences between the pairs of data is unstable and unreliable as a representation of the mean difference present in the population. A low standard error indicates that the calculated mean of the differences between the pairs of data is stable and a reliable representation of the mean difference present in the population.

• The critical value is found:

  • Critical value = t_{p, n-1}

    • One-tailed test: p = 1 - significance level

    • Two-tailed test: p = 1 - (significance level / 2)

    • n-1 is the degree of freedom.

  • The critical value is found by using a t-distribution table and locating the value of p and the degrees of freedom. We make the critical value positive for a right-tailed test, negative for a left-tailed test, or matching the sign of the test statistic for a two-tailed test.

• The null hypothesis H₀ is then not rejected if the calculated test statistic is less extreme than the critical value and is rejected if the calculated test statistic is more extreme than the critical value.

• The conclusion is stated:

  • If H₀ is not rejected: ‘there is insufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

  • If H₀ is rejected: ‘there is sufficient evidence to claim [H₁]’, where H₁ is written in the context of the question.

Confidence interval for a population mean

• The confidence interval for a population mean is the central region of a t-distribution curve that represents the reliability of the population mean being within that interval if the estimation process was repeated multiple times. For example, a 95% confidence interval would have a 95% theoretical long-run expected average, meaning 95% of the random samples taken will have the population mean within the interval.

• When the population variance is unknown, the t-distribution is used.

• Confidence interval = x̄ ± (t_{p, n-1})(s/√n), where x̄ is the sample mean, t_{p, n-1} is the critical value, s is the square root of the unbiased estimator of the variance, and n is the sample size.

  • s/√n is the standard error of the mean and represents the average distance the sample mean will miss the true population mean due to random sampling fluctuation.

  • The confidence interval is written in the form (lower bound, upper bound)