Practical 2 - Inferential Statistics Notes

Inferential Statistics with R

Overview

This session covers data screening and hypothesis testing for differences between two groups using R. The practical steps include:

  • Creating a dataframe from a .csv file.

  • Plotting histograms and Q-Q plots to explore data distribution.

  • Testing for non-normality.

  • Testing for homogeneity of variances.

  • Conducting a t-test for independent samples.

  • Conducting a Mann-Whitney test for independent samples.

Save answers in a Word file named “Answers to practical 2” and R code in a script named “Practical 2”. Always run code sequentially, line by line, and avoid copying and pasting.

Setting the Working Directory

Ensure the working directory is set to the location of your data files (…./RStudio-stats subfolder). Check the Files tab in RStudio to confirm. If incorrect, change via: Session -> Set working directory -> Choose directory.

Step 1: Creating a Dataframe

1.1 Create a dataframe from a .csv file

  • Context: A meteorological station in Manchester measures atmospheric variables (solar radiation, relative humidity, temperature, pressure, wind speed, etc.).

  • Objective: Investigate the relationship between relative humidity (RH) and average daily temperature (TAVG) from January 1, 2012, to December 31, 2012.

  • Data Source: Manchester_RH_temp.csv file.

# Open a csv file from the console
manchester_data <- read.csv("Manchester_RH_temp.csv", header = TRUE, stringsAsFactors = TRUE)

# Inspect the type of object
class(manchester_data)

  • read.csv(): Reads the CSV file into a dataframe.

  • header = TRUE: Specifies that the first row contains variable names.

  • stringsAsFactors = TRUE: Converts character vectors to factors.

Questions:

  • Question 1.1: How many cases and how many variables does the dataset have?

  • Question 1.2: Specify the type of variable and the scale of measurement for each.

  • Question 1.3: What are the groups you would like to compare?

Step 2: Summary Statistics

1.2 Perform summary statistics and describe the data

Calculate and interpret descriptive statistics.

# Summary Statistics

# Get the mean, median and quartiles along with the range, maximum and minimum values
summary(Manchester_data$rh)
summary(Manchester_data$TAVG)

# Get the mean for each set of variable
mean(Manchester_data$rh)
mean(Manchester_data$TAVG)

# Get the standard deviation for each set of variables
sd(Manchester_data$rh)
sd(Manchester_data$TAVG)

# Get the min and max values for each group
range(Manchester_data$rh)
range(Manchester_data$TAVG)

  • summary(): Provides min, max, median, mean, and quartiles.

  • mean(): Calculates the mean.

  • sd(): Calculates the standard deviation.

  • range(): Provides the minimum and maximum values.

Question:

  • Question 2: Write a short narrative describing the data. Remember to include the basic information that a reader needs for them to have an idea of the data obtained by the researchers.

Step 3: Plotting Histograms

1.3 Plot a histogram

Visualize the distribution of RH and TAVG using histograms.

# PLOT A HISTOGRAM

# for RH
hist_RH <- hist(Manchester_data$rh, xlab= "RH (%)", main= "Relative humidity measured in a Manchester met station", col="skyblue")

# for TAVG
hist_TAVG <- hist(Manchester_data$TAVG, xlab= "Degree Celcius", main= "Avg. daily temperature measured on a Manchester Met station", col="darkred")

  • hist(): Creates a histogram.

  • xlab: Sets the x-axis label.

  • main: Sets the title.

  • col: Sets the bar color.

Question:

  • Question 3: Copy and paste the histograms into the narrative in your Answer document and include a caption.

Step 4: Hypothesis Testing

1.4 Hypothesis testing

Test for a significant difference between the trends of RH and TAVG.

Step 4.1: The Null Hypothesis

1.4.1 The null hypothesis

Define the null hypothesis.

Questions:

  • Question 4.1: Write the logical hypothesis and its null equivalent (logical null hypothesis) in an IF-THEN statement.

  • Question 4.2: Write the null statistical hypothesis.

Step 4.2: Data Screening

1.4.2 Data screening

Assess whether the data meets the assumptions of the statistical test.

1.4.2.1 Testing for non-normality

Test the null hypothesis that the data comes from a normal distribution. Significance level α = 0.05.

Null Hypothesis (Ho): Data comes from a population with normal distribution.

Alternative Hypothesis (Hi): Data does not come from a population with normal distribution.

# Test for non-normality

# use the Shapiro-Wilk test for RH
shapiro.test(Manchester_data$rh)

# use the Shapiro-Wilk test for TAVG
shapiro.test(Manchester_data$TAVG)

  • shapiro.test(): Performs the Shapiro-Wilk test for normality.

Question:

  • Question 4.3: Is the data from both groups normally distributed? Write your results in narrative form. Remember to include the value of the test statistic and the p-value.

Q-Q Plots

Visualize normality using Q-Q plots.

# QQ-plots

# specify that we want the plots together. Build an empty table to put the plots in. So, 1 row and 2 columns
par(mfrow=c(1,2))

# create Q-Q plot for RH
qqnorm(Manchester_data$rh, main="Q-Q Plot for RH")
qqline(Manchester_data$rh, col = "steelblue", lwd = 2)

# create Q-Q plot for TAVG
qqnorm(Manchester_data$TAVG, main="Q-Q Plot for TAVG")
qqline(Manchester_data$TAVG, col = "red", lwd = 2)

  • par(mfrow=c(1,2)): Sets up a plotting area with 1 row and 2 columns.

  • qqnorm(): Creates a normal Q-Q plot.

  • qqline(): Adds a line to the Q-Q plot.

Alternative method using qqPlot() from the car package:

library(car)
par(mfrow=c(1,2))
qqPlot(Manchester_data$rh, main="Q-Q Plot for RH", ylab="Sample Quantiles")
qqPlot(Manchester_data$TAVG, main="Q-Q Plot for TAVG", ylab="Sample Quantiles")
Question:

  • Copy and paste the plots in your answer file and remember to reference them in the narrative where you are describing the results for the normality test.

1.4.2.2 Testing for homogeneity of variances

Test if the variances of the groups are homogeneous using Fligner-Killeen’s test.

Null Hypothesis (Ho): The variances are homogeneous (equal).

Alternative Hypothesis (Hi): The variances are heterogeneous (different).

Other tests for homogeneity of variance:

  1. F-Test

  2. Bartlett’s test

  3. Levene’s test

  4. Fligner-Killeen test

# Test for Homoscesdasticity

# run the Filgner-Killeen test
fligner.test(rh ~ TAVG, data = Manchester_data)

  • fligner.test(): Performs the Fligner-Killeen test.

Question:

  • Question 4.3: Are the variances homogeneous? Write your results in narrative form. Remember to include the value of the test statistic, degrees of freedom and the p- value.

Step 4.3: Student's t-test

1.4.3 Students’ t-test

Perform a t-test for independent samples using the Welch test statistic.

# Independent samples t-test

# perform the t-test
t.test(Manchester_data$rh, Manchester_data$TAVG, data=Manchester_data, paired=TRUE)

  • t.test(): Performs a t-test.

  • paired = TRUE: Performs a paired t-test.

Questions:

  • Question 4.4: Do you failed to accept or to reject your Null Hypothesis? Write the results of your t-test in a narrative form and remember to include the value of the test statistic, degrees of freedom and p-value.

  • Question 4.5: What argument(s) would you change in the t.test function and how if you would like to perform a t-test for unpaired samples?

Mann-Whitney U Test

If the dataset fails the normality test, use the Mann-Whitney U test (non-parametric) to test for differences between groups.

# Mann-Whitney U test
wilcox.test(Manchester_data$rh, Manchester_data$TAVG, data=Manchester_data)

  • wilcox.test(): Performs the Mann-Whitney U test.