large dataset new

Regression gradient units

The units of the gradient are units of y divided by units of x. For rainfall y against sunshine x, this is mm per hour.

Large data set locations

The UK weather stations are Camborne, Hurn, Heathrow, Leeming and Leuchars. Camborne, Hurn and Heathrow are in the south. Leeming and Leuchars are in the north. Camborne, Hurn and Leuchars are coastal.

International LDS locations

The international locations are Jacksonville in the USA, Beijing in China and Perth in Australia. Perth is the only southern hemisphere location.

LDS years and months

The large data set records data for 1987 and 2015, from May to October only.

Why the LDS may not represent the whole year

The LDS only covers May to October, so winter is missing. Winter is usually colder, wetter, darker and windier, so the LDS may not represent the whole year.

UK vs international variables

The international locations only have daily mean temperature, daily total rainfall, daily mean pressure and daily mean wind speed. The UK locations have these plus extra variables such as gusts, wind direction, sunshine, humidity, cloud cover and visibility.

Daily mean temperature units

Daily mean temperature is measured in degrees Celsius and rounded to 1 decimal place.

Daily mean temperature trends

Temperature is generally higher in summer. In the UK, northern stations tend to be colder. Perth has reversed seasons because it is in the southern hemisphere.

Perth in the large data set

Perth is in Australia, so it is in the southern hemisphere and its seasons are reversed compared with the UK. June, July and August are winter months for Perth.

Jacksonville large data set

Jacksonville is in Florida, USA, and there were hurricanes affecting the October 1987 and October 2015 data.

Beijing temperature outliers

For Beijing in the large data set, unusually low daily mean temperatures are likely to be from October, because the LDS only covers May to October and October is the coldest month in that range.

Daily total rainfall

Daily total rainfall is measured in millimetres and includes all precipitation, such as rain, snow and sleet.

Trace rainfall

“tr” means trace rainfall, which is less than 0.05 mm. To clean the data, you can replace tr with 0, which slightly underestimates the mean, or use 0.025 for a midpoint estimate.

Rainfall normal model suitability

Rainfall is often not suitable for a normal model because it is skewed and has lots of zero values.

NA values in the LDS

NA means the value is not available or was not recorded. Ignore NA values when doing calculations.

Daily mean wind speed

Daily mean wind speed is measured in knots. UK values are integers, while international values may be decimals.

Maximum gust units

Maximum gust is measured in knots. One knot is approximately 1.15 miles per hour.

Daily maximum gust

Daily maximum gust is the highest wind speed recorded that day. It is measured in knots and is only recorded for UK stations.

Wind speed trends

Wind speed is usually lower in summer and higher in autumn. Coastal UK locations are usually windier.

Beaufort scale

The Beaufort scale converts wind speed into categories such as light, moderate and fresh. It is qualitative/categorical, so it is not normally suitable for a normal model.

In the large data set, what does wind direction mean?

Wind direction means the direction the wind is coming from, not where it is going.

Wind direction bearing

Wind direction bearings are measured clockwise from north and show where the wind is coming from.

Wind direction categories

There are 16 compass directions for wind direction, e.g. N, NNE, NE, ENE, etc. If all are equally likely, each has probability 1/16.

Distribution for equally likely wind directions

Use a discrete uniform distribution, because wind direction is categorical with a fixed set of equally likely outcomes.

Mean wind direction and gust direction

Mean wind direction and gust direction are only recorded for UK stations. They can be given as bearings or compass directions, and they are usually similar.

Pressure units

Pressure is measured in hectopascals, written as hPa. Do not write kPa.

Pressure range in the LDS

Daily mean pressure is measured in hPa and is usually around 988 to 1038 hPa.

How do you handle coded data where y = x - 1010

For the mean, decode after finding the coded mean: x mean = y mean + 1010. For standard deviation, adding or subtracting a constant does not change it.

Pressure systems and wind direction

For pressure systems, high pressure circulates clockwise and low pressure circulates anticlockwise.

Daily total sunshine

Daily total sunshine is measured in hours and rounded to 1 decimal place. It means hours of bright sunshine, not daylight hours.

Sunshine trends

There is usually more sunshine in summer and less sunshine further north in the UK.

Relative humidity

Daily maximum relative humidity is measured as a percentage, usually between 80% and 100%.

High humidity

Humidity above about 95% can lead to mist or fog.

Cloud cover units

Cloud cover is measured in oktas, using integer values from 0 to 8, so it is discrete.

Cloud cover meaning

Cloud cover is measured in oktas from 0 to 8. 0 means no cloud, 8 means complete cloud cover. It must be an integer.

LDS visibility

Visibility means the greatest distance at which an object can be seen and recognised in daylight. It is recorded in decametres, so 1100 means 11,000 m or 11 km.

Visibility rounding

Visibility is measured in decametres and is usually rounded to the nearest 100 decametres.

Great Storm large data set

The Great Storm in the UK was on 15 to 16 October 1987, so October 1987 wind and gust values may be unusually high.

1987 vs 2015 temperature trend

Daily mean temperature is generally higher in 2015 than in 1987, which may be linked to global warming.

Using LDS data for weather models

The LDS may not be reliable for general predictions because it only uses certain months, certain years and specific locations.

Identifying LDS variables by range

Use the range and units to identify variables. Temperature is in Celsius, rainfall is in mm, pressure is around 1000 hPa, cloud cover is 0 to 8, humidity is a percentage, and visibility has large values in decametres.

Normal model suitability for LDS variables

To say a variable is not suitable for a normal model, give a distribution-shape reason, not just “it is discrete”. Good examples: rainfall is skewed with lots of zeroes, date is uniform, Beaufort is qualitative, or pressure/wind speed may be skewed or not symmetric.

Refining a uniform LDS model

If a uniform model is not suitable, refine it by using a non-uniform discrete distribution based on observed frequencies. Mention context, such as variation by month or location.

Cleaning LDS data

Cleaning the data means dealing with trace and NA values. Trace can be replaced with 0 or 0.025, and NA values should be ignored.

Seasonal trends in the LDS

For northern hemisphere locations, summer is generally hotter, brighter, drier and less windy. Autumn is usually windier.

UK regional trends in the LDS

In the UK, northern stations tend to be colder, wetter and darker. Coastal stations tend to be windier.