2. scales of measurements of data

what is the mean

computing the average

what is the median

the number in the middle (after sorting)

what is the mode

the number with the highest frequency

types of data

1. nominal

2. ordinal

3. interval

4. ratio

what is nominal data

• lowest level of measurement

• discrete categories

• NO natural order

• estimating a mean, median or standard deviation, would be meaningless

• possible measure: mode, frequency distribution

what are the examples of nominal data

• gender

• occupation

what is ordinal data

• ordered categories

• relative ranking

• unknown “distance” between categories: orders matters but not the difference between values

• possible measure: mode, frequency distribution + median

what is the example of ordinal data

1. very easy

2. easy

3. about right

4. difficult

5. very difficult

what is interval data

• ordered categories

• well-defined ‘unit’ measurement

  • distances between points on the scale are measurable and well-defined

  • can measure differences

• equal interval (between 2 consecutive unit points)

• zero is arbitrary (not absolute), in many cases human-defined

  • if the variable equals zero, it does not mean there is of that variable

• ratio is meaningless

• possible measure: mode, frequency distribution + median + mean, standard deviation, addition/subtraction

what is an example of interval data

• temperature measured in celsius

  • eg 10°C, 28°C

• year of someone’s birth

  • 1990, 2005, 2010, 2022

what is ratio data

• most precise and highest level of measurement

• ordered

• equal intervals

• natural zeros:

  • if the variable equals zero, it means there is none of that variable

  • not arbitrary

• possible measure: mode, frequency distribution, median, mean, standard deviation, addition/subtraction + multiplication and division (ratio)

what are examples of ratio data

• weights

  • 10kg, 20kg, 30kg

• time

  • 10 seconds, 1 hr, 1 day

what is another way of saying numerical variables

quantitative data

what is another way of saying categorical variables

qualitative data

what are the types of data under categorical data

• nominal data (unordered categories)

• ordinal data (ordered categories)

what are the types of data under numerical variables

• discrete data

• continuous data

• interval data

• ratio data

what is missing data

data that is missing and you do not know the mechanism

what is data wrangling

the process of transforming and mapping data from one “raw” data form into another format with the intent of making it more appropriate and valuable for a variety of downstream purposes such as analytics

what is formatting data

• binary coding to convert categories into binary form

  • one hot encoding: unify several entities within one vector

    • example: the colour of a pixel can be red, yellow or green

    • very common in classification tasks

• normalisation

  • linear scaling: scale each variable to [0 1]

  • z-score standardisation: each independent dimension of data is normally distributed

what is data cleaning

the process of detecting and correcting (or remobing) corrupt or inaccurate records from a record set, table or database

what is an example of data cleaning

• clipping outliers

• handling missing features

how do you handle missing features

1. removing the examples with missing features from the dataset

• can be done if the dataset is big enough so we can sacrifice some training examples

2. use learning algorithm that can deal with missing feature values

• example: random forest

3. using data imputation technique (make something up)

what are the methods for imputation

1. replace the missing value of a feature by an average value of this feature in the dataset

2. highlight the missing value

• replace the missing value with a value outside the normal range of values

what is data integrity

it is the maintenance and the assurance of data accuracy and consistency

• a critical aspect to the design, implementation, and usage of any system that stores, processes or retrieves data

what are the types of data visualisation

• distribution

• bars

• boxplots