Theme 1 ~ Foundations of Data Fluency

What is Data

Raw form of what informs our information, once its organized and process it turns into valuable insights

What three things influence data reliability

Reviews, Best seller badges, Photos

Human Basis

Search words, past uses & preferences, filling in what we don’t know with assumptions

Platform Basis

Sponsorships, location and delivery access, past purchases, popularity and assumptions about you

Visible data

Information that can be easily observed, collected, or quantified.

Invisible data

Information that is hidden, less accessible, or not immediately measurable.

Steps of the Decision making process

1. Define Goal (feature matters most) 

2. Collected data 

3. Compared Options 

4. Watch for Bias 

5. Apply Decision Rule

Quantitative

Measurable, countable data

Qualitive

Non numerical, descriptive data that provides meaning

Structured Data Sets

Excel sheet vibes, organised columns and rows, consistent format with predefined categories

Pros of Structured Data Sets

Easier to analyse, consistent, more accurate (rules), Easy to store / link with others

Cons of Structured Data Sets

Real world data is typically more complex, you need to know upfront what your getting

Unstructured Data Set

No fixed format (emails, social media content etc)

Pros of Unstructured

Lots of info, better insights into human behaviour

Cons of Unstructured

Getting it to store and link is very hard

Semi Structured Data Set

Put anchors to grab what is streamlined while allowing the  unstructured bits

Pros of Semi

Flexible, adaptable, easy to store some

Cons of Semi

Inconsistent and can be hard to operate on

Format

Structure and encoding of data set (JPEG, link etc)

Modality

Fundamental type of information or the method by which it was collected (text, images etc)

Granularity

Level of detail given within the data set

Individual Granularity

Each record is a single entity, allows you to do individual information (variability increase)

Aggregate Level Granularity

Data is summarised or grouped (Averages of several entities),  trends, averages

Sensitivity

Level of risk or harm if that data is lost / leaked

Static Temporal

Cross-sectional, one point in time, can't use to infer things, Snapshot of time

Dynamic Temporal

Collected over time, longitudinal, time series

Spatial

Anything related to location or physical space

5 V’s of Big Data

• Volume – The sheer amount of data generated. 

• Velocity – The speed at which data is created and processed. 

• Variety – The diversity of data types and sources. 

• Veracity – The quality, accuracy, and trustworthiness of data. 

• Value – The usefulness and actionable insights derived from data.

Small Data

Focused, curated dataset with higher control, and interpretability

Open Data

freely available for anyone to use, often provided by the governments or organizations thou portals

Closed Data

restricted, requiring payment or permissions to access

Absolute Change

Simply subtracting one from another

Relative Change

Percent change, typically over time

Probability/Risk

Calculated chance that something occurs, or doesn't 

Uncertainty/Margin of Error

A number estimate with a range (typically analysed good or bad)

Sampling

What is/is not included in the data results (think online revs, health norms)

Correlation

Two variables are related and tend to change together, not ALWAYS directly related. Try avoid saying words like x causes y, x leads to y, etc

Causation

One variable directly causes a change in another, there is a guaranteed cause and affect

Spurious Correlations

Happen when two variables follow the same trend BUT have absolutely no meaningful causal connection

Probability

How likely something is to happen [0,1]

Margin of Error

The natural gap between a prediction/estimate says will happen vs what actually happens

Confidence Interval

A range of values that gives a span for the true answer, showing how much uncertainty the estimate has

Risk

Risk is the chance that something negative or unwanted might happen and the consequences if it happens