LESSON 2 MALITICS

Big Data

Massive amounts of data characterized by Volume, Variety, Veracity, Velocity, and Value.

Volume

The amount of data collected per time unit.

Variety

Whether data is structured or unstructured.

Veracity

The accuracy and reliability of data.

Velocity

The speed at which data arrives.

Value

The usefulness of data for making accurate decisions.

Relational Database

A database that stores data in rows and columns.

Primary Key

A unique key that identifies a record in a database table.

Foreign Key

A key that refers to a primary key in another table.

SQL - structured querying language

A language used to join, select, manipulate, retrieve, and analyze relational data.

NoSQL Database

A non-relational database that stores large volumes of structured or unstructured data.

ETL - Extract, Transform, Load.

a process that combines data from different sources into one location.

Extract

Taking key data from its source.

Transform

Converting data into the appropriate storage format.

Load

Loading transformed data into a storage system.

Hadoop

Open-source software that divides big data processing across multiple computers.

MapReduce

A Hadoop method that divides data into subsets and combines the results.

Data Warehouse

Stores historical data from various company databases for high-speed querying.

Data Mart

Stores data for a specific group of users.

Data Lake

Stores large amounts of data in its native format.

Data Management

The lifecycle management of data from acquisition to disposal.

Data Quality

The condition of data that determines how reliable analytical insights are.

Garbage In, Garbage Out

Poor-quality data produces unreliable analytical results.

Data Preparation

Preparing data by selecting features, handling missing values, and identifying outliers.

Feature Selection

Choosing relevant variables or features while avoiding overfitting.

Unit of Analysis

The what, when, and who of an analysis.

Missing Values

Absent data that can be handled through imputation, omission, or exclusion.

Outliers

Unusual observations in a dataset.

Data Transformation

Changing data into a form useful for analysis.

Aggregation

Combining data into groups, such as weekly sales into monthly sales.

Normalization

Bringing variables to the same scale using the mean and standard deviation.

Dummy Coding

Coding used for nominal categorical variables.