Big Data

Big Data

A term describing data sets so large, fast, or varied that traditional RDBMSs cannot handle them efficiently.

The 3 Vs

Volume, Velocity, Variety.

Volume

A characteristic of Big Data describing the quantity of data to be stored.

Velocity

A characteristic of Big Data describing the speed at which data enters the system and the speed at which it must be processed.

Variety

A characteristic of Big Data describing variations in the structure of the data being stored.

Scaling up

Handling data growth by migrating to a more powerful single system (more CPUs, more storage on one machine).

Scaling out

Handling data growth by distributing storage across a cluster of commodity servers; the dominant approach for Big Data.

Why RDBMSs are ill-suited for clusters

Distributing an RDBMS requires heavy communication and coordination among nodes, with significant performance cost.

Stream processing

Processing data as it enters the system to decide what to keep and what to discard before storage (focuses on inputs).

Feedback loop processing

Analyzing stored data to produce actionable results (focuses on outputs).

Structured data

Data that conforms to a predefined data model (e.g., relational tables).

Unstructured data

Data that does not conform to a predefined data model (e.g., images, video, audio).

BLOB (Binary Large Object)

An RDBMS data type for storing unstructured objects as a single atomic value; semantic content is opaque to the DBMS.

Variability

Big Data characteristic where the same data values may have different meanings or interpretations over time.

Veracity

Big Data characteristic regarding the trustworthiness/quality of the data.

Value

Big Data characteristic regarding the degree to which data can provide meaningful insights.

Visualization

The ability to graphically present data in a way that makes it understandable to users.