4. Databases & Analystics

Why use a database instead of storing data on disk (EFS, EBS, EC2 Instance Store, S3)?

You can structure the data, build indexes to efficiently query/search through it, and define relationships between your datasets

How are databases optimised?

For a purpose — they come with different features, shapes and constraints

What do relational databases look like, and how do you query them?

Like Excel spreadsheets with links between them; you can use SQL to perform queries/lookups

What does NoSQL mean?

Non-SQL = non-relational databases, purpose built for specific data models with flexible schemas for building modern applications

What are the benefits of NoSQL databases?

Flexibility (easy to evolve data model), scalability (designed to scale-out using distributed clusters), high performance (optimised for a specific data model), and highly functional (types optimised for the data model)

What are examples of NoSQL database types?

Key-value, document, graph, in-memory, and search databases

What is JSON and why does it fit the NoSQL model?

JavaScript Object Notation — data can be nested, fields can change over time, and it supports new types such as arrays

What are the benefits of AWS-managed databases?

Quick provisioning, high availability, vertical and horizontal scaling, automated backup & restore, operations, upgrades, OS patching handled by AWS, and monitoring/alerting

What must you handle yourself if you run a database on EC2 instead of a managed service?

Resiliency, backup, patching, high availability, fault tolerance, and scaling

What does RDS stand for and what is it?

Relational Database Service — a managed DB service for databases using SQL as a query language

Which database engines does RDS support?

Postgres, MySQL, MariaDB, Oracle, Microsoft SQL Server, IBM DB2, and Aurora (AWS proprietary database)

What are the advantages of RDS over deploying a database on EC2?

Automated provisioning, OS patching, continuous backups with point-in-time restore, monitoring dashboards, read replicas, Multi-AZ setup for disaster recovery, maintenance windows for upgrades, and storage backed by EBS

What can you NOT do with RDS instances?

SSH into them

What is Amazon Aurora?

A proprietary (not open sourced) database technology from AWS, "AWS cloud optimised"

Which database engines does Aurora support?

PostgreSQL and MySQL

What performance does Aurora claim over RDS?

5x performance improvement over MySQL on RDS, over 3x the performance of Postgres on RDS

How does Aurora storage grow?

Automatically, in increments of 10GB, up to 256TB

How does Aurora's cost compare to RDS?

Costs 20% more than RDS — but is more efficient

What is Amazon Aurora Serverless?

Automated database instantiation and auto-scaling based on actual usage — no capacity planning needed, least management overhead, pay per second (can be more cost-effective)

Which engines does Aurora Serverless support?

PostgreSQL and MySQL

What are use cases for Aurora Serverless?

Infrequent, intermittent or unpredictable workloads

What do RDS Read Replicas do?

Scale the read workload of your DB — data is only written to the main DB

How many RDS Read Replicas can you create?

Up to 15

What does RDS Multi-AZ provide?

Failover in case of an AZ outage (high availability) — data is only read/written to the main database

How many failover AZs can RDS Multi-AZ have?

Only 1 other AZ as failover

What does RDS Multi-Region (Read Replicas) provide?

Disaster recovery in case of a region issue and local performance for global reads — with a replication cost

What is ElastiCache?

A managed Redis or Memcached service — the same way RDS is for managed relational databases

What are caches?

In-memory databases with high performance and low latency

How does ElastiCache help your database?

It helps reduce load off databases for read-intensive workloads

What does AWS take care of with ElastiCache?

OS maintenance/patching, optimisation, setup, configuration, monitoring, failure recovery and backups

What kind of database is DynamoDB?

A fully managed, highly available NoSQL key/value database with replication across 3 AZ — not a relational database

What scale can DynamoDB handle?

Massive workloads — millions of requests per second, trillions of rows, 100s of TB of storage; a distributed "serverless" database

What latency does DynamoDB provide?

Single-digit millisecond latency — low latency retrieval, fast and consistent performance

How does DynamoDB integrate with IAM?

For security, authorisation and administration

What are DynamoDB's cost and scaling characteristics?

Low cost with auto-scaling capabilities

What table classes does DynamoDB offer?

Standard & Infrequent Access (IA)

What is DynamoDB Accelerator (DAX)?

A fully managed in-memory cache for DynamoDB — secure, highly scalable & highly available

What performance improvement does DAX give?

10x — from single-digit millisecond latency to microsecond latency when accessing DynamoDB tables

What is the difference between DAX and ElastiCache at the CCP level?

DAX is only used for and integrated with DynamoDB, while ElastiCache can be used for other databases

What do DynamoDB Global Tables do?

Make a DynamoDB table accessible with low latency in multiple regions

What replication do DynamoDB Global Tables use?

Active-Active replication — read/write to any AWS region

What is Redshift based on, and what is it NOT used for?

Based on PostgreSQL, but not used for OLTP

What is Redshift used for?

OLAP — online analytical processing (analytics and data warehousing)

How often is data loaded into Redshift?

Once every hour, not every second

What performance and scale does Redshift offer?

10x better performance than other data warehouses, scales to PBs of data

How does Redshift store data?

Columnar storage (instead of row based), with massively parallel query execution (MPP); highly available

How is Redshift priced?

Pay as you go based on the instances provisioned

How do you query Redshift, and which BI tools integrate with it?

It has a SQL interface for queries; BI tools such as AWS QuickSight or Tableau integrate with it

What is Redshift Serverless?

Automatically provisions and scales data warehouse underlying capacity — run analytics workloads without managing infrastructure, pay only for what you use

What are use cases for Redshift Serverless?

Reporting, dashboarding applications, real-time analytics

What does EMR stand for?

Elastic MapReduce

What does EMR do?

Helps create Hadoop clusters (Big Data) to analyse and process vast amounts of data

What can EMR clusters be made of?

Hundreds of EC2 instances

Which frameworks does EMR support besides Hadoop?

Apache Spark, HBase, Presto, Flink

What does EMR take care of, and what is it integrated with?

All the provisioning and configuration; auto-scaling and integrated with Spot instances

What are use cases for EMR?

Data processing, machine learning, web indexing, big data

What is Amazon Athena?

A serverless query service to perform analytics against S3 objects using standard SQL

Which file formats does Athena support?

CSV, JSON, ORC, Avro, and Parquet (built on Presto)

How is Athena priced?

$5 per TB of data scanned

How can you save costs with Athena?

Use compressed or columnar data (less data scanned)

What are use cases for Athena?

Business intelligence / analytics / reporting; analysing & querying VPC Flow Logs, ELB logs, CloudTrail trails, etc

What is the exam tip for Athena?

To analyse data in S3 using serverless SQL, use Athena

What is Amazon QuickSight?

A serverless, machine learning-powered business intelligence service to create interactive dashboards

What are QuickSight's characteristics?

Fast, automatically scalable, embeddable, with per-session pricing

What are use cases for QuickSight?

Business analytics, building visualisations, performing ad-hoc analysis, getting business insights using data

Which services does QuickSight integrate with?

RDS, Aurora, Athena, Redshift, S3

What is DocumentDB?

The AWS-implementation of MongoDB (a NoSQL database) — the same way Aurora is an AWS-implementation of PostgreSQL and MySQL

What is MongoDB used for?

To store, query and index JSON data

What are DocumentDB's availability and storage characteristics?

Fully managed, highly available with replication across 3 AZ; storage automatically grows in increments of 10GB

What workloads does DocumentDB scale to?

Automatically scales to workloads with millions of requests per second

What is Amazon Neptune?

A fully managed graph database

What is a popular example of a graph dataset?

A social network — users have friends, posts have comments, comments have likes from users, users share and like posts

What are Neptune's availability and replica characteristics?

Highly available across 3 AZ, with up to 15 read replicas

What is Neptune optimised for?

Building and running applications working with highly connected datasets — optimised for these complex and hard queries

What scale and latency does Neptune offer?

Can store up to billions of relations and query the graph with milliseconds latency

What are use cases for Neptune?

Knowledge graphs (e.g. Wikipedia), fraud detection, recommendation engines, social networking

What is Amazon Timestream?

A fully managed, fast, scalable, serverless time series database

How does Timestream scale?

Automatically scales up/down to adjust capacity

What scale of events can Timestream handle?

Store and analyse trillions of events per day

How does Timestream compare to relational databases?

1000s of times faster and 1/10th the cost

What analytics features does Timestream have?

Built-in time series analytics functions to help identify patterns in your data in near real-time

What does blockchain make possible?

Building applications where multiple parties can execute transactions without the need for a trusted, central authority

What is Amazon Managed Blockchain?

A managed service to join public blockchain networks or create your own scalable private network

Which frameworks is Amazon Managed Blockchain compatible with?

Hyperledger Fabric & Ethereum

What is AWS Glue?

A managed extract, transform, and load (ETL) service — fully serverless

What is Glue useful for?

Preparing and transforming data for analytics

What is DMS?

Database Migration Service — quickly and securely migrate databases to AWS; resilient, self-healing

What migration types does DMS support?

Homogeneous migrations (e.g. Oracle to Oracle) and heterogeneous migrations (e.g. Microsoft SQL Server to Aurora)

Which AWS services are the relational OLTP databases?

RDS & Aurora (SQL)

Which AWS service is the in-memory database?

ElastiCache

Which AWS services cover key/value databases?

DynamoDB (serverless) & DAX (cache for DynamoDB)

Which AWS service is the OLAP data warehouse?

Redshift (SQL)

Which AWS service provides Hadoop clusters?

EMR

What is DocumentDB in one phrase?

"Aurora for MongoDB" — a JSON NoSQL database

Which AWS service is the graph database?

Neptune

Which AWS service is the time-series database?

Timestream