AWS - Storage and Databases

Amazon EC2 instance store

provides temporary block-level storage for your EC2 instance; disk storage that is physically attached to the host computer for an EC2 instance, and thus has the same lifespan as the instance; best for temporary memory-based storage needs like buffers, caches, and scratch data

Amazon Elastic Block Store (EBS)

provides persistent block-level storage volumes for use with Amazon EC2 instances; volumes act like external hard drives, offering consistent and low-latency performance for workloads like databases and file systems; doesn’t automatically scale

EBS snapshot

point-in-time backups of EBS volume; EBS snapshots are incremental, so they only save the blocks on the volume that have changed after your most recent

Amazon Simple Storage Service (S3)

service that provides object-level storage, containers in buckets; can upload any type of file; max file size for an object is 5tb; can set permissions to control visibility and access, and versioning to track feature changes; 99.99999% durability in first year

S3 Standard storage class

designed for frequently accessed data; general-purpose storage for cloud applications; provides high availability for objects and has a higher cost than other classes

S3 Standard-Infrequent Access (S3 Standard-IA)

ideal for infrequently accessed data but requires high availability when needed; lower storage price and higher retrieval price than S3 Standard

S3 One Zone-Infrequent Access (S3 One Zone-IA)

stores data in a single Availability Zone, reducing costs compared to S3 Standard-IA, which uses three zones.

S3 Intelligent-Tiering

ideal for data with unknown or changing access patterns; requires a small monthly monitoring and automation fee per object; automatically moves your data to the most cost-effective storage tier based on frequency of access

S3 Glacier Instant Retrieval

archiving data that is rarely accessed and requires millisecond retrieval

S3 Glacier Flexible Retrieval

low-cost storage designed for data archiving; able to retrieve objects within a few minutes to hours

S3 Glacier Deep Archive

lowest-cost object storage class ideal for archiving; data retrieval from 12 to 48 hours; supports long-term retention and digital preservation for data; designed for customers that retain data sets for 7–10 years or longer, to meet regulatory compliance requirements

S3 Outposts

creates S3 buckets on Amazon S3 Outposts; makes it easier to retrieve, store and access data on Outposts; delivers object storage to your on-premise Outposts environment; well for local data residency requirements

Amazon Elastic File System (EFS)

fully managed, scalable file storage service; multiple EC2 instances can access data that is stored in shared file folders, at the same time; using the Linux Network File System (NFS) protocol, and automatically scales

EFS benefits

• automatically replicates data across multiple AZ in region; high availability

• multiple instance can access file system simultaneously

• automatically scales

Amazon Relational Database Service (RDS)

service that enables you to run relational databases in the AWS Cloud; managed service that automates tasks such as hardware provisioning, database setup, patching and backups.

Amazon Aurora

enterprise-class relational database; compatible with MySQL and Postgres SQL relation databases; provides high performance and availability, and automatically scales alongside your workloads & replicates data across multiple Availability Zones

Amazon Dynamo DB

NoSQL database; key-value pairs that delivers millisecond performance at any scale; serverless so not have to install, maintain or operate software; automatically scales

Amazon Redshift

a data warehousing service that you can use for big data analytics; ability to collect data from many sources and helps you to understand relationships and trends across your data

data warehouse

a centralised repository that stores and integrates data from various sources, enabling organisations to perform analysis, reporting and business intelligence (BI) tasks

AWS Database Migration Service (AWS DMS)

enables you to migrate relational databases, nonrelational databases and other types of data stores from source to target; the source remains operational, reducing downtime for any applications that rely

Other cases for AWS DMS

• Development and test database migrations

• Database consolidation

• Continuous replication; copies of data to other target sources

Amazon DocumentDB

document database service that supports MongoDb workloads, so it manages JSON-like documents with dynamic schemas

Amazon Neptune

graph database service; build and run applications that work with highly connected datasets, such as recommendation engines, fraud detection and knowledge graphs

Amazon Quantum Ledger Database (QLDB)

ledger database service; complete history of the changes that have been made to your application data

Amazon Managed Blockchain

service that you can use to create and manage blockchain networks with open-source frameworks

blockchain

distributed ledger system that lets multiple parties run transactions and share data without a central authority

Amazon ElastiCache

service that adds caching layers on top of your databases to help improve the read times of common requests; to help reduce the complexity of administering in-memory caching systems (e.g. redis)

Amazon EC2 instance store benefits

• instance volumes come automatically attached to many EC2 instance types

• high I/O performance for data

• dont have to pay additional fees, included in instance price

• buffers, caches or scratch data

EBS benefits

• easily migrate between Availability Zones using snapshots

• can attach to different EC2 instance types

• modify types and sizes without downtime

Benefits of EBS snapshot

• fast recovery from corruption, accidental deletion, or system failures

• cross-region data migration, volume resizing, volume cloning and sharing data across AWS accounts

• incremental technology = charging for changed blocks

Amazon Data Lifecycle Manager Workflow

automate the creation, retention, and deletion of EBS snapshots using this; create a snapshot policy

S3 benefits

• virtually unlimited storage; no fixed storage limit, scales

• only pay for storage you use

• S3 lifecycle policies automatically move objects between storage classes

• usages: contend distribution, static website hosting, media files

S3 bucket policy

resource-based policies, be attached to buckets, specifies which actions are allowed or denied on bucket, and every object in bucket

Amazon FSx

supports multiple filesystem protocols, including Windows File Server, Lustre, OpenZFS, and NetAPP ONTAP; as a fully managed service, it handles hardware provisioning, patching, and backups.

AWS Storage Gateway

hybrid cloud storage service;extend your local storage to the cloud while maintaining low-latency access to frequently used data.

Amazon S3 File Gateway

store and retrieve cloud objects using familiar file operations; Files written to this server are automatically uploaded to Amazon S3; data is securely stored in the AWS Cloud

Volume Gateway

create virtual storage volumes while maintaining local access to your data; presenting your cloud data as iSCSI volumes that can be mounted by your existing applications

Cached volume mode

stores primary data in the cloud while frequently accessed data is cached locally for low-latency access.

Stored volume mode

locally keeps your complete dataset while asynchronously backing it up to the cloud as EBS snapshots.

Elastic Disaster Recovery

replicates critical workloads to AWS with minimal downtime; lock-level data is continuously replicated to AWS, making it ideal for uses that require robust disaster recovery solutions

Amazon Relational Database Service (Amazon RDS)

managed relational database service that handles routine database tasks such as backups, patching, and hardware provisioning; supports multiple database instance class types that optimize for memory, performance, or input/output (I/O).

in-memory cache

a high-speed storage layer that temporarily stores frequently accessed data in a computer's main memory, or RAM.

AWS Backup

centralizes, eliminates the complexity of managing multiple backup strategies by supporting multiple storage types, including Amazon Elastic Block Store (Amazon EBS) volumes, Amazon Elastic File System (Amazon EFS) file systems, and various databases