Not needed

AWS Compute & Storage Services – Comprehensive Notes

EC2: Elastic Compute Cloud – Overview

  • EC2 is a web service to run virtual server instances in the cloud.
  • Instances can run Windows, Linux, or macOS operating systems.
  • Key economic shift: you pay only for capacity you use (elastic, on-demand provisioning).
  • Traditional server vs AWS EC2:
    • Traditional: Physical hardware, dedicated CPU, RAM, disk, NIC; fixed OS on hardware.
    • AWS EC2: Virtualized hardware (CPU, RAM, disk, NIC) presented as instances; flexible mix of instance types; scalable.

EC2 Launching and Core Concepts

  • An EC2 launch involves:
    • Amazon Machine Image (AMI) defines the instance configuration (OS and base software).
    • Instance type defines the hardware profile (CPU, memory, storage, networking).
    • Storage includes root volume and any additional EBS volumes.
    • Snapshot is a point-in-time backup of an instance (EBS Snapshot).
    • Instance details include network, IAM role, public IP, monitoring, tenancy.
    • Tags help manage and identify resources.
    • Security Groups control network traffic to the instance.
  • EC2 instances can be customized; you can create a customized AMI (Custom AMI).
  • Instance type categories (examples shown):
    • General purpose: balanced compute, memory, and networking
    • Compute optimized: high CPU performance
    • Memory optimized: large memory per CPU
    • Storage optimized: high I/O for storage-heavy workloads
    • GPU/Other specialized: GPU, HPC, etc.
    • Examples listed: t2.micro, c5n.large, r5ad.large, d2.xlarge, g2.2xlarge
  • AWS provides a spectrum of compute services beyond EC2 (see page 13), including ECS, EKS, Lambda, Fargate, etc.

AMIs and Images

  • Amazon Machine Image (AMI) definitions include:
    • General OS images (Linux, Windows, etc.).
    • Snapshot-based backups and base software bundles.
  • AMI types:
    • Community AMIs (AWS users/community)
    • AWS Marketplace AMIs (vendor-based, often commercial software)
    • My AMIs (your own custom images)
  • You can customize and create a Custom AMI from your configured instance.

EC2 Instance Lifecycle

  • States: pending → running → stopping → stopped → shutting-down → terminated
  • Billing implications:
    • Pending: not billed
    • Running: billed
    • Stopping/Stopped: not billed
    • Shutting-down/Terminated: not billed
  • Special notes: Reserved Instances billed through term even if terminated; RI term continues per schedule
  • EBS-backed instances support stop/start lifecycle; other instance types may differ

EC2 Pricing Models

  • On-Demand: pay per hour or per second (depending on instance type); flexible; suitable for unpredictable workloads; non-interruptible
  • Reserved Instances and Savings Plans: substantial discounts for predictable, steady-state usage; term options typically 1-3 years; unused capacity may be resold in some cases
  • Spot Instances: deep discounts (up to ~90% off); instances can be interrupted with little notice; suitable for fault-tolerant, flexible workloads
  • Dedicated Instances/Servers: physical tenancy isolation; higher cost; for regulatory or compliance requirements
  • Summary concept: choose a pricing model based on workload predictability and tolerance for interruption

EC2 Auto Scaling

  • Purpose: ensure the correct number of EC2 instances are available to handle load
  • Mechanism: collections of instances organized into Auto Scaling Groups (ASGs)
  • Types of scaling:
    • Vertical Scaling (Scale-up/Scale-down): adjust capacity of existing instances
    • Horizontal Scaling (Scale-out/Scale-in): add or remove instances
  • Benefits/Features:
    • Health monitoring and replacement of unhealthy instances
    • Load balancing across Availability Zones (AZs)
    • Support for multiple instance types and purchase options
    • Automatic replacement of Spot Instances
    • Integration with Load Balancers
    • Scalability and Instance Refresh
  • Limits: Vertical scaling adds resources to one machine; Horizontal scaling adds more machines
  • Coverage: ensures minimum capacity in ASG; can scale based on metrics (noted generally in practice)

Elastic Load Balancing (ELB)

  • Purpose: distribute inbound network traffic across a group of backend servers to improve scalability and availability
  • How it works: health checks on registered targets; avoids routing traffic to unhealthy targets
  • Types of Load Balancers:
    • Classic Load Balancer: traditional, layer 4/5 and some layer 7 support; limited routing features
    • Application Load Balancer (ALB): HTTP/HTTPS layer 7 routing; supports path-based and host-based routing; dynamic port mapping
    • Network Load Balancer (NLB): TCP/SSL layer 4 routing; high throughput, millions of requests per second
    • Gateway Load Balancer: deploys virtual appliances; transparent gateway with distribution
  • Integration: can work with ASGs, Lambda, ECS/EKS, and other AWS services

AWS Compute Services Overview (context for other services)

  • AWS compute services include: EC2, Auto Scaling, ECR (Elastic Container Registry), ECS (Elastic Container Service), EKS (Elastic Kubernetes Service), Fargate, Lambda, Lightsail, Batch, Elastic Beanstalk, VMware Cloud on AWS, and Outposts
  • Cloud-native and serverless options complement EC2 for diverse architectures

Amazon S3 (Simple Storage Service) – Overview

  • Object storage service for storing and retrieving any amount of data from anywhere
  • Highly scalable, available, secure, and high performance
  • Durability: designed for 99.999999999% durability (11 9's) across regions
  • Unlimited storage; single object limit: 5 TB
  • Regions, buckets, and object keys define organization; data stored within a region and available globally
  • Common use cases: data lakes, websites, cloud-native apps, backups, archives, ML, analytics

S3 Storage Classes

  • Range of storage classes for different use cases:
    • Amazon S3 Standard
    • Amazon S3 Intelligent-Tiering
    • Amazon S3 Standard-Infrequent Access (S3 Standard-IA)
    • Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)
    • Amazon S3 Glacier
    • Amazon S3 Glacier Deep Archive
    • S3 Outposts (storage on AWS Outposts for on-prem workloads)
  • Each class has distinct cost, durability, availability, and access patterns
  • Examples of use cases:
    • S3 Standard: frequently accessed data (web assets, images, videos)
    • S3 Standard-IA / One Zone-IA: infrequent access with rapid retrieval; disaster recovery backups
    • S3 Intelligent-Tiering: automatic tiering based on access patterns
    • S3 Glacier / Glacier Deep Archive: archival storage with longer retrieval times
    • S3 Outposts: on-premises storage with AWS Outposts

S3 Lifecycle Management

  • Lifecycle rules help optimize storage spend
  • Transitions: move objects between storage classes as they age
  • Expiration: define when objects expire and are deleted
  • Example classes involved: S3 Standard, S3 Standard-IA, S3 Intelligent-Tiering, S3 One Zone-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, S3 Glacier Deep Archive

S3 Accessibility and URLs

  • Bucket URLs can be path-style or virtual-hosted–style
    • Path-style: https://s3.[region].amazonaws.com/[bucket-name]
    • Virtual-hosted: https://[bucket-name].s3.[region].amazonaws.com
  • Data remains within the region; bucket and object paths determine access

Common S3 Use Cases

  • Storing application assets (static web hosting)
  • Backups and disaster recovery
  • Staging area for big data
  • Archival data and machine learning datasets

Amazon S3 Pricing (Summary)

  • Pay for what you use: per GB-month, data transfer (OUT to other regions), and requests (PUT, COPY, POST, LIST, GET)
  • Not charged for: in-transfers within AWS services in same region (IN to S3, OUT to CloudFront/EC2 in same region)
  • Pricing examples depend on class and region

Amazon S3 vs EBS vs EFS: Quick Comparison

  • EBS (Elastic Block Store): block storage for EC2; max per volume: up to 16 TiB; data lives in a single AZ; high IOPS/throughput options with gp3/io2/sc1/st1 families
  • EFS (Elastic File System): scalable, fully managed file storage; accessible from multiple EC2 instances concurrently; shared NFS file system; distributed across AZs; high throughput and low latency; POSIX-compliant
  • S3 (Simple Storage Service): object storage; unlimited storage; region-based durability and availability; best for unstructured data, backups, media, data lakes
  • Cleanup mental model: choose block storage for databases and OS disks (EBS), shared file storage for multi-EC2 workloads (EFS), and object storage for large-scale, durable data with varied access patterns (S3)

Elastic Block Store (EBS) – Deep Dive

  • EBS provides persistent block storage volumes for EC2 instances
  • Key features:
    • Block-level storage with persistent volumes
    • Volumes are replicated within an Availability Zone for durability
    • Snapshots can back up volumes to S3 automatically
    • Use cases: boot volumes, data storage for EC2, databases, enterprise apps
  • Volume types:
    • Solid State Drives (SSD)
    • General Purpose (gp3, io2) – fast I/O; suitable for most workloads
    • Hard Disk Drives (HDD)
    • Throughput-Optimized (st1)
    • Cold (sc1)
  • Volume characteristics (typical values shown in the slides):
    • gp3: Max size 16 TiB; 16,000 IOPS; 250 MiB/s throughput; latency: low
    • io2: Max size 16 TiB; 64,000 IOPS; 1,000 MiB/s throughput; latency: very low
    • st1: Max size 16 TiB; 500 IOPS; 500 MiB/s throughput
    • sc1: Max size 16 TiB; 250 IOPS; 250 MiB/s throughput
  • Example use cases by class:
    • gp3/io2: critical workloads needing high performance (databases, high-IOPS apps)
    • st1: streaming, large sequential throughput workloads
    • sc1: large volumes with lower cost, infrequent access

Amazon EFS – Elastic File System

  • EFS is a scalable, fully managed cloud-based file storage service for shared access by multiple EC2s
  • Characteristics:
    • File storage (NFS) with POSIX permissions; supports NFS v4.0 and v4.1
    • Scales on demand to petabytes; thousands of EC2 instances can share a single file system
    • Highly available and durable across AZs within a region
    • On-demand elasticity: no upfront provisioning; storage capacity grows/shrinks with need
  • Architecture and access:
    • Created in a region, mounted across multiple AZs via mount targets
    • Can connect to on-premises resources via AWS Direct Connect or VPN
    • Data is distributed across AZs for high availability and durability
  • Use cases:
    • Home directories, application data sharing, lift-and-shift for apps requiring shared storage
    • Big data analytics, media workflows, content management, development/test environments
  • Performance characteristics:
    • Massively parallel shared access; aggregate throughput and IOPS scale with demand
    • Throughput can exceed gigabytes per second and scale with number of clients
  • Security and access:
    • Data at rest encryption; data in transit encryption; access control via VPC, IAM policies, and security groups
    • Fine-grained app-level permissions and directory-based access control

AWS Lambda – Serverless Compute

  • Lambda is serverless compute: developers upload code (ZIP) and AWS runs it without provisioning servers
  • Automatic scaling: handles demand by scaling functions up or down automatically
  • Pricing:
    • Requests: billed by the number of requests to the function
    • Duration: billed for the time your code runs (from invocation to termination)
    • Free tier: 1,000,000 free requests per month
  • Benefits:
    • Zero server management
    • Automatic high availability and scaling
    • Cost efficiency: pay only while code runs; no charge when idle
    • Focus on writing logic rather than managing infrastructure

AWS Lambda – Practical Things to Know

  • Typical use cases: event-driven processing, lightweight APIs, data processing pipelines, real-time file processing
  • How it fits with other services: integrates with API Gateway, S3 events, DynamoDB streams, CloudWatch, and more

Quick Reference: Key Numbers and Concepts (LaTeX-formatted)

  • SLA and durability: AWS EC2 and related services offer high availability and reliability such as a SLA commitment of $$99.99\ ext{\