Cloud Computing & Big Data Systems Fundamentals

Real-world Examples of Cloud Computing

  • Cloud providers and scale
    • Meta (Facebook):
      • In 2012, had 180,000 servers.
      • Currently has multi-millions servers.
      • Operates 35 data centers (DCs).
    • Microsoft:
      • Over 3 million servers in 2017.
      • Operates 300 data centers worldwide.
    • Google:
      • Over 2.5 million servers in 2016.
    • Alibaba:
      • Multi-million servers in 2022.

Anatomy of a Datacenter

  • Datacenter components

    • Cooling towers.
    • Server farms.
    • Power substations.
    • Server racks.
    • Network rooms.
    • Server cages.

  • Datacenter as a computer

    • Typical elements in warehouse-scale systems:
      • 1U server.
      • 7' rack with Ethernet switch.
      • Small cluster with a cluster-level Ethernet switch/router.

Evolution of Datacenter Networks

  • Small-sized cluster (2004)

    • In 2004, Google had only 20k servers in a datacenter.
    • Employed a traditional 2Tbps four-post cluster network.

  • Modern DCN with Spine-Leaf Architecture

    • Uses a rail-optimized network under dual-ToR (Top of Rack).

  • Tree-like DC Network

    • Consists of:
      • Datacenter border router.
      • Cluster router.
      • Layer-2 aggregation switches.
      • Layer-2 switches.
      • Top of rack switches.

  • Inter-DC WAN

    • Utilizes 100 GbE links for communication between data centers.
    • Example: AWS Inter-DC Cable.

Datacenter Infrastructure

  • Cooling

    • Essential for dissipating heat generated by servers.

  • Power

    • Powers all the servers and infrastructure.

Cost Structure of Datacenters

  • Monthly Costs Breakdown

    • Servers: 284,686
    • Power & Cooling: 1,042,440
    • Other Infrastructure: 2,997,090
    • Power: 1,296,902

  • Amortization

    • 3-year server amortization.
    • 15-year infrastructure amortization.

Dominant Cloud Providers

  • Market Share (Q3 2022)

    • AWS: 34%
    • Azure: 21%
    • Google Cloud: 11%
    • Alibaba Cloud: 5%
    • IBM Cloud: 3%
    • Salesforce: 3%
    • Tencent Cloud: 2%
    • Oracle: 2%

  • Cloud Infrastructure Service Revenues (12 months ended September 2022)

    • 217 billion

The Booming Cloud Market

  • Cloud Infrastructure Services Market (IaaS, PaaS, Hosted Private Cloud)

    • Q3 2022 Worldwide Revenues: 50 billion

  • Market Share Q3 2022

    • Amazon: 34%
    • Microsoft: 21%
    • Google: 11%
    • Next 20 Companies: 25%
    • Others: 9%

Cloud-Based Services

  • Examples
    • SaaS: Slack, Zendesk, Salesforce, Acquia, G Suite, Office 365, SugarCRM
    • FaaS: Cloud Functions, OpenWhisk
    • DaaS: Cloud Datastore, CouchDB, DynamoDB, Cassandra, SQL Azure, Oracle Data Cloud, CockroachDB
    • PaaS: Microsoft Azure, OpenShift Origin, Salesforce Platform, OpenPaaS Suite, App Engine, ZADARA
    • STaaS: ownCloud, Google Drive, Amazon S3, Dropbox, Google Cloud Storage, cozy.io, Rackspace, SoftLayer, Apache CloudStack, OpenStack, OneDrive, iCloud
    • IaaS: Amazon EC2, Google Compute Engine

Definition of Cloud Computing

  • NIST Definition
    • Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Utility Computing

  • Concept
    • Computing as the 5th utility (after water, electricity, gas, and telephony).
    • Applications and computing resources delivered as a service over the Internet.
    • Pay-as-you-go model.
    • Provided by the hardware and system software hosted in data centers.

Visions of Cloud Computing

  • Key Aspects
    • The illusion of infinite computing resources available on demand.
    • The elimination of an up-front commitment by Cloud users.
    • The ability to pay for use of computing resources on a short-term basis as needed.

Amazon EC2 Example

  • 1 instance runs 1000 h = 1000 instances run 1 h

Traditional vs. Cloud Approach

  • Traditional (Pre-Cloud Era)

    • Opening a startup and needing 100 servers requires significant upfront investment and management.

  • With Cloud

    • A consumer can unilaterally provision computing capabilities, such as servers and network storage, as needed automatically without requiring human interaction with each service provider.
    • Cloud computing makes the underlying technology, beyond the user device, almost invisible.
    • Always-on services.

  • Advantages for Consumers

    • Flexible.
    • Minimal overhead.
    • Quick and easy.

Elasticity

  • Elastic Demands

    • The 100 servers are only needed in peak time.

  • Provisioning Models

    • Provisioning for peak load: Results in unused resources during off-peak times.
    • Underprovisioning: Leads to potential loss of service during peak times.

Cloud Provisioning on Demand

  • Pay for what you used.

Animoto Case Study

  • Animoto on Amazon EC2
    • Scaled to a peak of 3,500 instances in 3 days during the launch of a Facebook modification.

Cloud Economics

  • Decision Making

    • Profit from cloud >= profit from in-house infrastructures.

  • Equation

    • UserHours{cloud} \times (revenue - Cost{cloud}) > UserHours{datacenter} \times (revenue - Cost{datacenter} \times Utilization)

Resource Pooling

  • Provider's Perspective

    • Resources are pooled to serve consumers using a multi-tenant model.
    • Different physical and virtual resources dynamically allocated according to consumer demand.
    • Creates an illusion of an infinite amount of resources.

  • Location Independence

    • The customer generally has NO control or knowledge over the exact location of the provided resources.
    • But may be able to specify location at a higher level of abstraction (e.g., country, state, or data center).

  • High Utilization

    • Resource pooling enables high utilization.

Economy of Scale

  • Cost Comparison (2006)

    • Medium-sized datacenter (~1k servers) vs. a very large datacenter (~50k servers).

  • Economies of Scale

    • Network: 95perMbit/sec/month(medium)vs.95 per Mbit/sec/month (medium) vs.13 per Mbit/sec/month (large)
    • Storage: 2.20perGByte/month(medium)vs.2.20 per GByte / month (medium) vs.0.40 per GByte / month (large)
    • Administration: ~140 Servers / Administrator (medium) vs. >1000 Servers / Administrator (large)

  • Electricity Price

    • Idaho: 3.6¢ per KWH (Hydroelectric power; not sent long distance)
    • California: 10.0¢ per KWH (Electricity transmitted long distance over the grid; limited transmission lines in Bay Area; no coal fired electricity allowed in California).

Statistical Multiplexing

  • Concept
    • Multiple users sharing the same resources over time.

Profitability

  • Cloud Providers
    • Cloud business is highly profitable.

Strategic Advantages of Cloud

  • Examples
    • Leverage existing investment (e.g., Amazon).
    • Defend a franchise (e.g., Microsoft Azure).
    • Attack an incumbent (e.g., Google Cloud Platform).
    • Leverage customer relationships (e.g., IBM).
    • Become a platform (e.g., Facebook, Apple, etc.).

Summary: Why Cloud?

  • Advantages
    • Better capital utilization.
    • Elasticity, easy to scale up and down.
    • Access to complex infrastructure and resources without internal resources.
    • Providers: better resource utilization, lower cost.

Cloud Pricing

  • Fundamental Drivers of Cost
    • Compute (EC2): Charged per hour/second, varies by instance type.
    • Storage (S3, EBS): Charged typically per GB w/ tiered pricing.
    • Data transfer: Outbound is aggregated and charged, typically per GB; inbound has no charge (w/ some exceptions).

Instance Pricing Examples (Amazon EC2)

  • US East (N. Virginia)

    • t2.nano: 0.0059 per Hour
    • t2.micro: 0.012 per Hour

  • Asia Pacific (Tokyo)

    • t2.nano: 0.008 per Hour
    • t2.micro: 0.016 per Hour

Location Matters

  • Factors Affecting Pricing
    • Cooling cost.
    • Manpower cost.
    • Land price.
    • Policy issues.

Diverse Pricing Options

  • On-demand.
  • Reservation-based.
  • Spot pricing.

Reserved Pricing

  • Concept

    • Pay an up-front reservation fee to reserve an instance for a long period (e.g., 1 to 3 years).
    • Enjoy a significant discount during the reservation period.

  • Cost Equation

    • Cost(t)=U+discount×R×tCost(t) = U + discount \times R \times t
    • U = Upfront cost
    • R = On-demand rate

  • Guaranteed Availability

    • Users signed up for the reserved pricing are always serviced, regardless of the DC load.
    • Not possible for on-demand pricing.

Spot Pricing

  • Concept

    • Used to be an auction-like pricing option.
    • Users submit bid for instance acquisition.
    • Cloud posts a spot price periodically.
    • Users with a higher bid than the spot price wins.
    • The spot price is applied until a new one is posted.
    • Running users with a lower bid get their instances terminated.

  • Characteristics

    • Spot price is usually much cheaper than on-demand.
    • No service guarantee.
    • Running spot instances get terminated when the spot prices rises above the bid.

Summary of Pricing Models

FeatureOn-demandReservedSpot
Service GuaranteeNoneHighО
Cost

Market Segmentation

  • Reserved Pricing

    • Locks in long-term users.
    • Helps predict future demand: better for capacity planning.

  • On-Demand

    • The fundamental cloud business model.

  • Spot Pricing

    • Leftover capacity on sale: increase utilization.

Provider's Problems

  • Challenges
    • Datacenter has a limited capacity.
    • How to allocate the capacity for each pricing model?
    • If not planned well, one model can cannibalize the other.
    • How to set the price of each model?

User's Problems

  • Challenges
    • How to cut down the cloud bill by combining different pricing models?
    • Demand/workload prediction.
    • Predict spot price.
    • Creative use of spot instances (periodic checkpointing and recovery upon instance revocation).
    • Save over 50% compared with on-demand.

The Rise of Brokerage Service

  • Cloud Brokerage Service
    • Helps users to make instance acquisition strategies.
    • Trade-in unused instances in a secondary cloud marketplace.
    • Hybrid cloud: connects to multiple cloud providers to explore the best deal.
    • Many innovative business models coming…