06 - Big Data and Cloud
Page 1: Introduction
Overview of Cloud and Big Data presentation
Presenter: Ivika Jäger
Date: November 3, 2024
Page 2: What is THE CLOUD?
Definition: A data warehouse accessed over the internet.
Page 3: Cloud Types
Private Cloud:
Dedicated to a single organization.
Maximum security.
High cost.
Public Cloud:
Shared services offered to the general public.
Affordable and accessible.
Hybrid Cloud:
Combines private and public cloud elements.
Sensitive data stored on private servers; public platforms used for analytics.
Page 4: Network vs. Internet
Traditional data warehousing (DW) accessed via network connection.
A network can be closed and not connected to the internet.
The Cloud is always accessed over the internet.
Page 5: Cloud Computing
Definition: Computing services delivered over the internet.
Resources: Storage, servers, networking, databases, and software.
Model: Businesses and individuals can rent the resources they need (pay-as-you-go model).
Example: Amazon Web Services (AWS).
Page 6: Differences Between Traditional and Cloud Data Warehousing
Feature | Traditional Data Warehousing | Cloud Data Warehousing |
Location | On-premises, within the organization's data centers | Hosted on cloud platforms (e.g., AWS Redshift, Snowflake) |
Control | Fully controlled by the organization | Managed by cloud provider, reducing admin overhead |
Scalability | Limited by hardware investments | Easily scalable on-demand |
Costs | High upfront costs for infrastructure and maintenance | Pay-as-you-go model |
Access | Limited to internal networks | Accessible globally via the internet |
Page 7: Future of Cloud and Traditional DW
Regulations and Compliance:
Laws may require sensitive information to be stored on-premises.
High Sensitivity:
Proprietary data is safer under in-house control.
Risk Factors:
Data breaches, insider threats, dependency on third-party providers.
Performance Requirements:
Traditional DW systems may outperform cloud-based systems.
Page 8: Cloud Computing Models
Infrastructure as a Service (IaaS):
Renting land; complete control of everything on the land.
Platform as a Service (PaaS):
Renting a workshop; tools and space are pre-configured.
Software as a Service (SaaS):
Renting a fully furnished hotel room where everything is ready to use.
Page 9: Key Differences in Cloud Models
Feature | IaaS | PaaS | SaaS |
What's Provided | Virtual machines, storage, network | Development tools, frameworks | Fully functional applications hosted by the provider |
User Control | Full control over OS and software | Focus on application development | Limited to using application's features |
Management | Hardware, virtualization | Hardware, OS, middleware | Everything (hardware, software, updates) |
Best For | IT teams managing environments | Developers creating apps quickly | End-users needing ready-to-use applications |
Page 10: IaaS Example
Use Case:
Useful for startups needing scalable server capacity.
Example: Netflix on AWS: rents servers and optimizes capacity.
Page 11: PaaS Example
Case Study:
Linas Matkasse builds on Microsoft Azure App Services.
Utilizes Azure products for various services including app services and analytics.
Page 12: Examples of SaaS Companies
Popular SaaS Applications:
iCloud, Microsoft 365, Zoom, Dropbox, Salesforce, Google Docs.
Page 13: SaaS Characteristics
Cloud-based, subscription model
Centralized updates; no installation required
Examples: Gmail, ChatGPT, Microsoft 365, Dropbox.
Page 14: BI on the Cloud
Benefits:
Accessibility: available anytime, anywhere.
Scalability: scale resources based on needs.
Cost-efficiency: no upfront investment, pay-as-you-go.
Inclusivity: affordable AI, big data, and analytics for small businesses.
Page 15: Analytics as a Service (AaaS)
Provides advanced analytics capabilities without infrastructure investment.
Examples: Amazon Quicksight, Tableau Cloud, PowerBI (Cloud version).
Page 16: Data Measurement Overview
Unit Measurements:
From Kilobyte (KB) to Yottabyte (YB) in data storage.
Page 17: When Data Becomes Big
Data types and sizes vary considerably, emphasizing the sheer volume of big data.
Page 18: Data Creation Trends
Yearly data creation visualization across regions.
Page 19: Characteristics of Big Data
Volume: Rapid data accumulation.
Variety: Types of data: structured, semi-structured, unstructured.
Velocity: Speed of data creation and processing.
Page 20: New Technologies
Traditional methods struggle with big data due to the need for real-time processing.
Tools: Hadoop, Apache Spark for distributed storage and analytics.
Page 21: Location Data Usage
Insights into consumer behavior, retail trends, agriculture, and crime analysis.
Page 22: Generating Insights
Analyzing location data for market trends and customer experiences.
Page 23: Satellite Data Utilization
Usage of remote sensing and data for monitoring and analysis.
Page 24: Social Media Insights
Analyzing user interactions to inform business strategies.
Page 25: Financial Data Analytics
Processing large volumes of financial transactions for trend forecasting and risk management.
Page 26: Sensor Data and IoT
Devices tracking various physical phenomena for monitoring and analysis.
Page 27: Internet of Things (IoT)
IoT creates vast amounts of data through connected devices and sensors.
Page 28: IoT Infrastructure
Components include sensors, connectivity, and software for data management.
Page 29: Smart Home Example
Integrating automation through sensors for everyday activities.
Page 30: IoT Applications in Industries
Enhancing efficiency and decision-making across various sectors.
Page 31: Challenges and Opportunities
Complexity of data analysis and security concerns.
Page 32: The Role of 5G in IoT
Accelerating IoT capabilities through improved connectivity.
Page 33: Integration of AI in IoT
Combining smart sensors with AI for enhanced analysis.
Page 34: Summary of Key Points
Cloud computing enables scalable storage and advanced data analytics through AI.
Outlook: A future of cloud-based, AI-driven data management and decision-making.