03_Types_of_Data_and_DataTypes_annotated

INTRODUCTION TO DATA SCIENCE

  • Overview of data science

  • Importance of data types in data analysis

Types of Data, Data Types and Data Categories

Recap of Last Week

  • Code snippet for loading a dataset:

    • df = pd.read_csv('/kaggle/input/california-housing-prices/housing.csv')

    • Sample data includes features like longitude, latitude, and housing prices:

      • Example Rows:

        • Row 0: -122.23, 37.88, 41.0, ...

        • Row 1: -122.22, 37.86, 21.0, ...

What is Data?

  • Definition: Data refers to raw information, facts, or statistics in various forms (numbers, text, images, etc.).

Broad Categories of Data

  • Quantitative

    • Discrete: Countable set of values (e.g., number of cars)

    • Continuous: Infinite values within a range (e.g., height, weight)

  • Qualitative: Descriptive data which can be further classified into structured and unstructured data.

Types of Data

  • Tabular Data: Structured data formatted in rows and columns, resembling spreadsheets.

  • Text Data: Human-readable text (e.g., reviews, articles).

  • Graph Data: Represents relationships and connections among entities (e.g., social networks).

  • Unstructured Data: Lacks a predefined structure, includes video, audio, images, etc.

Examples of Data Types

  • Tabular Data: Demographic info, grades.

  • Text Data: Emails, social media posts.

  • Graph Data: Roads, social connections.

  • Unstructured Data:

    • Videos: TikTok, face recognition

    • Images: Photos, road signs

    • Audio: Music, real-time translation

    • Biometrics: Fingerprints, facial recognition

Data Formats

  • Common Formats:

    • CSV, TSV: Type of structured data formats used for storage.

    • Image formats: .jpg, .png

    • Audio formats: .wav, .mp3

    • SQL databases: mySQL, PostgreSQL

    • NoSQL databases: Bigtable, MongoDB

  • CSV vs. TSV: Both are used for tabular data but differ by how they separate values (comma vs. tab).

Image Data

  • Structure: Organized as grids of pixels; color information in RGB channels (0-255).

  • Compression Types:

    • Lossy Compression: Reduces file size with some quality loss (e.g., JPEG).

    • Lossless Compression: Maintains original quality (e.g., PNG).

Databases

  • Databases: Organized collections of structured data, enabling efficient data management and retrieval.

JSON Format

  • JSON (JavaScript Object Notation): A lightweight format for data interchange, easily readable and writable for both humans and machines.

  • Structure: Uses key-value pairs and can represent various data types including strings, arrays, and objects.

XML/HTML

  • HTML: Used for creating web pages with a fixed set of tags.

  • XML: Designed for transporting and storing data, allowing for custom tags.

Data Acquisition

  • Sources for data include company data, databases, the internet, and RESTful APIs.

Beautiful Soup for Parsing HTML

  • A Python library used for web scraping to extract data from web pages.

RESTful APIs

  • Tools providing a structured way to access data from services, allowing for communication between applications.

Data Types

  • Overview to categorize data efficiently to facilitate analysis.

Summary

  • Types of data (tabular, text, graph, unstructured) are crucial in the data science lifecycle.

  • Understanding file formats (e.g., CSV, JSON) aids in data preparation.

  • Databases play a central role in data management.

  • Data acquisition methods like RESTful APIs and web scraping are vital for initial data gathering.