Introduction to Data Analysis

Fundamentals of Data Analysis

  • Definition of Data Analysis: Data analysis is the process of taking raw data, examining it systematically, identifying underlying patterns, and leveraging those patterns to answer questions and make informed strategic decisions.

  • Estimated Study Duration: Approximately 1 hour1\,hour

  • Primary Learning Objectives:

    • Understand the core definition and scope of data analysis.

    • Comprehend the responsibilities and daily role of a data analyst.

    • Master the 5-step data analysis process.

    • Define and retain key vocabulary used in data analysis.

    • Learn how to examine data critically through an analytical mindset.

  • Example E-Commerce Purchase Dataset:

    • Raw customer data for analysis:     

      Customer Purchase Data Table

    • Dataset breakdown:

    • Customer: John | Age: 2525 | Product: Phone | Price: $500\$500 | Purchased: Yes

    • Customer: Sarah | Age: 4242 | Product: Laptop | Price: $1,000\$1,000 | Purchased: Yes

    • Customer: Mike | Age: 3131 | Product: Phone | Price: $500\$500 | Purchased: No

    • Customer: Lisa | Age: 2727 | Product: Tablet | Price: $300\$300 | Purchased: Yes

  • Target Analytical Questions for E-Commerce Data:

    • Which product sells the most?

    • What is the average customer age?

    • How much total revenue did the business generate?

    • Which customer segments are most likely to purchase?

    • Why are specific customers opting not to make a purchase?

The Five-Step Data Analysis Process

  • Step 1: Ask

    • Description: Determine and clearly define the key question or problem that needs to be solved.

    • Example Inquiry: "Why did our sales decrease this month?"

  • Step 2: Collect

    • Description: Gather all raw data necessary to investigate the core question.

    • Common Data Repositories and Sources:

    • Excel spreadsheets

    • Relational and non-relational databases

    • Websites

    • Surveys

    • Internal company enterprise systems

    • Application Programming Interfaces (APIs)

  • Step 3: Clean

    • Description: Correct structural defects, errors, and inconsistencies across the collected dataset.

    • Text Standardization Example: Converting variations such as "John", "john", and "JOHN" into a single uniform format to represent one distinct individual accurately.

    • Standard Data Cleaning Tasks:

    • Handling missing or null values.

    • Eliminating duplicate entries.

    • Fixing misformatted or incorrect dates.

    • Correcting spelling errors.

    • Verifying and correcting inaccurate numerical entries.

  • Step 4: Analyze

    • Description: Inspect the cleaned data to detect relationships, trends, and patterns.

    • Example Finding: Uncovering that overall sales dropped by 20%20\% specifically due to a severe decline in Product A sales.

  • Step 5: Communicate

    • Description: Share findings clearly and effectively with decision-makers.

    • Comparison of Analysis Value:

    • Weak Analysis: "Sales decreased."

    • High-Value Analysis: "Sales decreased by 20%20\% in August, primarily because Product A sales fell by 35%35\%"

Core Data Analysis Vocabulary

  • 1. Dataset

    • Definition: A structured collection of data formatted in rows and columns.

    • Example Dataset:     

      Dataset Structure Example

    • Header Row: Name | Age | Salary

    • Record 1: John | 2525 | $2,000\$2,000

    • Record 2: Sarah | 3030 | $3,000\$3,000

  • 2. Row

    • Definition: A single horizontal record representing one individual entry or entity within a dataset.

    • Example Row Record: | John | 25 | $2,000 |

  • 3. Column

    • Definition: A vertical field in a dataset designated for a specific category or attribute of information.

    • Example Column Field:     

      Name Column Example

    • Field Header: Name

    • Field Values: John, Sarah, Mike

  • 4. Variable

    • Definition: Any data element, property, or characteristic that can take on different values.

    • Examples: Age, Salary, Country, Product, Sales

  • 5. Metric

    • Definition: A specific numerical measurement used to track and evaluate performance or quantity.

    • Examples:

    • Revenue = $50,000\$50,000

    • Sales = 1,2001,200

    • Profit = $10,000\$10,000

    • Customers = 500500

  • 6. Insight

    • Definition: A meaningful, strategic conclusion derived from data evaluation that informs decisions.

    • Example: "Customers aged 25–3425\text{--}34 generate the highest revenue."

Practical Application: Sample Dataset Analysis

  • Primary Dataset:   

    Sample Customer Transactions Table

  • Complete Transaction Records:

    • Customer: John | Age: 2222 | Country: USA | Product: Phone | Amount: $500\$500

    • Customer: Sarah | Age: 3535 | Country: USA | Product: Laptop | Amount: $1,200\$1,200

    • Customer: Mike | Age: 2828 | Country: Canada | Product: Phone | Amount: $500\$500

    • Customer: Lisa | Age: 4141 | Country: USA | Product: Tablet | Amount: $300\$300

    • Customer: David | Age: 2525 | Country: Canada | Product: Laptop | Amount: $1,200\$1,200

  • Step-by-Step Dataset Solutions:

    • Question 1: How many customers are there?

    • Count of distinct customer rows: 5 customers5\,customers

    • Question 2: What is the total revenue?

    • Calculation: $500+$1,200+$500+$300+$1,200=$3,700\$500 + \$1,200 + \$500 + \$300 + \$1,200 = \$3,700

    • Question 3: Which product generated the most revenue?

    • Phone Revenue: $500+$500=$1,000\$500 + \$500 = \$1,000

    • Laptop Revenue: $1,200+$1,200=$2,400\$1,200 + \$1,200 = \$2,400

    • Tablet Revenue: $300\$300

    • Highest Revenue Product: Laptop ($2,400\$2,400

    • Question 4: What is the average customer age?

    • Calculation: 22+35+28+41+255=1515=30.2 years\frac{22 + 35 + 28 + 41 + 25}{5} = \frac{151}{5} = 30.2\,years

    • Question 5: Which country generated more revenue?

    • USA Revenue: $500+$1,200+$300=$2,000\$500 + \$1,200 + \$300 = \$2,000

    • Canada Revenue: $500+$1,200=$1,700\$500 + \$1,200 = \$1,700

    • Result: USA generated more revenue ($2,000\$2,000 vs $1,700\$1,700

    • Question 6: What is one interesting insight you can find?

    • Insight: Laptops generate the majority of overall revenue ($2,400\$2,400 out of $3,700\$3,700, or 64.86%64.86\%

Analytical Mindset and Questioning

  • Core Rule: Do not simply observe numbers; actively generate deeper questions about the numbers.

  • Question Transformation Examples:

    • Standard Surface Observation: "We have 1,000 customers."

    • Analytical Inquiry: "Which customers generate the most revenue?"

    • Standard Surface Observation: "Sales were $100,000."

    • Analytical Inquiry: "Which products generated the $100,000?"

    • Standard Surface Observation: "Sales decreased."

    • Analytical Inquiry: "When did sales decrease, why did they decrease, and which products were responsible?"

  • The Analytical Decision Workflow:   Data→Question→Analysis→Insight→Decision\text{Data} \rightarrow \text{Question} \rightarrow \text{Analysis} \rightarrow \text{Insight} \rightarrow \text{Decision}

Hands-On Homework Assignment

  • Dataset Creation Task: Build a custom dataset containing entries for 10 customers10\,customers

  • Required Column Structure:   

    Homework Dataset Column Schema

    • Schema fields: Customer Name, Age, Country, Product, Price

  • Required Assignment Questions:

    1. What is the total revenue?

    2. What is the average age?

    3. Which product is most popular?

    4. Which country has the highest sales?

    5. Who spent the most?

    6. What is one interesting insight?

  • Calculation Method: Calculations may be performed manually during introductory practice before learning automated calculations in software programs like Excel.

Day 1 Checklist

  • [ ] Definition of data analysis.

  • [ ] Responsibilities of a data analyst.

  • [ ] Core vocabulary: Dataset, Row, Column, Variable, Metric, Insight.

  • [ ] The 5-step analysis process (Ask, Collect, Clean, Analyze, Communicate).

  • [ ] How to formulate analytical questions about data.