COGS9_8_EDA

Page 1: Introduction

  • Course Title: Exploratory Data Analysis (COGS 9)

  • Instructor: Dominic Burrows, PhD

  • Institution: UCSD Cog Sci

  • Contact Information: dburrows@ucsd.edu | Twitter: @dominicburrows_

Page 2: Course Announcements

  • Quiz 3: Due Friday at 11:59 pm.

  • Assignment 1: Due next Friday, February 7th.

  • Final Project: Group project; part 1 due February 14th.

  • Feedback: Anonymous feedback link provided.

Page 3: Cognitive Science Board Event

  • Event: Networking event on January 31st from 4-6 pm in Price Center East Ballroom.

  • Highlights: Free food, opportunity to network with alumni.

  • RSVP: Required for attendance.

Page 4: Class Question Sign Up

  • Platform: ClassQuestion (lecture Q&A).

  • Link: Access via provided URL.

  • Entry Code: RQFKP (must sign in with UCSD email).

Page 5: Learning Objectives

  • Understand:

    • The purpose of EDA.

    • Key steps in EDA process.

Page 6: EDA Process Overview

  • Key Steps: Start with a question, clean the data, visualize, model, predict, analyze & conclude, share insights.

  • Process Flow: Visual representation of EDA steps, emphasizing exploration and refinement.

Page 8: Definition of EDA

  • Exploratory Data Analysis (EDA): Initial phase of data analysis focused on examining data structure, contents, and patterns without directly answering the primary data question.

  • Purpose: Identify patterns to guide further analysis and model refinement.

Page 9-11: EDA Skills & Components

  • Key Skills:

    • Hacking skills

    • Machine learning understanding

    • Data processing, mathematical/statistical knowledge

  • Visualizing Data: Essential for understanding trends and insights.

Pages 12-18: EDA Analogy

  • Analogy: Exploring British culture before visiting the UK.

    • Aim: Obtain an authentic experience; gather insights from key locations.

    • Planning: Identifying relations between locations and preparing a travel plan.

Page 20-21: Importance of EDA

  • Objective: "Get a feel for the dataset" to avoid errors and refine analysis.

  • Key Benefits:

    • Identifies data errors and relationships between features.

    • Reduces noise and aids in hypothesis refinement.

Page 22-24: EDA Case Study

  • Example: Study on the effects of seizures on zebrafish by Ahrens et al.

  • Methods: Visually inspect data, identify seizure time points, inspect distributions.

Pages 25-30: Key EDA Steps

  • Steps:

    • Explore data distributions

    • Look at measures of central tendency and variability

    • Perform statistical tests

    • Investigate relationships between features

  • Understanding Distributions: Can help in refining models and analyzing relationships.

Pages 31-36: Understanding Distributions

  • Spread of Data: Explains why it's essential to know data distribution types (e.g., Normal, Poisson).

  • Use Cases: Identifying data characteristics helps in statistical modeling and inference.

  • Key Theorems: Discusses the implications of the Central Limit Theorem.

Pages 37-42: More on Distributions

  • Different Distribution Types: Uniform, Exponential, Bernoulli, etc.

  • Applications: Understanding the distribution helps in choosing statistical tests.

Pages 43-49: Implications of Distribution Understanding

  • Statistical Tests: Using the appropriate test based on data distribution enhances analysis accuracy.

  • Modeling for Prediction: Different distributions guide the type of models used in predictions.

Pages 50-53: Modeling for Prediction and Inference

  • Modeling Types: Linear regression for normal data, logistic regression for categorical data, Poisson regression for count data.

  • Importance: Accurate modeling is crucial for both prediction and understanding data relationships.

Page 54-56: EDA Conclusion and Key Steps

  • Key Steps Recap: Emphasizes exploration, central tendency, variability, statistical tests, and relationships.

  • Class Engagement: Encouragement for participation and discussion.

Pages 57-62: Central Tendency and Variability

  • Measures of Central Tendency: Mean, Median - Importance in summarizing data.

  • Variability: Understanding diversity among data points to enhance confidence in sample data predictions.

Pages 63-70: Understanding Variability

  • Variability: Defines measuring variability and its role in understanding dataset differences.

  • Confidence Intervals: Provides ranges containing true population mean with a specified degree of confidence.

Pages 71-76: Statistical Tests Importance

  • Statistical Tests: Emphasizes their role in validating hypotheses and analyzing effects.

Pages 77-92: Refinement of Hypotheses

  • Hypothesis Refinement: Specific to domains like COVID-19; understanding differences in expression levels by gender.

  • Conclusions from EDA: Data-driven insights that guide further exploration and hypothesis testing.