Data Science Methodology Practice Flashcards

Overview of Data Science Methodology

• Definition: Data Science Methodology is a process with a prescribed sequence of iterative steps that data scientists follow to approach a problem and find a solution.

• Purpose: It provides a framework for designing AI projects, helping teams decide on methods, processes, and strategies to obtain correct outputs efficiently.

• Origin: Developed by John B. Rollins, a Data Scientist at IBM Analytics.

• Framework Structure: Comprises 10 steps organized into 5 core modules.

• Learning Approach: Professional implementation involves hands-on work, team discussions, web searches, and case studies.

• Prerequisites: Understanding of AI concepts from Class XI and familiarity with Capstone Projects.

• Iterative Nature: The methodology is not strictly linear; steps (such as data collection) can be revisited based on findings in later stages (such as data understanding).

Module 1: From Problem to Approach

Step 1: Business Understanding

• Key Question: What is the problem that you are trying to solve?

• Alternative Name: Problem Scoping and Defining.

• Core Objective: Understand customer needs through relevant questions and stakeholder discussions.

• Key Activities:

  • Identify specific requirements and create a comprehensive list of business needs.

  • Use the 5W1H Problem Canvas (Who, What, Where, When, Why, How) to deeply understand the issue.

  • Apply the Design Thinking (DT) Framework.

  • Define objectives that clearly support the customer's end goal.

Step 2: Analytic Approach

• Key Question: How can you use the data to answer the question?

• Process: Once the problem is established, the data scientist defines the type of analytics needed.

• Analytics Classification Table:

  • Descriptive Analytics: Asks "What happened?" Uses graphs, charts, mean, median, mode, and variance. Example: Average marks of students in an exam.

  • Diagnostic Analytics: Asks "Why did it happen?" Uses root cause analysis, hypothesis testing, and correlation. Example: Why company sales dropped.

  • Predictive Analytics: Asks "What will happen?" Uses regression, classification, and clustering. Example: Forecasting future sales based on past data.

  • Prescriptive Analytics: Asks "What should we do?" Uses optimization, simulation, and decision analysis. Example: Strategy to increase festival season sales.

• Algorithm Selection Guide:

  • Finding "how much/how many": Regression.

  • Identifying a category: Classification.

  • Grouping data: Clustering.

  • Identifying unusual patterns: Anomaly Detection.

  • Finding items for customers: Recommendation.

Module 2: From Requirements to Collection

Step 3: Data Requirements

• Key Question: What are the data requirements?

• Determinants: The analytical approach dictates the data needed.

• Tasks:

  • Identify data types (numbers, words, images).

  • Determine the structure (tables, text files, databases).

  • Identify potential data sources.

  • Plan cleaning and organization steps.

• Data Types:

  • Structured Data: Organized in tables, rows, and columns (e.g., spreadsheets, databases).

  • Unstructured Data: No predefined structure (e.g., images, videos, social media posts).

  • Semi-Structured Data: Contains some organization but is not fully tabular (e.g., XML files, JSON, emails).

Step 4: Data Collection

• Key Question: What occurs during data collection?

• Definition: The systematic process of gathering observations or measurements.

• Source Types:

  • Primary Data Source: Original firsthand data collected through surveys, interviews, experiments, IoT sensors, and feedback forms.

  • Secondary Data Source: Pre-existing data from books, journals, websites, and databases.

  • Online Source Examples: data.gov, World Bank, UNICEF, Kaggle, WHO, Google.

• Strategic Flexibility: This stage is often revisited after the Data Understanding stage if gaps are identified.

Module 3: From Understanding to Preparation

Step 5: Data Understanding

• Key Question: Is the data collected representative of the problem to be solved?

• Techniques:

  • Descriptive Statistics: Univariate analysis and pairwise correlation.

  • Visualization: Histograms and scatter plots.

  • Evaluation: Checking for relevance, comprehensiveness, and suitability of the data.

Step 6: Data Preparation

• Key Question: What additional work is required to manipulate and work with the data?

• Nature: It is considered the most time-consuming step in the entire methodology.

• Core Activities:

  • Data Cleaning: Dealing with invalid or missing values, removing duplicates, and assigning suitable formats.

  • Data Integration: Combining data from multiple platforms, tables, or archives.

  • Data Transformation: Converting data into meaningful input variables.

  • Feature Engineering: The process of selecting, modifying, or creating new features from raw data to improve model performance.

• Feature Engineering Example: To predict house prices from Area ($sq.\,ft$), Bedrooms, and Year built:

  • New Feature 1: $Age\,of\,house = Current\,year - Year\,built$

  • New Feature 2: $Price\,per\,sq.\,ft = Price\,of\,house / Area$

Module 4: Modelling to Evaluation

Step 7: AI Modelling

• Key Question: In what way can the data be visualized to get to the required answer?

• Modelling Types:

  • Descriptive Modelling: Summarizes characteristics of the dataset WITHOUT making predictions. It focuses on "what is happening."

  • Predictive Modelling: Uses data to make predictions about future outcomes using techniques like regression and classification.

• Descriptive Techniques: Summary statistics including Mean, Median, Mode, Standard Deviation, Variance, Range, and Percentiles.

Step 8: Model Evaluation

• Key Question: Does the model answer the original question satisfactorily?

• Validation Methods:

  • Train-Test Split: The dataset is divided into a training set (commonly $80\%$) and a testing set (commonly $20\%$). The training set builds the model; the test set evaluates it.

  • K-Fold Cross Validation: Data is divided into $k$ equal subsets (folds). The model trains on $k-1$ folds and tests on the remaining fold, repeating this process $k$ times. Every data point serves as both training and testing data at different points.

• Key Notes:

  • There is no single optimal split percentage, though $80/20$ is standard.

  • Cross-validation is preferred for smaller datasets to ensure reliability.

Module 5: Deployment to Feedback

Step 9: Deployment

• Activity: Rolling out the model into a production environment where stakeholders can use it regularly.

Step 10: Feedback

• Activity: Collecting feedback from users and monitoring real-world performance.

• Loop: Feedback initiates iterative cycles, potentially sending the data scientist back to any previous stage of the methodology (e.g., gathering more data or redefining requirements).

Model Performance Evaluation - Classification Metrics

1. Confusion Matrix

• A table used for binary classification ($N=2$) to evaluate performance.

• Key Terms:

  • True Positive (TP): Model predicted YES; Actual was YES.

  • True Negative (TN): Model predicted NO; Actual was NO.

  • False Positive (FP): Model predicted YES; Actual was NO (Type I Error).

  • False Negative (FN): Model predicted NO; Actual was YES (Type II Error).

2. Classification Formulas

• Precision: Measures the accuracy of positive predictions.

  • $\text{Precision} = \frac{TP}{TP + FP}$

• Recall: Measures how many actual positives were correctly identified.

  • $\text{Recall} = \frac{TP}{TP + FN}$

• F1 Score: The harmonic mean of Precision and Recall. Ideal value is $1.0$.

  • $\text{F1\,Score} = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$

• Accuracy: The overall percentage of correct predictions.

  • $\text{Accuracy} = \frac{TP + TN}{TP + FP + FN + TN}$

Model Performance Evaluation - Regression Metrics

• MAE (Mean Absolute Error): Sum of absolute differences between predicted and actual values. A value of $0$ represents perfect predictions.

  • $\text{MAE} = \frac{\sum |\text{Actual} - \text{Predicted}|}{n}$

• MSE (Mean Squared Error): Mean of squared distances between predicted and actual values. It penalizes large errors more heavily than small ones and is the most commonly used metric.

  • $\text{MSE} = \frac{\sum (\text{Actual} - \text{Predicted})^2}{n}$

• RMSE (Root Mean Squared Error): The square root of MSE. It is preferred for interpretation because it is in the same units as the target variable.

  • $\text{RMSE} = \sqrt{\text{MSE}}$

Practical Activities and Code Implementation

Activity 4.1: Regression Calculation

• Dataset (Actual): $17, 18, 18, 15, 18, 11, 20, 18, 13, 19$

• Dataset (Predicted): $14, 19, 17, 13, 12, 7, 24, 23, 17, 18$

• Calculation Logic:

  • Total Squared Residuals: $9 + 1 + 1 + 4 + 36 + 16 + 16 + 25 + 16 + 1 = 125$

  • $n = 10$

  • $\text{MSE} = \frac{125}{10} = 12.5$

  • $\text{RMSE} = \sqrt{12.5} \approx 3.54$

Activity 4.2: Confusion Matrix Calculation

• Input Data: $TP = 35$, $TN = 50$, $FP = 10$, $FN = 5$.

• Results:

  • Precision: $\frac{35}{35 + 10} = 77.8\%$

  • Recall: $\frac{35}{35 + 5} = 87.5\%$

  • F1 Score: $82.3\%$

  • Accuracy: $\frac{35 + 50}{100} = 85\%$

Activity 4.3: Python Implementation Highlights

• Uses pandas for data loading and numpy for array manipulation.

• train_test_split from sklearn.model_selection requires parameters: test_size=0.2 (for an $80/20$ split) and shuffle=True (to randomize data).

• LinearRegression from sklearn.linear_model is used to fit the model.

• Evaluation uses model.score() for training/testing scores and mean_squared_error() from sklearn.metrics.

Quick Revision - Key Formulas & Memory Aids

10 Steps Mnemonic: BAD-UP-MEDF

• B: Business Understanding

• A: Analytic Approach

• D: Data Requirements

• U: Data Understanding

• P: Data Preparation

• M: AI Modelling

• E: Evaluation

• D: Deployment

• F: Feedback

• (Note: Data Collection sits between Requirements and Understanding)

Analytics Quick Reference

• Descriptive: What happened?

• Diagnostic: Why did it happen?

• Predictive: What will happen?

• Prescriptive: What should we do?