Artificial Intelligence Project Cycle, Ethics, and Data Modelling

Problem Scoping

• Definition: Problem Scoping is the initial process of identifying and defining a specific problem that is intended to be solved using Artificial Intelligence (AI).

• Key Steps in Problem Scoping:

  • Selecting a Broad Theme: Choosing a broad area or domain such as Environment, Education, Cyber Security, Infrastructure, or Healthcare.

  • Focusing on a Specific Topic: Narrowing down the selected broad theme into a clear, targeted subject.

  • Listing Related Problems: Identifying and detailing underlying or associated issues to gain a comprehensive understanding of the problem space.

The 4Ws Problem Canvas

• Definition: The 4Ws Problem Canvas is a structured framework designed to guide practitioners step-by-step through the problem scoping process by addressing four core questions: Who, What, Where, and Why.

• 1. Who?:

  • Focuses on identifying all stakeholders who are directly or indirectly affected by the problem.

  • Key Stakeholder Examples: Government officials, Landowners, Farmers, Local residents, Construction companies, and Environmental groups.

• 2. What?:

  • Focuses on defining the precise nature of the issue and gathering evidence that proves its existence.

  • Example: Identifying that "Garbage is not being collected on time", supported by evidence such as resident complaints, dirty streets, and foul odors.

• 3. Where?:

  • Identifies the specific context, situation, or physical location where the problem occurs or is most prominent.

  • Example: "The problem happens in crowded market areas where garbage bins overflow."

• 4. Why?:

  • Focuses on understanding the specific benefits of the solution for the target stakeholders, establishing the direct value proposition.

  • Example: "Solving it will keep the city clean, improve health, and make life better for residents."

Problem Statement Template

• Definition: The Problem Statement Template formalizes and summarizes the insights gained during the 4Ws Problem Canvas phase into a single, cohesive, fill-in-the-blanks statement.

• Standard Sentence Structure:

  • "Our AI project will help [Who] who have the problem of [What] in [Where] by [Why]."

• Core Components Breakdown:

  • Our: [stake holder(s)] (Addresses the Who)

  • Has / Have a problem that: [issue, problem, need] (Addresses the What)

  • Context / Location: [context, situation] (Addresses the Where)

  • An ideal solution would be: [benefit of solution for them] (Addresses the Why)

Iterative Nature of Problem Scoping

• Definition: Problem scoping is inherently iterative, requiring continuous refinement of the design or AI model through repeated cycles of prototyping, testing, and feedback until the final objective is met.

• Key Advantages of the Iterative Approach:

  • Continuous Improvement: Each iterative cycle enables developers to correct flaws and build upon findings from previous cycles.

  • Cost Efficiency: Catching and addressing design flaws early prevents costly full-scale rework.

  • Simplified Testing and Debugging: Smaller, initial iterations isolate components, making debugging and validation significantly easier.

Data Acquisition and Management

• Definition of Data Acquisition: The second stage of the AI project cycle, involving the collection of raw facts, figures, statistics, or visual assets from relevant sources for model analysis and training.

• Challenges: Data acquisition is time-consuming because relevant raw data is typically scattered across multiple sources and requires careful selection and curation to align with project requirements.

• Key Principles for Data Acquisition:

  • Timeliness and Currency: Data must be collected regularly to ensure it reflects contemporary trends and dynamics.

  • Completeness and Sufficiency: Datasets must be comprehensive enough to enable accurate predictions. Incomplete or unreliable data leads to faulty and misleading AI outputs.

• Training Data vs. Testing Data:

  • Training Data: Dataset utilized to train the AI model and fit its parameters, allowing the machine to learn patterns and underlying rules.

  • Testing Data: Independent dataset reserved exclusively to evaluate the final model's performance and generalization ability post-training. It is never shown to the model during training.

  • Example: To predict student marks, historical student performance data from prior academic years is used as training data. Predictions are then tested using new, unseen student data as testing data.

• Reliable Data Sources:

  • Surveys: Collecting targeted quantitative or qualitative data directly from individuals or groups using questionnaires.

  • Web Scraping: Automated extraction of structured or unstructured data from websites for analysis.

  • Sensors: Gathering real-time physical or environmental parameters (e.g., temperature, motion, pressure).

  • APIs (Application Programming Interfaces): Extracting structured data directly from software platforms and social media services.

  • Observations: Directly tracking and recording real-time behavioral data or physical occurrences as they happen.

  • Cameras: Capturing visual media (such as CCTV footage) for surveillance, tracking, and pattern recognition.

• Data Features:

  • Definition: The specific characteristics, attributes, or parameters within a dataset that influence the problem being solved.

  • Role: Dictate the exact properties and parameters required during data collection for model training and testing.

  • Examples:

  • Example 1: Average marks of students.

  • Example 2: Total number of students taking an examination.

System Maps

• Definition: A visual diagram representing all individual elements within a system and detailing the precise cause-effect interconnections between them.

• Functionality: Helps visualize complex systemic relationships, dependencies, feedback loops, and root causes of problems to support effective problem-solving.

• Notation and Structure:

  • Arrows: Indicate the direction of influence from one element to another.

  • Plus Sign (+): Denotes a direct relationship where both variables move in the same direction (e.g., both increase or both decrease together).

  • Minus Sign (-): Denotes an inverse relationship where an increase in one variable leads to a decrease in the other.

  • Example: "Driving" shares an inverse relationship with "Safe Streets", represented by an arrow marked with a - sign.

Data Visualisation

• Definition: The graphical representation of data using charts, graphs, maps, and diagrams to expose underlying patterns, trends, and correlations.

• Core Purpose: Simplifies high-volume, complex numerical data into intuitive, human-friendly representations to support decision-making.

• Primary Reasons for Data Visualisation:

  • Simplifies Complex Data: Converts vast, intricate datasets into accessible visual formats.

  • Explores Trends: Highlights anomalies, patterns, clusters, and outliers quickly.

  • Enhances Communication: Transmits insights clearly to diverse stakeholders.

• Benefits across the AI Project Cycle:

  • Rapid Trend Identification: Allows immediate spot-checking of data patterns.

  • Informed Model Selection: Helps practitioners select appropriate machine learning algorithms based on visual data distribution.

  • Easier Comprehension: Eliminates the cognitive load of reading raw numerical spreadsheets.

  • Effective Stakeholder Engagement: Simplifies complex statistical relationships for non-technical audiences.

• Data Visualisation Tools:

  • Microsoft Excel: A traditional desktop application suitable for basic, manual plotting of small datasets. Inadequate for big data handling.

  • Tableau Public: A free, advanced visualisation tool capable of rendering interactive dashboard displays for massive datasets.

• Comparative Visualisation Formats:

  • Bar Chart vs. Line Chart:

  • Bar Chart: Uses rectangular bars to compare discrete categorical groups or discrete temporal changes (e.g., vehicle sales broken down by vehicle type).

  • Line Chart: Uses continuous line segments connecting data points to plot continuous quantitative trends over time (e.g., national literacy rates over several decades).

  • Pie Chart: A circular chart divided into proportional slices representing percentage components of a whole ($100\%$). Ideal for relative market share (e.g., tea brand distribution), grade breakdowns, or item sales breakdowns.

  • Bubble Map: A spatial combination of a geographic map and a bubble plot where circle surface areas represent quantitative values located geographically.

  • Utility: Displays geographic phenomena (e.g., earthquake hotspots, regional wildlife populations, regional revenue) without suffering from land-area visual distortions.

Model Evaluation

• Definition: The stage in the AI project lifecycle where a trained model is rigorously tested against unseen testing data by measuring its predicted outputs against real-world actual results.

• Core Objectives:

  • Measures output precision and overall prediction accuracy.

  • Verifies reliability, stability, and operational consistency.

  • Assesses alignment with original project goals.

  • Exposes flaws, underfitting, or overfitting to guide model tuning and optimization.

AI Model Deployment

• Definition: The final phase of the AI project cycle where the trained, evaluated model is integrated into production environments (servers, cloud infrastructure, APIs) to process live data and serve end users.

• Three Primary Steps in Deployment:

  • 1. Validating and Testing the AI Model: Conducting final operational verification to confirm speed, stability, accuracy, and performance under production-like loads.

  • 2. Integration with Current Systems: Establishing secure connections and infrastructure linking the AI model with enterprise data pipelines, software applications, and external APIs.

  • 3. Monitoring and Maintaining the Deployed Model: Tracking live operational metrics, resolving software bugs, addressing data drift, and periodically re-training or upgrading the model.

• Importance:

  • Transforms abstract theoretical models into practical operational tools.

  • Enables practical execution within real-world environments.

  • Facilitates real-world evaluation of enterprise productivity, efficiency, and ROI.

  • Makes the AI system directly accessible and functional for end users.

AI Ethics and Governance

• Definition of AI Ethics: The set of guidelines and rules regarding right and wrong in developing, deploying, and operating artificial intelligence systems to ensure responsible decision-making.

• Role in Personal Data Handling: Mandates privacy preservation, explicit user consent, total transparency, and strong security to prevent unauthorized exploitation or data leaks.

• Ethical Development Guidelines:

  • Systems must be understandable and interpretable by non-expert users.

  • Every component of the system logic must be self-explanatory and transparent.

  • Models must be built inclusively to serve all demographic groups without discrimination.

• Four Core Principles of AI Ethics:

  • 1. Human Rights: Safeguards individual liberties, including privacy rights, freedom of expression, and protection from discriminatory profiling.

  • 2. Bias: Prevents unfair treatment across all demographic groups regardless of race, gender, socio-economic background, or religion.

  • 3. Privacy: Strictly protects individual personal data throughout data lifecycle stages.

  • 4. Inclusion: Ensures equal access to AI innovations across diverse socio-economic groups.

• Ethics vs. Morals Framework:

• Critical Ethical Concerns in AI Deployment:

  • 1. Job Loss: Widespread automation risks displacing millions of jobs. Studies project that between $400\text{--}500\,\text{million}$ jobs worldwide could be affected. (Case Study: A hotel in Japan deployed robotic staff to replace human employees but had to bring back human workers due to operational inefficiencies).

  • 2. Personal Privacy Risks: Persistent, unnoticed data collection through social networks, IoT sensors, and biometric tracking. (Case Study: Apple's Face ID system was successfully hacked using custom $3\text{D}$-printed masks shortly after release).

  • 3. Critical System Errors:

  • Autonomous navigation failure: Uber's self-driving test vehicle ran a red traffic light during operational testing.

  • Chatbot corruption: Microsoft's chatbot Tay generated offensive and toxic tweets within $16\,\text{hours}$ of going live due to unsupervised learning from Twitter interaction.

  • 4. Autonomous Weapons: AI-driven offensive military platforms ("killer robots") operating without direct human intervention raise profound moral, safety, and operational concerns.

  • 5. The Black Box Problem: High complexity in deep neural networks hides decision-making pathways within complex neural layers, rendering logic untraceable and opaque.

• Sources of AI Bias:

  • Data: Derives from unrepresentative or historically biased training datasets. (Example: Amazon's automated AI recruitment tool systematically favored male candidates over female candidates due to training on historical resumes predominantly submitted by men).

  • Algorithms: Machine learning algorithms amplify existing biases present in the dataset. (Example: Image classifiers trained on predominantly male faces suffer higher error rates when identifying female faces).

  • Developers: Developer assumptions, worldviews, and subconscious biases implicitly embedded into data collection rules, metric definitions, and objective functions.

• Prerequisites for AI Access:

  • Data availability: Access to large-scale, high-quality datasets.

  • AI abilities: Access to high-performance compute hardware for computational processing and learning.

Data Modelling Techniques

• Definition of Data Modelling: The process of designing decision-making algorithms that are trained on curated datasets (acquired during data acquisition) to recognize patterns and make predictions.

• Illustrative Example: Feeding a machine $100$ images of cars and $100$ images of cycles allows it to extract pattern features and correctly classify new, unseen images as either a car or a cycle.

• AI vs. Machine Learning vs. Deep Learning Hierarchy:

• Two Major AI Modelling Approaches:

  • 1. Rule-Based Approach:

  • Mechanism: Relies on explicit, human-coded conditional rules and facts predefined by the developer.

  • Operational Flow: Labelled Datasets → Rule-based Approach Model → Training Data → Testing Data → Output → Machine identifies the image.

  • Characteristics: Operates purely on hardcoded parameters; cannot learn or generalize from new data unless explicitly reprogrammed.

  • Example - Decision Tree:

    • Uses a tree-like conditional branching model to navigate from an initial decision node to final output outcomes.

    • Root node: The initial top-level node representing the entire undivided dataset.

    • Branching: The process of splitting a node into two or more dependent sub-nodes.

    • Decision node: An intermediate node that divides further into conditional sub-nodes.

    • Leaf node: A terminal node representing the final decision or classification (cannot be split further).

    • Parent node: Any node that branches into sub-nodes.

    • Child node: A sub-node resulting from a higher-level parent split.

  • 2. Learning-Based Approach:

  • Mechanism: Algorithms process raw, unlabelled datasets, extracting features and clustering data based on observed patterns (color, shape, size) without explicit human-coded rules.

  • Operational Flow: Unlabelled Data → Learning-based Approach Model → Output (Clustered based on color or shape).

  • Characteristics: Machine formulates its own rules dynamically; highly effective for complex data where explicit rules are hard to define; widely used in image processing and recognition.

  • Example Dataset: Grouping $1000$ unlabelled flower images into distinct clusters based on color or shape properties.

  • Example System - Pixel It:

    • A machine learning approach used mainly in computer vision applications.

    • Focuses on understanding pixels, resolution, and megapixels to process and classify images.

    • Demonstrates how computers read, quantify, process, and classify pixel-based image data.

Yes, you can write that definition. It is accurate and directly aligns with the notes. The Problem Statement Template formalizes and summarizes the insights gained during the 4Ws Problem Canvas phase into a single, cohesive fill-in-the-blanks statement for clear reference (e.g., "Our AI project will help [Who] who have the problem of [What] in [Where] by [Why]").