AI Exam

Ai Fluency

working with ai well. Deciding what you hand over, saying what you want, judging what comes back, owning the result

Delegation

Deciding which of a task belong to a person, which to a model, and which to both (fails when you hand it over to make a decision rather than just a draft)

Description

Stating the goal, context, format, constraints, and audience clearly enough to be acted on

Discernment

Reading the output like a skeptical editor: accuracy, relevance, completeness, appropriateness, bias

Diligence

Creation: Verify before you use it

Transparency: Say which parts were AI assisted before anyone asks you

Deployment: Stay accountable for where the work lands and who it affects.

What best describe what a generative model does when it answers you

Writes one pice at a time, each a plausible continuation of the text so far.

The three build stages

1. Train on text (reads a large body of text and learns the patterns in it)

2. Tune with human feedback (the ai adjusts the responses based on the ones they preferred)

3. Ship it frozen (what is learned stops there. It can’t think/teach itself new ideas)

RAG ( Retrieval Augmented Generation)

The tool searches a specific set of documents pulls the relevant passages in. It still write by prediction. It can misread a retrieved passage or fill gaps between passages with invention. It changes what the AI can see.

Fine tuning

Teaches a model how to behave. It isn’t about what’s necessarily true.

What are the four consequences of AI

1. The same prompt gives different answers

2. It invents things (Hallucination)

3. It only knows the what is in front of it ( in a long exchange early details fall out of view) a new conversation means forgetting th least one

4. Is Knowles stops at a date (you don’t know when the last time the AI has been updated so anything recent is assumed/out of date

The four stages of where the model comes from

1. The data (Before AI can learn a language, it needs to be given a huge amount of text to learn from. The Ai learns from the information and language patterns in that text. If certain perspective appear a lot, the AI may learn to favor those perspectives.

2. The training ( he AI practices predicting what words or pieces of words should come next, and its internal settings are adjusted to improve its predictions. )

3. The parameters (After training, the AI has billions of numbers called parameters. These numbers have been adjusted during training and help determine how the AI processes information and generates answers.The parameters helps the AI notice patterns)

4. Inference (Inference is when you actually use the trained Ai to get an answer.)

Three ways to allocate a task

1. Human

2. Hybrid

3. AI

Four Criteria that decides to allocate a task

1. Risk (If this is wrong and nobody catches, what’s the cost? Money, trust, legs exposure etc.)

2. Reversibility (Once this leaves your hands, is the result reversible?)

3. Need for judgment ( Does this require weighing values, reading a room, or knowing something that is not written down anywhere?)

4. Data sensitivity ( Would you have to paste in important information like customer records, salaries, private information?)

The cost of over delegating or under delegating

Overdelegating( you ship something you cannot defend.)

Underdelegating ( your waste precious time by doing easy work that AI could’ve solved)

Three shapes of Business Data

Structured: Data organized nearly into rows and columns, like a spreadsheet, with clearly defined fields that make it easy to calculate, filter, and create charts.

Semi Structured: Data that has labels or categories but doesn't follow a completely fixed format, such as customer support tickets or product catalogs. Coming data from different sources can be difficult

Unstructured: Dat that doesn't followed a predefined table format, such as emails contracts photos, audio recordings, and meeting notes. It makes up a large portion of what companies own, and AI makes it easier to analyze and extract useful information from it

Four ways Data breaks to a hard question

Incomplete: Missing data can lead to conclusions based only on the information that was recorded, giving a misleading picture of reality

Inconsistent: When different departments define the same metric differently, combining their numbers can produce incorrect results

Stale: Outdated data can lead to recommendations based on information that is no longer accurate

Skewed: When data comes mostly from people from very strong opinions, it may not represent the experiences of the average customer.

The two questions you ask before you trust the output

1. where did this data come from and who created it for what?

2. who is missing from it?

Five part description

1. Task (clearly state what you want done using a specific action verb, such as summarize, classify, rank or draft.

2. Source ( Specify what information the Ai should use so it doesn’t make up details beyond the available information)

3. Audience and purpose: Identify who will read the output and what they need it for since this affects what information matters and how it should be presented

4. Form: Define the required length, structure, and format so the output meets clear, measurable expectations

5. Constraints: State what the AI must avoid, such as inventing figures, including customer names, or making recommendations

Predictive AI

Uses historical data to predict a specific outcome, usually a number or category, that can be compared with what actually happened.

Generative Ai: Uses patterns learned from existing data to create new content, such as text, images, or audio.

Five Families inside “generative”

General chat models: Tools like ChatGPT, Claude, gemini, and Microsoft Copilot are useful drawing, summarizing, rewriting, and brainstorming but struggle with complex multi step problems

Reasoning models: Modes like Chapt GPT’s thinking mode, Claude’s extended thinking, and DeepSeek - R1 are designed for complex tasks like math, planning, and multi step analysis, they tend to be slower and more expensive

Small and open weight models: Models like Llama, Gemma, Mistral, and Owen can run on your computer, offering greater control over your data and lower costs, but they are less capable on complex, open ended tasks

Specialized media models: Tools like Whisper and Otter for transcription, and Midjourney or built in image generators for visuals, perform specific media tasks well but have a narrower range of capabilities and may raise copyright or likeness concerns

Build in assisted: Tools like Copilot in Word or Excel, Gemini in google Docs or Sheets, and Notion AI work directly within existing apps and can access relevant files, but you often have less control over which underlying AI model they use.

6 dimensions to compare models on

Capability: How well the model preforms he specific task using your actual data

Cost: How much it cost per use at your expected volume

Speed: How quickly it process results, depending on how urgently the task needs to be completed

Context: How much information it can process at once and how up to date its knowledge is

Data Handling: How your input data is stored, used, and protected, including whether it is used for training

Control: how much control you have over where the model runs and when its behavior or settings change

Ai Benchmark

A fixed public test used to compare AI models, but it cannot guarantee how well a model will perform on your specific tasks and data.

Ai out put variability

Ai responses can change between runs, so test the same input multiple times to check whether important answers stay consistent

The four failure modes

Hallucination, Sycophany, Drift, Bias

Hallucination

When AI generates false or made up information, such as facts, numbers, quotes, or sources, and presents it confidently as if it were true. This happens because Ai predicts likely response rather than always verifying information against a source. To catch hallucinations trace every citation and every number back to the source.

Sycophancy

When Ai agrees with he user’s opinions or assumptions instead of giving an accurate, objective answer. it may change a correct answer just because the user disagrees. This can happen because AI is trained to produce answers people prefer, which can reward agreement or accuracy. To avoid it, ask question neutrally without suggesting an answer, and check whether the Ai sticks to correct info when challenged.

Drift

When an AI model’s outputs change over time even though the user’s prompt stays the same. This can happen because of updates made by the AI provider or because earlier instructions get lost in long conversations. To catch drift, regularly test the model using the same set of inputs and compare its answers over time.

Ai Bias

When AI consistently favors, stereotypes, or leaves out certain groups because of patterns or gaps in its training data or input data. This can cause important information about some groups to be overlooked. To catch bias, check whose information is included or missing and test whether changing a name, language, or group changes the output.

The five step check

1. Trace

2. Recompute

3. Look for the Gap

4. Push Back

5. Scale to the stakes

Trace

Find each claim and citation in the source

Recompute

Redo every number that matters.

Look for the gap

look for the exception, group, or contrary figure

Push Back

Challenge one claim. An answer that folds under mild pressure was never solid

Scale to the stakes

The more important the AI’s answer is, the more carefully you need to check it.