Ai exam review

Week 1: AI fluency

Working with AI well; distinct from knowing how it works or coding it.

Week 1: AI literacy

Knowing how AI technology works, rather than necessarily working effectively with it.

Week 1: Delegation

Decide which work to give AI, keep human, or split.

Week 1: Description

Specify goal, context, format, constraints, and audience clearly enough for AI to act.

Week 1: Discernment

Read AI output like a skeptical editor; do not confuse confident writing with correct writing.

Week 1: Diligence

Own the final result: verify before use, disclose AI help proactively, and remain accountable.

Week 1: Delegation failure

Handing AI the final decision when you intended to delegate only the draft.

Week 1: Discernment failure

Treating fluent, confident AI text as automatically true.

Week 2: Generative AI response mechanism

Produces one plausible piece of text at a time; it does not inherently look up a verified fact.

Week 2: Does polished writing prove accuracy?

No. Well-written answers may still be false.

Week 2: Three build stages

Train on text; tune using human feedback; deploy the frozen model.

Week 2: Human feedback tuning

Adjusts model behavior toward preferred responses; can incline a model to agree.

Week 2: Frozen model

Ordinary user corrections during inference do not retrain its parameters.

Week 2: RAG

Retrieval-augmented generation: changes what relevant current or private information AI can see; can make claims traceable.

Week 2: Fine-tuning

Changes model behavior; not the preferred way to give reliable access to changing facts.

Week 2: RAG versus fine-tuning

RAG changes what AI can see; fine-tuning changes how it behaves.

Week 2: Answers vary

A generative model can give different answers; re-run and compare.

Week 2: Hallucinations

The model can fluently invent claims; trace every specific claim to a source.

Week 2: Limited context

The model only has the context available to it; check what you provided.

Week 2: Knowledge cutoff

Trained knowledge stops at a cutoff; independently confirm recent facts.

Week 4: Model-building pipeline

Data → training → parameters → inference.

Week 4: Training

Learning by predicting hidden text and adjusting parameters.

Week 4: Parameters

Stored numerical values representing what training learned.

Week 4: Inference

Using the frozen trained model to generate an answer; parameters are not rewritten.

Week 4: Human, hybrid, or AI

Three possible ways to assign work: human-owned, split between humans and AI, or AI-handled.

Week 4: Complete hybrid plan

Name the owner and reviewer for every step; merely labeling a task hybrid is not enough.

Week 4: Four delegation criteria

Risk, reversibility, need for judgment, and data sensitivity.

Week 4: Risk

Consider how costly a mistake could be.

Week 4: Reversibility

Consider whether an action or decision can be undone.

Week 4: Need for judgment

Consider whether the work depends on tacit human judgment.

Week 4: Data sensitivity

Consider whether input contains confidential or sensitive information.

Week 4: When should a human own a task?

When stakes are high, actions hard to reverse, tacit judgment necessary, or data sensitive.

Week 4: Over-delegation

Giving AI work or decisions you cannot defend or properly oversee.

Week 4: Under-delegation

Spending valuable human time on cheap work AI could appropriately do.

Week 5: Structured data

Rows and columns.

Week 5: Semi-structured data

Labels with loosely organized contents.

Week 5: Unstructured data

Free-form contracts, emails, or audio.

Week 5: Four data failures

Incomplete, inconsistent, stale, and skewed.

Week 5: Incomplete data

Necessary information is missing.

Week 5: Inconsistent data

The same term or field is defined differently in different sources.

Week 5: Stale data

Information is out of date.

Week 5: Skewed data

Data disproportionately represents certain people or groups.

Week 5: Questions to ask about data

Where did it come from? Who created it and why? Who is missing?

Week 5: Five-part description

Task; Source; Audience & purpose; Form; Constraints.

Week 5: Task

Specify the question or action the AI must complete.

Week 5: Source

Specify the information the AI should rely on.

Week 5: Audience & purpose

Specify who the output is for and what they need from it.

Week 5: Form

Specify length and organization of the response.

Week 5: Constraints

Set rules, such as requiring traceable evidence for figures.

Week 5: Output is too generic

Likely missing or inadequate Source.

Week 5: Figures cannot be traced

Likely missing or inadequate Constraints.

Week 5: Output is too long

Likely missing or inadequate Form.

Week 5: Output is pitched at wrong level

Likely missing or inadequate Audience & purpose.

Week 5: AI answers the wrong question

Likely missing or inadequate Task.

Week 6: Predictive AI

Predicts a number or label from historical data.

Week 6: Generative AI

Creates new content.

Week 6: Five model families

General chat; Reasoning; Small & open-weight; Specialised media; Built-in assistants.

Week 6: Reasoning family

Suited to more involved reasoning; typically slower and pricier.

Week 6: Small & open-weight family

Potentially usable locally, so data can stay local.

Week 6: Specialised media family

Models specialized for media-related tasks.

Week 6: Built-in assistants

AI tools integrated into other applications.

Week 6: Six evaluation dimensions

Capability; Cost; Speed; Context; Data handling; Control.

Week 6: Fair model comparison

Same input; fresh chats; no follow-ups; blind scoring.

Week 6: Public benchmark limitation

A leaderboard is someone else’s test; it does not prove the model is best for your exact task.

Week 6: Model recommendation structure

Use X because of a deciding requirement, unless some constraint changes the choice.

Week 7: Four failure modes

Hallucination, sycophancy, drift, and bias.

Week 7: Hallucination

Presents invented content as factual.

Week 7: Sycophancy

Agrees with a user or backs down without sound evidence.

Week 7: Drift

Changes behavior when evaluated on the same test over time or across versions.

Week 7: Bias

Systematic default or omission that distorts representation.

Week 7: Check for hallucination

Trace claims to the underlying source.

Week 7: Check for sycophancy

Ask neutrally and then push back; check whether the answer changes without evidence.

Week 7: Check for drift

Re-run a fixed test set and compare outputs.

Week 7: Check for bias

Compare who appears in the input against who appears in the output.

Week 7: Five-step check

Trace → Recompute → Look for the gap → Push back → Scale to the stakes.

Week 7: Trace

Verify specific claims against their sources.

Week 7: Recompute

Recalculate figures rather than trusting model arithmetic.

Week 7: Look for the gap

Check for missing information or overlooked groups.

Week 7: Push back

Challenge assumptions or conclusions.

Week 7: Scale to the stakes

Increase verification effort when consequences are more serious.

Week 7: Can drift be proven from one response?

No. Changes must be detected by comparison.