10-13-2026 AI Notes

What is the wirth statemtn about program?

program = data + algorithm (step by step logic)

What is the wirth idea of LLM?

knowledge + reasoning; knowldge =data; reasoning = alogrhtim

you convert het data into weights; inference

What does the p in GPT stand for ?

pretrainign

cmpress all data into parmaeters

How does FSD work?

you take hte data from the input into the scanenrsand hten it perofmrs real time reaosning using its pretraned alogrhti

What are LLM about in Karpathy’s POV?

paramter and code

What is antoher way of saying paramter?

weights

How many lines of code of reaosning do yo uneed?

~500 lines of code from 140GB compression of 10 TB original data

Wheen it comes to closed eight what do you need to use to access?

Using API

What do tehy pretrain on ?

corpora

What is corpora?

a respostiry of avaialbe data—-wikiedpia, gutenrbeg etc

What data sources are used for finesets?

task data set, daily chat data set, syntehtic datasta

What does scale ai try to think about? Turing?

the idea is the use of exeprt feddback pretriang

What is used for pretraining and for finetuning?

pretrainnig uses corpurs

finetuing uses the cotraining that can end up doing things that are proepr to say and so ofrth;

What is instructoin and aslignmetn used for?

it’s used for postraining

What technique is used in visual recongition?

Residual learning; every few layers you end up imposing a condition; its like a checkpoint and that you end up putting in the orignal image in there; its like that one part of the u net where ou end up bringi t bac

why does cheaper model cost more per task?

you end up having a complex task and it actually creates a situation where you end up haivng to go bcak and forh as opposed to just doing it in one shot or

BLEU test what does it do?

use n grams (what words go with one another0: how mnay 1 gram, 2 gram etc end up overlapping tocombine them into ascore; degree of overlapping`

What does BLEU use in terms of optimizatoin?

use the breviry pentality which works exponentialyl

what type of average is BLEU?

it’s a weighted average

look up random llm code on github and see if you can identify it; ah and cs code identfication task

What are we trying to do in the code?

we measure the amount of agreemnet beween the human translatoin and the progarmming genrated translatoin

How does ROGUE end up working?

You end up looking at the n-gram overlarp

What are the two varities of ROGUE?

ROGUE-n—-n gram overlpa; sscore of 50% exapl

ROGUE -L___longest common subsunq;ce; score of 1 example; one setnence shodl end up matchng up tere

Do we use ROGUE both or just completmanry?

What is perplexitiy doing?

it is trying to use the contex sto shape how perplexed the model is; how many branches can be made?

What is Pass@K doing?

If we give it many tries, can it at least get it once? whats htei dea ehind this, it sesm to be very limited test to go for

iCLR (learning repseitonat0

Look up iCLR workshop; look up workshops and confnerce for differnt fields

What is the branchign trying to do?

it’s trying to cross out hte least used branches

What was the 2025 paper trying to say?

that if you end up having reinfocment learning over multiple samples, it doens’t actually end up doing better; why is that the case?

the correct model is actually in it the

it doens’t enchane the serach scpace; it just makesit accurate with in tha tsapce

What is ELO rating trying to do?

let’s gie it head to head competion

It seems like you end up using the ELO rating to stack the BLEU or ROGUE together

what is it called when it can use prompts from 3 months ago?

cross infernece;

What is a key thing to specficy when using API?

specifiy the role

Why is it necessry to specify theAPI?

the pretraning is just a tmeplate; you want ot

if i can use api what exaclty is ai presnting it to me?

does promp or ocmpeliton encodnig vs deocidng cost more?

How to put files into the api?

Customer serv