Into to Applied data science w5

What are LLMs?

They are neutral networks to predict and generate text.

How do LLMs learn?

They learn patterns from vast amounts of text data

How are LLMs different from traditional text mining methods?

They understand context and relationships between words instead.

What are some task LLMs can do?

Answer questions, translate languages and write code.

What is next-word prediction in LLMs?

The task of predicting the next word that it learned a pattern.

What is tokenization?

The process of converting words into numbers/IDs so that models can work with these numerical representations.

What is Embedding?

The process where each token ID is converted into a vector( a list of numbers)

What does a 4-dimensional embedding look like?

[…,…,…,…]

What happens to the vectors when words have similar meanings?

They vectors become similar as well.

What is this famous example used for: king - man + woman = queen

It is used to show the mathematical relationship of Embeddings.

How many dimensions do LLMs have?

Hunderds or thousands

What is a neral network?

A neural network is a mathematical function that transforms inputs into outputs through layers. Wh

What is the formula for a layer?

Output = activation(weights x input + bias)

In the formula for a layer how are weights deffined?

They are parameters that the model learns during training.

What happens when multiple layers are stacked?

They allow the network to learn complex patterns.

What does word2vec/cbow do?

They allow us to make words numerical so that we can put them into a neral network.

What does One-hot encoding do?

Represent each word as a vector of length V(vocabulary size).

What is Win?

A V x N matrix where V is vocabulary size and N is embedding dimension.

What does each Row of Win represent?

The iput emdedding for a word. W

What is Wout?

An N x V matrix where each column represents the output embedding for a word.