Generative AI

LLMs

statistical pattern recognition + prediction systems

token

unit of text, based on words or segments of words

LLM output

next likely word (token) in a sentence (sequence)

text corpus

source of digitalized text

large language models are supervised/unsupervised

unsupervised, large dataset => cannot label

transformer

attention-based feed-forward network

retrieval augmented generation (RAG)

•  retrieve domain specific information

vector database

• parts of documents converted to vectors

chunking

• long document broken up into coherent sections

  • Eg: paragraphs in a book, functions in code

discriminative model

categorize, answer yes/no

genrative model

creates new examples, answer  What could this be?

Generative Adversial Network (GAN):

combination of generative + discriminatory models:

• Generator creates fake images for discriminator to test, discriminator trained with real images

GAN use case

realistic videos or pictures

Variational Auto Encoder (VAE)

learns patterns in data to recreate original or generate new examples

VAE use case

• Good for generating Audio

  • Can blend song A into song B

  • Can recognize cord progression + rhythm