5. Genetic AI and Large Language Models Study Notes

These flashcards cover essential vocabulary related to generative AI and large language models, providing definitions and key concepts discussed in the lecture.

Generative AI

A type of AI that can create new content, including text, images, music, and videos.

Large Language Models

Probabilistic models of text that determine the likelihood of sequences of words occurring based on previous words.

Deep Learning

A subset of machine learning that uses neural networks to learn from complex data.

Machine Learning

A subset of AI that uses algorithms to learn from past data to identify trends and predict outcomes.

Fine-Tuning

The process of further training a pretrained model on a labeled dataset to optimize it for a specific task.

Inference

The process of making predictions based on a trained model using new input data.

Transformers

A deep learning architecture that allows models to focus on relevant parts of input data for predictions.

Self Attention Mechanism

A component of transformer architecture that helps the model understand the importance of different words in relation to each other.

Tokens

Units of text processed by large language models, which can represent whole words or parts of words.

Embeddings

Numerical representations of text data that capture the contextual meaning of the input.

Retrieval-Augmented Generation

A framework that allows language models to query external knowledge bases to ground responses.

Hallucination

When a model generates text that is non-factual or ungrounded.

Prompt Engineering

The iterative process of refining input prompts to elicit desired output from a language model.

Training Dataset

A collection of data used to train machine learning models, consisting of input features and corresponding output labels.

Probabilistic Model

A type of model that uses probability to predict outcomes based on observed data patterns.

Supervised Machine Learning

A type of machine learning where the model is trained with labeled data, consisting of input features and output labels.

Unsupervised Learning

A type of learning where models learn patterns in data without labeled outputs.

Which sequence model can maintain relevant information over long sequences?

Which type of Recurrent Neural Network (RNN) architecture is used for Machine Translation?

3. Which essential component of Artificial Neural Network performs weighted summation and applies activation function on input data to produce an output?