Algoverse Research AI Lecture 1

Artificial Intelligence (AI)

The field concerned with creating machines capable of intelligent behavior, including learning from experience.

Machine Learning (ML)

A subset of AI where algorithms learn from and make predictions based on data.

Language Model (LM)

A type of machine learning model designed to understand and generate human language.

Generative Models

Models that learn the underlying distribution of data and can generate new data points.

Autoregressive Models

A specific type of generative model that predicts the next item in a sequence based on previous items.

Weights

Numerical parameters in a model that are adjusted during training to improve predictions.

Training

The process of optimizing a model's weights using labeled data.

Inference

Using a trained model to make predictions on new data.

Overfitting

When a model learns the training data too well, including noise, leading to poor generalization.

Underfitting

When a model fails to capture the underlying patterns in the data, resulting in poor performance.

Neural Networks (NNs)

A class of models inspired by the human brain, consisting of layers of interconnected nodes.

Transformers

A type of neural network architecture particularly effective for language tasks.

Classification

A task where the output variable is categorical (e.g., spam vs. not spam).

Regression

A task where the output variable is continuous (e.g., predicting house prices).

Literature Review

The process of reviewing existing research to identify gaps and formulate new research questions.

Benchmark

A standard dataset and metric used to evaluate model performance.

Baseline

An existing method used as a comparison point for new methods.

Ablations

Experiments that systematically remove parts of a model or method to evaluate their contribution.

Prompting

Providing a specific input (or prefix) to guide a language model's output.

Abstract

A concise summary of a research paper's content and findings.

Introduction

The section of a paper that motivates the research problem and outlines the approach.

Methods

Describes the techniques or algorithms used in the research.

Experimental Setup

Explains the datasets, benchmarks, and evaluation metrics used.

Results

Summarizes the findings and outcomes of experiments.

Discussion

A section that reflects on the findings, limitations, and potential future work.