Introduction to Machine Learning, Ensemble Models, and Neural Networks

Comprehensive vocabulary flashcards covering basic machine learning algorithms, ensemble modeling techniques (boosting and bagging), deep learning neural network structures, and network optimization principles.

Support Vector Machines

Classification or regression machine learning models that operate along linear boundaries or trends, often applied within multi-dimensional hyperspaces.

Hyperspace

A multi-dimensional space formed by placing different covariates on separate axes, allowing data visualization and analysis across multiple dimensions.

MaxEnt Modeling

Maximum entropy modeling; a species distribution model that maximizes information retention from a dataset by comparing background location points to species presence locations.

k-Nearest Neighbor Algorithm

A machine learning algorithm that classifies or regresses new data points by assessing $k$ number of nearest neighboring points in an already classified dataset.

Euclidean Distance

The straight-line distance measured between points in a coordinate space, used to determine spatial relationships or adjacencies in datasets.

Decision Tree

A machine learning model consisting of a structured sequence of choices leading to an outcome, used to assess the relative importance of decisions.

Pruning

The process of removing unnecessary decisions from a decision tree that do not significantly improve prediction accuracy, simplifying the model and reducing runtime.

Ensemble Models

Machine learning architectures created by combining the outputs of multiple lesser models (weak learners) to improve overall predictive performance.

Weak Learner

A relatively simple or low-performing base model that is aggregated with other models within an ensemble framework.

Boosting

An iterative ensemble technique where models are trained sequentially on the same dataset, with each subsequent model fitting the residual error of the previous model.

Bagging

An ensemble technique where multiple models are trained concurrently on unique subsampled subsets of the primary dataset.

Gradient Boosting

A common form of boosting (also termed boosted regression trees) that sequentially minimizes loss by fitting new models to the error produced by prior models.

Random Forest

A widely used bagging method (also referred to as a classification decision tree or CART) that aggregates predictions across multiple decision trees trained on data subsamples.

Perceptron

An artificial neuron that evaluates input signals against an internal activation function to determine its output response.

Node

An individual computational unit or artificial neuron located within a specific layer of a neural network.

Edge

A connection between two nodes in a neural network along which data signals pass.

Weight

A tunable parameter assigned to an edge in a neural network that scales the signal strength and determines the relative importance of an input connection.

Bias

An added constant value assigned to a node in a neural network layer that shifts the activation threshold of that neuron.

Activation Function

A mathematical formula (such as $a = \frac{1}{1 + e^{-z}}$ or binary functions) applied to a node's aggregated inputs to determine if and to what extent the node fires.

Interpolation Threshold

A threshold in deep learning model complexity beyond which model variance decreases and performance improves despite high parameter count.

Loss Function

A mathematical expression (also known as a cost function) measuring the discrepancy between a neural network's predictions and the true data values.

Gradient Descent

An optimization technique that minimizes model error by evaluating gradients across parameters and iteratively moving down the steepest slope of the loss function.

Local Minimum

A regional low point in a loss function landscape where error increases in immediate surrounding steps, potentially causing simple optimization routines to get stuck.

Backpropagation

An algorithm for neural network training that calculates error gradients backward from the output layer through hidden layers to update edge weights throughout the network.

Forward Propagation

The forward transmission of input data through neural network layers and nodes to produce an output prediction.