Neural Systems: Machine Learning and Artificial Neural Networks

Vocabulary practice flashcards covering machine learning workflows, artificial neural networks, perceptrons, backpropagation, biological visual processing pathways, and deep learning models of single cortical neurons.

Training Data

The portion of cleaned data used to teach a machine learning model patterns and relationships.

Test Data

A separated subset of data, typically comprising $20\%$ to $30\%$ of the total dataset, used to evaluate a model's performance in a new context.

Model Fitting

The alignment between a mathematical model and data patterns, categorized into under-fitting (too simple to capture patterns), good fit, and over-fitting (too specific to training data to generalize to new data).

Under-fitting

A modeling flaw that occurs when a model is too simple and cannot adequately capture the underlying patterns in the data.

Over-fitting

A modeling flaw that occurs when a model is excessively customized to the training data and fails to accurately predict new, unseen data.

Decision Tree

A machine learning structure where an algorithm learns to sort information into categories based on the fewest possible separating features.

Convolution Kernel

A 1D or 2D grid of numbers passed over pixels during image processing to determine the influence of adjacent pixels and extract specific features such as blurring, sharpening, embossing, or edges.

Kernel Trick

A mathematical and statistical approach applied to nonlinear, overlapping datasets to project them into a higher-dimensional space where they can be separated.

Cross-Validation

An evaluation approach, such as k-fold cross-validation, that assesses model performance across multiple distinct subsets of data to verify generalizability.

Regularization

The technique of introducing penalties, such as L1 (Lasso) or L2 (Ridge), to discourage excessive model complexity and avoid fitting to noise.

Early Stopping

A safeguard during training that halts optimization as soon as performance on a separate validation set begins to deteriorate, preventing data memorization.

Pruning

The process of limiting the depth or removing superfluous branches of a decision tree to stop it from learning overly specific, non-generalizable patterns.

Dropout

A deep learning regularization technique where randomly selected nodes are deactivated during each training cycle to reduce co-adaptation and over-fitting.

Ensemble Methods

Techniques such as bagging (e.g., Random Forests) or boosting (e.g., Gradient Boosting) that combine outputs from multiple models to enhance generalization and reduce over-fitting.

Data Augmentation

The artificial expansion of a training dataset in neural networks through transformations such as rotating, flipping, or scaling existing images.

Error (Machine Learning)

The magnitude of the difference between the actual ground-truth value and the predicted output produced by the machine learning algorithm.

Traditional Machine Learning

Algorithms that learn tasks from data patterns without explicit step-by-step programming, but require humans to manually curate inputs and perform feature extraction.

Artificial Neural Network (ANN)

A computational architecture composed of interconnected nodes inspired by biological neurons that autonomously learns and extracts features directly from raw data.

Perceptron

The simplest neural network model for binary classification, which maps multiple binary inputs $(0, 1)$ into a single binary output $(0, 1)$ by adjusting weights and biases.

Perceptron Weight Update Rule

The learning equation $\Delta w_i = \beta (t - y) x_i$, where $w$ represents weights, $\eta$ is learning rate, $t$ is the target output, $y$ is the actual output, and $x_i$ is the input feature.

McCulloch-Pitts (MCP) Neuron

A non-learning computational unit that integrates inputs and applies a strict threshold to produce a binary output based on fixed predetermined rules.

Multilayer Perceptron (MLP)

A feedforward artificial neural network formed by linking multiple perceptrons together across an input layer, one or more hidden layers, and an output layer.

Input Layer

The initial layer of an artificial neural network, wherein each node corresponds directly to an individual feature of the incoming data.

Hidden Layer

An intermediate network layer where nodes function as integrators by computing weighted sums of upstream inputs and applying activation functions.

Output Layer

The final layer of an artificial neural network that generates the model's ultimate classification, prediction, or decision.

Backpropagation

An optimization algorithm that computes the gradient of the loss function relative to each weight and routes error backward through the network to update weights.

Simple Neural Network

A shallow neural network architecture containing only one or two hidden layers, appropriate for straightforward problems with restricted computational demands.

Deep Learning Network

An artificial neural network containing numerous hidden layers (tens to hundreds) that extracts hierarchical, complex features from extensive datasets.

Self-Information ($I(E)$)

The quantification of information conveyed by a specific event $E$, defined as $I(E) = \log_2\left(\frac{1}{p(E)}\right)$.

Shannon's Entropy ($H(X)$)

The average expected information content or uncertainty across all states of a random variable $X$, calculated as $H(X) = \sum_x P(x) \log\left(\frac{1}{P(x)}\right)$.

Visual System Receptive Fields

Hierarchical stimulus preferences across the visual pathway: ON-center retinal ganglion cells respond to a spot surrounded by a dark annulus, V1 simple cells respond to an elongated bar of light, and inferotemporal cortex cells respond to complex forms such as faces.

Dorsal Stream

The visual processing pathway that projects from V1 toward MT and the parietal cortex, dedicated to encoding spatial location and motion ("where/how").

Ventral Stream

The visual processing pathway that projects from V1 toward V4 and the inferotemporal (IT) cortex, dedicated to decoding form, color, structure, and object identity ("what").

Parallel Processing

The simultaneous routing of identical visual signals across distinct pathways (such as dorsal and ventral streams) to enhance efficiency and provide functional redundancy.

Temporally Convolutional Neural Network (TCN)

A deep network structure with $5$ to $8$ layers shown by Beniaguev et al. (2021) to successfully mimic the input-output dynamics of a layer 5 cortical pyramidal cell at single-millisecond precision.

Neuronal Computational Complexity Drivers

The two primary biological factors identified in Beniaguev et al. (2021) that generate the deep-network complexity of single cortical pyramidal neurons: dendritic morphology and NMDA receptors.