Artificial Intelligence for Aerospace Control and Operations

Vocabulary practice flashcards covering all six core modules of Artificial Intelligence for Aerospace Control and Operations (AE4353): Introduction to Machine Learning & Perceptrons, Deep Neural Networks, Generative AI, Recurrent Networks & Sequence Modeling, Physics-Informed Neural Networks (PINNs), and Explainable AI.

Physics-Informed Neural Network (PINN)

A framework that leverages the high representational capacity of artificial neural networks to learn unknown, complex functions while embedding guidance and constraints from physical laws (such as partial differential equations) into the loss function to regularize learning.

Soft Constraints Loss (PINNs)

A composite objective function formulating physical laws and data fitting without rigid constraints: $\mathcal{L} = \mathcal{L}_{\mathcal{F}} + \mathcal{L}_{\mathcal{B}} + \mathcal{L}_{\mathcal{I}} + \mathcal{L}_{\mathcal{D}}$, where $\mathcal{L}_{\mathcal{F}}$ represents differential equation residuals, $\mathcal{L}_{\mathcal{B}}$ enforces boundary conditions, $\mathcal{L}_{\mathcal{I}}$ initial conditions, and $\mathcal{L}_{\mathcal{D}}$ available measurements.

Forward Problem (PINNs)

A problem formulation where the neural network solves for system outputs $\mathbf{u} = f(\mathbf{x}, t)$ given state $\mathbf{x}$ and time $t$, under the assumption that all parameters of the underlying differential equations (such as wave velocity $c^2$) are known.

Inverse Problem (PINNs)

A problem formulation where unknown physical parameters of the governing differential equations (such as wave speed parameter $c^2$) are treated as learnable variables and optimized simultaneously with the network parameters.

Hadamard's Well-Posedness Conditions

A mathematical criterion stating that a problem is well-posed if: (1) a solution exists, (2) the solution is unique, and (3) the solution depends continuously on the initial and boundary conditions.

Causality Violation in Vanilla PINNs

A failure mode where standard PINN training stagnates in local minima because the loss drops early for later time steps while remaining high for initial conditions. It is resolved by weighting earlier time steps higher with weights $w_i = \exp(-\varepsilon \sum_{k=1}^{i-1} \mathcal{L}_{\mathcal{F}}(t_k))$.

Neural-Fly

A physics-informed deep learning architecture designed for agile flight in wind, where a neural network learns aerodynamic basis functions that are element-wise multiplied with an adaptive parameter vector to predict wind force residuals.

i.i.d. Assumptions

The foundational supervised learning assumptions stating that: (1) data samples are mutually independent ($P(\mathbf{x}_i | \mathbf{x}_j) = P(\mathbf{x}_i)$), and (2) training and test samples are identically distributed ($P_{\text{train}}(\mathbf{x}) = P_{\text{test}}(\mathbf{x})$).

Design Matrix

A matrix $X \in \mathbb{R}^{m \times n}$ used to organize an entire supervised input dataset, where $m$ denotes the total number of samples and $n$ denotes the number of features per sample.

Perceptron

A foundational neural model introduced by Frank Rosenblatt (1958) following McCulloch & Pitts (1943) that computes an output $o = f(\sum_i w_i x_i)$; inherently limited to learning linearly separable functions.

Delta Learning Rule

A learning algorithm introduced by Widrow and Hoff (1962) that adjusts perceptron weights using gradient descent on the quadratic error $e = (o - t)^2$, resulting in weight updates $\Delta w_i = -\alpha \frac{\partial e}{\partial w_i} = -2\alpha (o - t) f'(a) x_i$.

Sigmoid Activation Function

A smooth, bounded activation function defined as $f(x) = \frac{1}{1 + e^{-x}}$ with derivative $f'(x) = f(x)(1 - f(x))$, which approximates a step function between $0$ and $1$ but suffers from vanishing gradients when saturated.

Universal Approximation Theorem

A mathematical theorem (Cybenko, 1989; Hornik, 1991) proving that a multi-layer perceptron with at least one hidden non-linear layer can approximate any continuous function to arbitrary precision, given an infinite number of hidden neurons.

Inductive Bias

The set of prior assumptions, structural choices, or constraints that a learning algorithm incorporates to prioritize certain solutions and generalize from observed training examples to unseen test samples.

Hypothesis Space

The complete space of candidate functions or models that a chosen machine learning algorithm is restricted to search within when attempting to map inputs to outputs.

Early Stopping

A regularization technique where model performance is monitored on an independent validation set during training, and optimization is halted when validation loss begins to rise, preventing the network from overfitting.

K-Fold Cross-Validation

A resampling evaluation procedure where the dataset is split into $K$ equal partitions (folds); the model is trained on $K-1$ folds and tested on the remaining fold, repeating the cycle $K$ times to obtain a robust estimate of generalization error.

Rectified Linear Unit (ReLU)

A piecewise linear activation function defined as $f(x) = \max(0, x)$ with derivative $f'(x) = (x > 0)$; fast to compute and resistant to vanishing gradients for positive activations.

Adam (Adaptive Moments)

An adaptive optimization algorithm that computes parameter updates by tracking decaying running averages of both the first gradient moment ($\mathbf{s}$) and second raw moment ($\mathbf{r}$), corrected for initialization bias: $\Delta \boldsymbol{\theta} = -\alpha \frac{\hat{\mathbf{s}}}{\sqrt{\hat{\mathbf{r}}} + \delta}$.

Ensemble Learning

A technique where multiple distinct neural networks are trained on different splits of data or initialized differently, aggregating their predictions to reduce variance, prevent overfitting, and quantify prediction uncertainty.

Dropout

A regularization method applied during training where hidden units are set to zero with probability $p$, and retained weights are scaled by $\frac{1}{1-p}$; prevents co-adaptation of features and approximates training an ensemble of thinned networks.

Weight Sharing

A structural constraint in convolutional neural networks where the same set of kernel weights is applied across every spatial location of the input, dramatically lowering the parameter count compared to fully connected layers.

Max-Pooling

A non-linear downsampling operation in CNNs that extracts the maximum value over local sliding spatial windows of size $k \times k$ with stride $s$, reducing spatial dimensions and conferring translation invariance.

Receptive Field

The spatial size of the local region in the input layer that contributes to the activation of a specific unit in a deeper convolutional layer, determined by kernel sizes, strides, and padding.

Transposed Convolution

An upsampling operation (sometimes called deconvolution) that expands feature map spatial dimensions by inserting $z = s - 1$ zeros between inputs, padding boundaries, and sliding a kernel with unit stride.

Residual Connection (Skip Connection)

An architectural bypass connection that directly adds the identity input vector to the output of a layer or block ($f(\mathbf{x}) + \mathbf{x}$), facilitating gradient backpropagation and preventing vanishing gradients in deep networks.

Discriminatory vs. Generative Models

Discriminatory models learn conditional mappings $P(\mathbf{y}|\mathbf{x})$ to separate classes or predict targets, whereas generative models model the joint/input probability distribution $P(\mathbf{x})$ to synthesize novel realistic data without labels.

Variational Autoencoder (VAE)

A directed probabilistic generative network whose encoder maps inputs to mean $\mathbf{z}_\mu$ and variance $\mathbf{z}_\sigma$ vectors of a latent distribution, and whose decoder reconstructs samples drawn from this continuous latent space.

Evidence Lower Bound (ELBO)

The objective maximized during VAE training: $\text{ELBO}(\Theta) = \sum_{i=1}^N \mathbb{E}_q[\log P(\mathbf{x}|\mathbf{z}, \Theta_d)] - D_{KL}(q(\mathbf{z}|\mathbf{x}, \Theta_e) \parallel P(\mathbf{z}))$, consisting of a reconstruction log-likelihood term and a Kullback-Leibler divergence regularization term.

Reparameterization Trick

A mathematical formulation in VAEs that expresses the stochastic latent variable as $\mathbf{z} = \mathbf{z}_\mu + \mathbf{z}_\sigma \odot \boldsymbol{\epsilon}$, where $\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$, isolating randomness and permitting gradient backpropagation through $\mathbf{z}_\mu$ and $\mathbf{z}_\sigma$.

Generative Adversarial Network (GAN)

A generative framework framing training as a zero-sum game between a Generator $g(\mathbf{z})$ synthesizing fake samples and a Discriminator $d(\mathbf{x})$ classifying samples as real ($1$) or fake ($0$) using cross-entropy loss.

CycleGAN

An unpaired image-to-image translation model utilizing two generator-discriminator pairs trained with an adversarial loss and a cycle-consistency loss $\mathcal{L}_{\text{cyc}}$ enforcing $g_X(g_Y(\mathbf{x})) \approx \mathbf{x}$ and $g_Y(g_X(\mathbf{y})) \approx \mathbf{y}$.

Diffusion Model

A generative model featuring a fixed forward Markov chain adding Gaussian noise step-by-step ($\mathbf{x}_0 \rightarrow \mathbf{x}_T$) and a trained neural network that reverses this corruption to iteratively denoise white noise into realistic data.

Classifier-Free Guidance

A conditional diffusion technique where class label $\mathbf{c}$ is randomly replaced with $\mathbf{0}$ during training, allowing generation to interpolate between conditional and unconditional score estimates: $\hat{\boldsymbol{\epsilon}} \leftarrow \gamma \boldsymbol{\epsilon}(\mathbf{x}_t, \mathbf{c}, \Theta, t) + (1 - \gamma)\boldsymbol{\epsilon}(\mathbf{x}_t, \mathbf{0}, \Theta, t)$, with $\gamma > 1$.

Self-Attention Mechanism

A core Transformer operation producing output $\mathbf{y}_i = \sum_j w_{i,j} \mathbf{v}_j$, where weights are softmax-normalized dot products between learned query ($\mathbf{q}_i$) and key ($\mathbf{k}_j$) vector projections: $w'_{i,j} = \frac{\mathbf{q}_i^T \mathbf{k}_j}{\sqrt{D}}$.

Backpropagation Through Time (BPTT)

An extension of standard backpropagation to recurrent networks where the recurrent graph is unrolled over multiple successive time steps, propagating error gradients backwards along the unrolled sequence.

Vanishing/Exploding Gradients in RNNs

A failure mode during BPTT caused by repeated multiplication by the transition weight matrix $W$ over $n$ steps ($W^n = Q^T D^n Q$), which causes error signals along directions with eigenvalues $|\lambda| < 1$ to decay to zero and $|\lambda| > 1$ to explode.

Leaky Neurons

Recurrent units featuring neural inertia defined by $\mathbf{h}_t = \alpha \mathbf{h}_{t-1} + (1 - \alpha) f(W\mathbf{x}_t + \mathbf{b})$, creating a low-pass filter of past activations where learning different leak rates $\alpha$ allows capture of multi-scale dynamics.

Reservoir Computing (Echo State Networks)

A recurrent network architecture where a large, randomly connected recurrent layer (the reservoir) is fixed at the edge of stability by setting its spectral radius $\rho(W)$, and training is restricted strictly to a linear readout layer.

Long Short-Term Memory (LSTM)

A recurrent architecture designed to eliminate vanishing gradients by maintaining a linear cell state $c_t$ whose contents are regulated by three multiplicative sigmoid gates: forget gate ($f_t$), input gate ($i_t$), and output gate ($o_t$).

Gated Recurrent Unit (GRU)

A gating mechanism that simplifies the LSTM by merging the cell state and hidden state, controlling temporal information flow using only two gates: an update gate ($z_t$) and a reset gate ($r_t$).

Structured State Space Sequence Model (S4)

A sequence modeling framework that discretizes continuous linear time-invariant state-space models ($\mathbf{h}_t = A\mathbf{h}_{t-1} + B\mathbf{x}_t, y_t = C\mathbf{h}_t + D\mathbf{x}_t$), structuring $A$ to allow long sequence evaluation via parallel convolutions: $y = K * \mathbf{x}_{1:t}$.

Linear Recurrent Unit (LRU)

A simplified recurrent unit that diagonalizes state transitions using a stable complex exponential parameterization $\lambda_j = \exp(-\exp(\nu_j^{\log}) + i\exp(\theta_j^{\log}))$ with normalization $\gamma_j = (1 - |\lambda_j|^2)^{1/2}$ to achieve S4-level performance.

Mamba

A selective structured state-space sequence model architecture that makes parameters $B$, $C$, and $\Delta$ dynamic, input-dependent functions of time ($B(\mathbf{x}_t)$, $C(\mathbf{x}_t)$, $\Delta(\mathbf{x}_t)$), maintaining linear-time inference.

Intrinsic vs. Post Hoc Interpretability

Intrinsic methods rely on simple model structures (e.g., short decision trees, linear models) whose internal parameters are directly interpretable; post hoc methods inspect, perturb, or analyze complex black-box models after training.

Surrogate Model

An intrinsically interpretable model (such as a decision tree or linear regressor) trained to approximate the input-output behavior of a complex deep neural network globally or in a local neighborhood.

LIME (Local Interpretable Model-agnostic Explanations)

A post hoc, model-agnostic technique that explains an individual prediction by generating perturbed samples around that instance, weighting them by proximity, and training a sparse linear surrogate model on the resulting predictions.

Shapley Values

A coalitional game-theory method computing the fair marginal contribution $\phi_i$ of feature $i$ to a prediction across all possible feature subsets, uniquely fulfilling Efficiency, Dummy, Additivity, and Symmetry axioms.

SHAP (SHapley Additive exPlanations)

A framework unifying Shapley values with local surrogate models to compute an additive feature attribution explanation model: $g(\mathbf{z}) = \phi_0 + \sum_{i=1}^n \phi_i z_i$, where $z_i \in \{0, 1\}$ indicates feature presence in a coalition.

Saliency Map (Vanilla Gradient)

A local visual explanation produced by computing the gradient of an output class score with respect to raw input image pixels ($\frac{\partial y_c}{\partial I}$) via backpropagation, visualizing which pixels most sensitively alter the output score.

Grad-CAM (Gradient-Weighted Class Activation Mapping)

A visual explanation technique that averages backpropagated gradients for class $c$ across spatial dimensions of the final convolutional feature maps ($\alpha_k^c = \frac{1}{Z}\sum_i \sum_j \frac{\partial y_c}{\partial A_{i,j}^k}$) and computes a rectified weighted sum: $L^c = \text{ReLU}(\sum_k \alpha_k^c \mathbf{A}^k)$.

Feature Visualization (Activation Maximization)

An interpretability method that identifies what a specific neural unit represents by running gradient ascent in pixel space to generate a synthetic input image that maximizes the unit's activation: $I^* = \text{argmax}_I \sum_{x,y} h(l, c, x, y)$.

t-SNE (t-Distributed Stochastic Neighbor Embedding)

A non-linear dimensionality reduction algorithm that projects high-dimensional latent space representations into two or three dimensions while preserving local neighborhood affinities and distances.