Comprehensive Numerical Methods and Differential Equations Study Guide

Differential Equations: Fundamentals & Classification

• Ordinary Differential Equations (ODEs): Equations that establish a mathematical relationship between an unknown function of a single independent variable and its derivatives.
• Order of an ODE: Defined by the highest-ordered derivative present in the equation. Algebraically clearing fractions or radicals involving derivatives leaves the order of the differential equation completely unchanged.
• Linearity Criteria: An ODE is classified as linear if the unknown dependent variable and all of its derivatives appear strictly to the first power and are nowhere multiplied together.
• Constant Coefficients: A linear ODE possesses constant coefficients if all multipliers attached to the dependent variable and its derivatives are real numbers or constants.
• Autonomous Differential Equations: First-order ODEs taking the explicit structural form $\frac{dy}{dt} = f(y)$, characterized by the complete absence of the independent variable ($t$ or $x$) on the right-hand side.
• Variable Separable Structural Form: A first-order ODE that can be written directly in the form $f(x)\,dx + g(y)\,dy = 0$.
• Exact Differential Equations: Equations arising from the total differential $dz = \frac{\partial f}{\partial x}\,dx + \frac{\partial f}{\partial y}\,dy = 0$. A first-order equation $M(x,y)\,dx + N(x,y)\,dy = 0$ is exact if and only if $\frac{\partial M}{\partial y} = \frac{\partial N}{\partial x}$. Geometrically, the resulting general solution $F(x,y) = C$ represents a one-parameter family of level curves forming a topographic map of a multivariable surface.
• Integrating Factors: An algebraic multiplier, such as $\frac{1}{x^2}$, applied to convert an inexact differential equation into an exact differential equation. An integrating factor $\mu(y)$ that depends strictly on $y$ alone requires that the expression $\frac{\frac{\partial N}{\partial x} - \frac{\partial M}{\partial y}}{M}$ evaluates to a function of $y$ only. Multiplying a linear first-order equation by its integrating factor transforms its left-hand side into the exact derivative of a product: $\frac{d}{dx}[p(x) \cdot y]$.
• Homogeneous Differential Equations: Equations of the form $M(x,y)\,dx + N(x,y)\,dy = 0$ where both functions are homogeneous of the same degree. The algebraic substitution $y = vx$ (expanding to $dy = v\,dx + x\,dv$ via the product rule) is preferred over $x = vy$ whenever $M(x,y)$ is algebraically simpler than $N(x,y)$.
• Non-Linear ODE Examples: Any equation where the highest-order derivative or dependent variable is raised to an algebraic power greater than $1$, such as $\left(\frac{dy}{dx}\right)^2 + y = x$, is classified as non-linear.
• Initial Value Problems (IVPs) vs. Boundary Value Problems (BVPs): An IVP supplements an ODE with constraints all specified at a single point or time, typically yielding a unique solution under mild conditions. A BVP specifies constraints at two or more distinct points; depending on these boundary conditions, a linear BVP may possess a unique solution, infinitely many solutions, or no solution at all.
• Particular vs. Singular Solutions: A particular solution is obtained by assigning specific numerical values to the arbitrary constants in a general solution. A singular solution is a valid solution that cannot be derived analytically by assigning any value to the arbitrary constants of the general solution.

Analytic Methods & Theory of Linear ODEs

• Lipschitz Condition & Picard-Lindelöf Theorem: The Lipschitz condition restricts how fast a function $f(t,y)$ can change by establishing a uniform upper bound on its rate of change. Under the Picard-Lindelöf theorem, ensuring $f(t,y)$ satisfies a Lipschitz condition guarantees both the existence and uniqueness of the solution to an Initial Value Problem.
• Wronskian Determinant: A determinant constructed from a set of functions and their successive derivatives. Used to evaluate the linear independence of solutions over a specified interval.
• Abel's Identity: Dictates that if the Wronskian of two solutions to a second-order linear homogeneous ODE evaluates to zero at even a single point within an interval, it must be identically zero everywhere across that entire interval, proving the solutions are linearly dependent. Conversely, if $\text{Wronskian} > 0$ across an interval (e.g., $y_1 = x$ and $y_2 = x^2$ producing $W = x > 0$ on $(0, \infty)$), the solutions are verified to be linearly independent.
• Fundamental Set of Solutions: A set of $n$ linearly independent functions that each individually satisfy an $n$-th order linear homogeneous ODE. The full complementary function $y_c$ is algebraically constructed as a linear combination: $y_c = c_1 y_1 + c_2 y_2 + \dots + c_n y_n$.
• Differential Operator Form: Denoted by $D^n$, commanding $\text{differentiating } y \text{ exactly } n \text{ consecutive times}$. The operator $L$ preserves the Principle of Superposition, satisfying the linearity identity $L(a f + b g) = a L(f) + b L(g)$.
• Characteristic (Auxiliary) Equations & Roots: Formed by replacing differential operators with a polynomial variable $m$.
  • Distinct Real Roots: Roots such as $m_1 = 0$, $m_2 = 2$, and $m_3 = -2$ produce a complementary solution composed of one constant term and two distinct real exponential terms.
  • Complex Conjugate Roots: Roots $m = \alpha \pm j\beta$ yield exponential-trigonometric solutions $y = e^{\alpha x}(c_1 \cos(\beta x) + c_2 \sin(\beta x))$. Pure imaginary roots ($m = \pm j\omega$) generate undamped, sustained simple harmonic oscillations.
  • Repeated Complex Roots: When complex conjugate roots are repeated, successive terms are multiplied by integer powers of the independent variable ($x$) to maintain linear independence.
• Method of Undetermined Coefficients: Applicable only when the forcing function $f(x)$ produces a finite closed set of linearly independent functions upon successive differentiation. For example, a forcing function $f(x) = x e^x$ yields an assumed particular solution $y_p = (A x + B) e^x$. If a proposed term in $y_p$ already exists in the complementary solution $y_c$, it must be multiplied by the lowest positive integer power of $x$ that eliminates the duplication.
• Inapplicability of Undetermined Coefficients: The method fails for functions like $\sec(x)$, $\tan(x)$, or $\ln(x)$ because their successive derivatives generate infinitely expanding new terms rather than a finite set.
• Method of Variation of Parameters: A universally applicable, robust analytical technique capable of solving non-homogeneous linear ODEs regardless of whether $\text{forcing function } f(x)$ contains non-polynomial terms like $\sec(x)$ or $\ln(x)$.
• Linear Combination Substitution: Non-separable first-order differential equations of the form $\frac{dy}{dx} = \frac{x + y + 1}{x + y - 1}$ are reduced to a variable separable form using the geometric substitution $v = x + y$.

Applications of Differential Equations

• Uninhibited Population Growth: Modeled by $\frac{dP}{dt} = k P$, where the constant $k$ physically signifies the relative growth rate.
• Logistic Population Growth: Modeled by $frac{dP}{dt} = P(a - b P)$, where the algebraic ratio$\frac{a}{b} represents the maximum carrying capacity of the environment.\n* **Radioactive Decay**: Modeled by \frac{dN}{dt} = -k N. The half-life parameter is defined as the precise time required for half of the radioactive nuclei in a sample to disintegrate.\n* **Newton's Law of Cooling**: Temperature decays according to T(t) = T_m + (T_0 - T_m)e^{-kt}$. Taking the limit as$t \to \infty$proves that the object's temperature asymptotically approaches the ambient temperature$T_m$. If$T_m varies sinusoidally with time, the system is governed by a non-homogeneous linear ODE.\n* **Liquid Mixing Tank Dynamics**: When the constant volumetric outflow rate exceeds the constant inflow rate, the total fluid volume V(t) inside the tank is represented as a linearly decreasing function of time.\n* **Series RL Circuit Time Constant**: For L \frac{di}{dt} + R i = 0$, the ratio$\frac{L}{R}$dictates the exponential time constant required for the transient current to decay to$36.8\%$($e^{-1}) of its initial value.\n* **Series RC Circuit Transient Dynamics**: Finite current available from a physical source requires a non-zero duration to accumulate charge on capacitor plates, enforcing the physical boundary condition that capacitor voltage cannot change instantaneously.\n* **Transient Response in RLC Circuits**: The particular integral y_p of the non-homogeneous ODE represents the sustained, steady-state forced response. A critically damped passive circuit corresponds to two real, identical repeated roots in the auxiliary equation.\n* **Slope Fields & Isoclines**: A slope field systematically maps short line segments representing exact derivative values across a grid. An **isocline** is a curve along which the slope of all crossing solution curves remains constant. Constant values of y$where$f(y) = 0 represent equilibrium solutions or critical points.\n\n# Power Series, Special Functions & Partial Differential Equations\n\n* **Ordinary vs. Singular Points**: In power series solutions near x = a$, if the limits defining the coefficient functions$p(x)$and$q(x)$are infinite and fail to be analytic,$x = a$is classified as an irregular singular point. If regular singular at$x = 0, the **Method of Frobenius** must be applied.\n* **Method of Frobenius & Indicial Equations**: Solving the indicial equation yields the characteristic exponents r$determining the form and behavior of the series solution near a regular singular point. Expansion about an ordinary point$x = 0 yields a standard Maclaurin series.\n* **Special Orthogonal Functions**: \n * **Gamma Function**: Extends the factorial concept to real and complex number arguments.\n * **Legendre Polynomials**: Generated by power series solutions to Legendre's ODE; critical for solving PDE boundary value problems in spherical coordinates and serving as the operational basis for Gaussian Quadrature.\n * **Bessel Functions**: Solutions J_n(x)$and$Y_n(x) generated by Bessel's ODE, arising in physical systems exhibiting cylindrical symmetry.\n * **Chebyshev Polynomials**: Orthogonal polynomials used in RF and antenna engineering to design array patterns with optimized, equal-ripple side lobes.\n * **Pochhammer Symbol**: Represents the rising factorial notation in generalized hypergeometric series expansions.\n * **Gauss's Hypergeometric Equation**: A linear second-order ODE possessing exactly three regular singular points located at 0$,$1$, and$\infty on the extended complex plane.\n * **Confluent Hypergeometric Equation**: Formed by coalescing singular points, resulting in exactly two singular points (one regular, one irregular).\n* **Classification of Partial Differential Equations (PDEs)**: Determined by the discriminant B^2 - 4AC for second-order PDEs in two variables:\n * **Elliptic (B^2 - 4AC < 0$)**: Modeled by Laplace's Equation ($ abla^2 u = 0$), representing steady-state phenomena where quantities do not change with time. Adding a non-zero source density function transforms it into **Poisson's Equation** ($ abla^2 u = f).\n * **Parabolic (B^2 - 4AC = 0)**: Modeled by the 1D Heat (Diffusion) Equation. Solutions rapidly dissipate and smooth out disturbances over time.\n * **Hyperbolic (B^2 - 4AC > 0)**: Modeled by the 1D Wave Equation. Solutions propagate initial disturbances without dissipation. Lossless transmission lines modeled by Telegrapher's Equations decouple into the 1D Wave Equation.\n* **PDE Solutions & Boundary Conditions**: \n * **Neumann Boundary Condition**: Specifies a heat flux (normal derivative or temperature gradient) across a physical boundary.\n * **Separation of Variables (Vibrating String)**: Spatial boundary conditions with fixed ends force spatial solutions to consist of sine functions evaluating to zero at both boundaries.\n * **D'Alembert's Formula**: Provides the explicit time-domain analytical solution for the 1D Wave Equation.\n * **Method of Characteristics**: Analytical technique predominantly utilized to solve first-order hyperbolic partial differential equations.\n\n# Complex Variable Theory & Mappings\n\n* **Cauchy-Riemann Equations**: Continuous complex function f(z) = u(x,y) + j v(x,y)$is analytic if and only if its real and imaginary parts satisfy$\frac{\partial u}{\partial x} = \frac{\partial v}{\partial y}$and$\frac{\partial u}{\partial y} = -\frac{\partial v}{\partial x}.\n* **Harmonic Functions & Harmonic Conjugates**: If f(z) = u + j v$is analytic, both$u(x,y)$and$v(x,y)$satisfy Laplace's equation and are harmonic. The function$v(x,y)$is defined as the harmonic conjugate of$u(x,y).\n* **Non-Analytic Complex Functions**: The complex conjugate function f(z) = \bar{z} is continuous everywhere in the complex plane, but analytic nowhere.\n* **Conformal Mapping**: A mapping implemented by an analytic complex function that preserves both the magnitude and the sense of direction of angles between intersecting curves.\n* **Complex Vector Rotations**: Geometrically, multiplying a complex vector z$by the imaginary unit$j$rotates the vector counter-clockwise by exactly$90^\circ$($\frac{\pi}{2}\text{ rad}).\n* **Cauchy's Integral Theorem**: If f(z)$is analytic inside and on a simple closed contour$C$, then$\oint_C f(z)\,dz = 0.\n* **Morera's Theorem**: The converse of Cauchy's Integral Theorem; states that if a continuous function f(z)$integrates to zero around every closed contour in a simply connected domain,$f(z) must be analytic throughout that domain.\n* **Cauchy's Integral Formula**: Integrating \frac{f(z)}{z - a}$around a closed contour enclosing$a$yields$2\pi j \cdot f(a).\n* **Residue Theorem**: The line integral of f(z)$around a closed contour enclosing multiple poles equals$2\pi j multiplied by the sum of all enclosed residues.\n* **Singularities & Laurent Series**: The function f(z) = \frac{1}{z}$exhibits a simple pole at$z = 0$. A Laurent series expands complex functions around isolated singularities by including negative integer powers$(z - z_0)^{-n}, unlike a Taylor series.\n* **Liouville's Theorem**: Asserts that every bounded entire (analytic everywhere) complex function must be a constant. This theorem provides the direct proof for the Fundamental Theorem of Algebra.\n* **Maximum Modulus Principle**: States that a non-constant analytic function cannot attain a local maximum of its absolute value inside the interior of its domain.\n\n# Numerical Methods: Root Finding & Optimization\n\n* **Bisection Method**: A bracketing method requiring function values at initial boundaries to have opposite signs (f(a)f(b) < 0$). Mathematically guaranteed to converge by the Intermediate Value Theorem. To reach an absolute error tolerance$E$, the required number of iterations$n$is calculated by$n = \log_2\left(\frac{b - a}{E}\right).\n* **Regula-Falsi Method (False Position)**: A bracketing method that uses consecutive secant lines while systematically retaining the prior boundary point that maintains a bracket around the root.\n* **Newton-Raphson Method**: An open iterative method that uses the tangent line at an initial guess to extrapolate the x-intercept. Exhibits quadratic asymptotic convergence (p = 2$) for simple roots. If a root has multiplicity$m > 1$, multiplying the correction term by$m restores quadratic convergence. For multivariable non-linear systems, the method requires computing the inverse of the Jacobian matrix.\n* **Secant Method**: An open method approximating derivatives using consecutive secant lines. Does not require analytical derivative evaluations, making it computationally efficient per iteration. Asymptotic order of convergence is p \approx 1.618 (the Golden Ratio).\n* **Müller's Method**: Root-finding technique that fits a parabola through three coordinate points instead of a straight line through two points, enabling it to compute complex roots.\n* **Direct Search Methods**: Applicable for root finding when functions are discontinuous or non-differentiable.\n* **Gradient Vector in Optimization**: Evaluated at a physical coordinate, the gradient vector indicates the exact direction of the most rapid increase of a multivariable function.\n* **Transcendental Equations**: Non-algebraic equations containing trigonometric, exponential, or logarithmic functions (e.g., x \cos(x) - e^x = 0), requiring iterative numerical approaches.\n\n# Numerical Linear Algebra & Linear Systems\n\n* **Singular Matrices & Condition Numbers**: A matrix is singular if its determinant is zero (\det(A) = 0). The condition number is a scalar metric measuring system sensitivity to input data errors or perturbations.\n* **Matrix Operations & Dimensions**: Multiplying A_{m \times n}$by$x_{n \times p}$yields a product matrix$Ax_{m \times p}$. Multiplying a square non-singular matrix by its inverse yields the Identity Matrix ($A \cdot A^{-1} = I).\n* **Cramer's Rule**: Solves linear systems via ratios of determinants. Fails entirely when \det(A) = 0, indicating either no solution or infinitely many solutions.\n* **Gaussian Elimination & Pivoting**: Standard direct solver for dense linear systems. Partial pivoting is applied to avoid division by zero and minimize round-off errors.\n* **Gauss-Jordan Elimination**: Reduces coefficient matrices to reduced row echelon form, eliminating the necessity for backward substitution.\n* **LU Factorization**: Factoring A = L U$minimizes computational effort when analyzing fixed topology systems subjected to varying right-hand side vectors. In Doolittle's algorithm, diagonal elements of$L$are ones ($1$s). In Crout's algorithm, diagonal elements of$U$are ones ($1s).\n* **Iterative Linear Solvers**: Jacobi splitting isolates the diagonal matrix D. Gauss-Seidel converges faster because it immediately incorporates newly updated variable values within the current iteration step. Absolute convergence for any initial guess is guaranteed if the coefficient matrix is diagonally dominant.\n* **System Inconsistency**: A system Ax = b$is inconsistent and has no solution if the rank of the augmented matrix$[A \mid b]$exceeds the rank of the coefficient matrix$A$.

Numerical Differentiation, Integration, Interpolation & Regression

• Error Analysis & Tradeoffs: Truncation error results from approximating continuous math operations with finite steps. Round-off error stems from floating-point arithmetic bounds, defined by Machine Epsilon (smallest positive $\epsilon$ where $1 + \epsilon > 1$). Making step size $h$ excessively small in numerical differentiation causes catastrophic cancellation (subtracting nearly identical numbers), sharply increasing round-off error.
• Finite Difference Approximations: Central difference formulas are most accurate because they utilize symmetric data points on both sides of interest. Newton's Forward Difference formula is best applied near the top/beginning of tabular data; Newton's Backward Difference formula is preferred near the bottom/end.
• Numerical Integration Techniques:
  
  • Trapezoidal Rule: Approximates integrands using first-degree polynomials (straight lines). Composite Trapezoidal adapts well to unequal step sizes.
  • Simpson's 1/3 Rule: Approximates integrands using piecewise parabolic arcs. Requires an even number of subintervals (odd number of points); cannot be applied unmodified across odd subinterval counts.
  • Simpson's 3/8 Rule: Requires total equal subintervals $n$ to be a multiple of 3$.\n * **Midpoint vs. Trapezoidal Error**: For functions with positive constant second derivative, Trapezoidal Rule error is twice as large as Midpoint Rule error and opposite in sign.\n * **Gaussian Quadrature**: Achieves maximum accuracy by evaluating integrands at non-uniformly spaced interior nodes; relies on Legendre polynomials.\n * **Richardson Extrapolation**: Combines step size approximations h$and$\frac{h}{2} to eliminate lowest-order error terms.\n* **Interpolation vs. Regression**: Interpolation curves must pass exactly through every data point (Lagrange weighting functions evaluate to 1$at their specific point and$0 elsewhere). Regression curves capture general trends without passing through every point.\n* **Cubic Splines**: Eliminates severe edge oscillations (Runge's phenomenon) seen in high-degree polynomial interpolation. Natural Cubic Spline sets second derivatives at boundaries to zero (y'' = 0); Clamped Cubic Spline sets endpoint slopes equal to the exact derivative of the original function.\n* **Least Squares Regression**: Normal equations are derived using the matrix transpose of the design matrix. A residual is the vertical distance between an observed data point and the model curve. Least squares regression isolates general trends from data containing random noise. Total Sum of Squares (SST) quantifies total variance around the mean.\n\n# System Dynamics, Signals & Transforms\n\n* **Laplace Transforms**: \n * **Definition**: \mathcal{L}{f(t)} = \int_0^\infty f(t) e^{-st}\,dt$. Transform of unit step$u(t)$is$\frac{1}{s}.\n * **Transfer Function**: Ratio of output to input transforms assuming zero initial conditions. Equal to the Laplace transform of the impulse response.\n * **Derivative Property**: \mathcal{L}\left{\frac{df(t)}{dt}\right} = s F(s) - f(0), incorporating initial conditions algebraically.\n * **Integration Property**: Time integration corresponds to dividing F(s)$by$s$.
  • Shifting & Delay: Pure time delay $T$ multiplies transform by $e^{-sT}$. Multiplying time signal by $e^{-at}$ shifts spectrum to $F(s + a)$.
  • Stability Criteria: A continuous system is stable if all poles lie in the left-half s-plane (LHP). The Final Value Theorem fails if poles reside on the imaginary axis or in the right-half plane (RHP).
  • Impedance Representation: An uncharged capacitor $C$ is represented as $\frac{1}{sC}$. Complex conjugate poles in the LHP denote an underdamped physical response.
• Fourier Series & Transforms:
  
  • Fourier Series: Represents periodic continuous-time signals satisfying Dirichlet conditions. Fundamental frequency corresponds to the lowest harmonic matching the overall period. Even periodic functions contain only cosine terms and DC constants; half-wave symmetry satisfies $f(t) = -f\left(t + \frac{T}{2}\right)$. Reconstructing step discontinuities via truncated series creates Gibbs phenomenon ($9\%$ overshoot).
  • Fourier Transform: Generalizes to continuous aperiodic signals. (Unstable signals require Laplace transforms because they fail absolute integrability required for Fourier convergence). Purely real and even time signals yield purely real and even frequency spectra. Transform of Dirac delta $\delta(t)$ is a constant value of $1$. Time compression expands the frequency spectrum and scales amplitude. Time multiplication corresponds to frequency domain convolution scaled by $\frac{1}{2\pi}$. LTI output spectrum: $Y(j\omega) = X(j\omega) H(j\omega)$. Parseval's Theorem asserts equivalence between total average time-domain power and the sum of harmonic powers.
• Z-Transforms & Discrete Systems:
  
  • Definition & Basic Transforms: Discrete equivalent of Laplace transform. Transform of Kronecker delta $\delta[n]$ is $1$; transform of unit step $u[n]$ is $\frac{1}{1 - z^{-1}}$ for ROC $|z| > 1$.
  • Properties & Mappings: Unit time delay $x[n - 1]$ multiplies transform by $z^{-1}$. Continuous s-plane maps to z-plane via $z = e^{sT}$, mapping the stable left-half s-plane into the interior of the unit circle ($|z| < 1$). Bilinear (Möbius) transformation maps LHP directly into unit circle interior.
  • Stability & Causality: Causal discrete LTI systems are unconditionally stable if all poles lie inside the unit circle ($|z| < 1$). A pole at $z = 1.2$ indicates instability. A pole at $z = -1$ generates oscillations at the Nyquist frequency. Initial Value Theorem: $x[0] = \lim_{z \to \infty} X(z)$. Final Value Theorem multiplies $X(z)$ by $(1 - z^{-1})$ before taking the limit as $z \to 1$. ROC for right-sided sequences extends outward from the outermost pole to infinity. Frequency response evaluates the Z-transform along the unit circle.

Initial Value Problems for Ordinary Differential Equations

• Euler's Method: Relies on the continuous application of a first-order Taylor series expansion.
• Forward vs. Backward Euler: Forward Euler evaluates derivatives at current known states (conditionally stable). Backward Euler evaluates derivatives at unknown future states (unconditionally stable, requiring algebraic equation solvers).
• Single-Step vs. Multi-Step Methods: Single-step methods (e.g., Runge-Kutta) rely exclusively on the current state point to advance. Multi-step methods (e.g., Adams-Bashforth) utilize information from several previous step points to advance to the next point.
• Stiff Systems of ODEs: Occurs when solution components decay at drastically different rates, requiring exceptionally small step sizes for explicit methods to remain numerically stable.

Probability, Inferential Statistics & Experimental Design

• Probability Rules & Discrete Distributions:
  
  • Complement Rule: $P(A) = 1 - P(A')$.
  • Multiplication Rule: Joint probability $P(A \cap B) = P(B) P(A \mid B)$. If independent, simplifies to $P(A \cap B) = P(A) P(B)$.
  • Independent Variables Covariance: Statistically independent variables strictly possess covariance equal to $0$
  • Geometric Distribution: Models the exact number of Bernoulli trials required to obtain the very first success.
  • Poisson Distribution: Models counts of independent events occurring in fixed continuous intervals. The time elapsed between successive Poisson events follows an Exponential distribution (used in reliability for constant failure rates).
  • Hypergeometric Distribution: Models sampling without replacement from a finite population with known success counts.
• Hypothesis Testing & Inference:
  
  • Null Hypothesis ($H_0$): Formalizes statements of "no difference", "no effect", or the "status quo".
  • Statistical Power ($1 - \beta$): Probability of correctly rejecting a false null hypothesis. Increasing sample size while holding significance level constant strictly decreases Type II error probability ($\beta$).
  • p-value: Probability of observing a test statistic at least as extreme as the computed value, assuming the null hypothesis is true.
  • Student's t-test: Used to test population means when population variance is unknown and sample size is small ($n < 30$).
  • F-distribution: Governs tests comparing the variances of two independent normal populations.
  • Standard Normal (Z) Distribution: Used to approximate single population proportion test statistics under large sample sizes.
  • Statistical Intervals: Confidence intervals bracket population parameters (center value is the point estimate sample mean). Increasing confidence level widens the interval. Prediction intervals estimate single future observations. Tolerance intervals capture specified proportions of individual population observations.
• Regression Analysis & Experimental Design:
  
  • Normality Assumption: Assumes residuals are normally distributed with mean equal to zero ($\mu = 0$).
  • Heteroscedasticity: Occurs when residual variance is non-constant across fitted values.
  • Influential / High-Leverage Points: Data points with extreme x-values exerting disproportionate pull on the regression slope.
  • Extrapolation: Predicts response variable $y$ using x-values far outside original dataset bounds; dangerous and often invalid.
  • ANOVA Degrees of Freedom: For $n$ data points in simple linear regression, Error Sum of Squares (SSE) has $n - 2$ degrees of freedom. F-statistic equals Mean Square Regression divided by Mean Square Error.
  • Exploratory Data Tools: Box plots use 5-number summaries (minimum, Q1, median, Q3, maximum) to visualize outliers. Empirical rule dictates $99.7\%$ of normal observations lie within $\pm 3\sigma$ of the mean.
  • Observational Studies: Data gathered from logs without actively controlling variables.
  • Blocking Factors: Known, controllable factors grouped to isolate variability from experimental error.
  • Full Factorial Design: 3 factors at 2 levels require $2^3 = 8$ unique treatment combinations per replicate.
  • Systematic Sampling: Selects a random starting point and tests every $k$-th item (e.g., every 50th meter).