Methods lecture 2 Classification of Differential Equations, Taylor Series Expansions, and Finite Difference Discretization

Classification of Differential Equations: Ordinary, Partial, Initial Value, and Boundary Value Problems

• Fundamental Classification by Independent Variables:

  • An ordinary differential equation (ODE) contains exactly one independent variable.
  • A partial differential equation (PDE) contains multiple independent variables (strictly greater than one).

• Initial Value Problems (IVPs):

  • An initial value problem is defined as a differential system where all boundary/initial conditions are specified at a single point in the independent variable domain.
  • Because all constraints are anchored at a single location, the state of the system at any subsequent point depends strictly on the state at the initial point and intermediate history leading up to that point.
  • The flow of physical information in an IVP is strictly unidirectional, propagating along a forward trajectory.
  • Physical Example: A three-mass vibration system undergoing transverse displacement $y$.
    • Governed by a second-order ordinary differential equation in time $t$.
    • Requires exactly two initial conditions specified at the base time $t = 0$.
    • Initial deflection: $y(0) = 1\,\text{mm}$.
    • Initial velocity / derivative: $y'(0)$ or $\dot{y}(0)$, representing the initial slope.
    • Damping behavior is governed by the damping coefficient $c$. If $c = 0$, the system exhibits zero damping, resulting in undamped harmonic vibration represented by a pure cosine response: $\cos(\omega t)$.

• Boundary Value Problems (BVPs):

  • A boundary value problem is defined as a differential system where conditions are specified at two or more distinct points of the independent variable domain.
  • Information does not propagate in a single direction; the solution at any interior point depends simultaneously on conditions on both sides of the domain.
  • Physical Example: Steady-state heat conduction across a solid block governed by a second-order ODE where temperature $T$ is a function of the spatial coordinate $x$ ($T = T(x)$).
    • Because the equation is second-order with respect to space, two conditions must be specified at different boundary locations (e.g., at $x = 0$ and $x = L$).
    • If the boundary temperature on the right end is abruptly altered from $500$ degrees to $1000$ degrees, or cooled to $0$ degrees, or subjected to heat flux via a parabolic solar collector directing concentrated solar rays onto the boundary, that change propagates inward and impacts the temperature at every single interior point across the entire domain.

Information Propagation: One-Way Versus Two-Way Coordinates

• One-Way Coordinates:

  • An independent variable along which physical information propagates strictly in one direction is termed a one-way coordinate.
  • Time ($t$) serves as the primary physical archetype of a one-way coordinate.
  • The solution state at any future time $t$ is entirely determined by prior history and the immediate local state; it cannot depend on states at subsequent or future times.
  • Numerical Design Constraint: Numerical algorithms formulated for IVPs must reflect this physical causality by computing approximations using only current and prior states.

• Two-Way Coordinates:

  • An independent variable along which physical information travels in both directions simultaneously is termed a two-way coordinate.
  • Spatial dimensions ($x$, $y$, $z$) typically function as two-way coordinates.
  • Interior points act as bidirectional receivers where states are governed by upstream and downstream boundary conditions concurrently.

• Mixed Propagation in Partial Differential Equations:

  • Many physical PDEs exhibit mixed coordinate behaviors.
  • Example: A PDE second-order in space $x$ and first-order in time $t$ for field variable $u(x, t)$.
    • Requires specification of an initial field profile across all spatial coordinates at the base time: $u(x, 0)$.
    • Requires two spatial boundary conditions at the domain extremes for all $t > 0$.
    • Information propagates unidirectionally along the temporal coordinate $t$ and bidirectionally along the spatial coordinate $x$.

System Reduction: Converting Higher-Order Equations to First-Order Systems

• Motivation for System Reduction:

  • Standard numerical solvers and algorithmic integrators are designed exclusively to integrate systems of first-order differential equations.
  • Any single higher-order differential equation of order $n$ can be systematically recast into an equivalent coupled system of $n$ first-order differential equations.

• Transformation Procedure for an $n$-th Order ODE:

  • Introduce a set of $n$ auxiliary state variables representing the function and its successive derivatives up to order $n - 1$.
  • Demonstration for a Third-Order Equation ($n = 3$):
    • Define the primary state variable: $p(x)$.
    • Define the second state variable as the first derivative of $p$: $q = \frac{dp}{dx}$.
    • Differentiate $q$ with respect to $x$ to obtain the second derivative: $\frac{dq}{dx} = \frac{d^2p}{dx^2}$.
    • Represent the third derivative $\frac{d^3p}{dx^3}$ by isolating it in terms of $p$, $q$, and $\frac{dq}{dx}$.
    • Linear Algebraic Rearrangement Example: For an equation containing terms such as $7\frac{d^2p}{dx^2} + 14\frac{dp}{dx} - 8p$, moving terms across the equality yields a closed system of three first-order ODEs:
      • $\frac{dp}{dx} = q$
      • $\frac{dq}{dx} = r$
      • $\frac{dr}{dx} = f(x, p, q, r)$
  • Multibody Dynamics Example: The mechanical modeling of a basketball trajectory utilizes a system of $6$ coupled first-order ODEs, which is fundamentally equivalent to a single $6\text{th}$-order differential equation collapsed into component state variables.

Mathematical Foundations of Taylor Series and Continuity Requirements

• Role of Taylor Series in Numerical Analysis:

  • Taylor series expansions provide the foundational theoretical framework for constructing finite difference approximations and discretization schemes for differential operators.
  • Taylor series converts differential operators into algebraic expressions relating discrete nodal values.

• Continuity and Differentiability Hypotheses:

  • Classical Taylor series expansion relies on the prerequisite that the target function $f(x)$ is continuous and infinitely differentiable ($C^\infty$) across the analytical domain.
  • Continuous and Differentiable Example: The sine function $\sin(x)$ is defined, continuous, and smoothly differentiable across all $x \in \mathbb{R}$.
  • Continuous but Non-Differentiable Example (Ramp / Triangular Wave):
    • A continuous ramp or saw-tooth wave contains sharp directional changes ("kinks").
    • At each kink, the first derivative $\frac{df}{dx}$ is undefined because the left-sided limit of the slope does not equal the right-sided limit of the slope:
      • $\lim_{x \to x_k^-} \frac{df}{dx} \neq \lim_{x \to x_k^+} \frac{df}{dx}$
    • This results in a jump discontinuity in the derivative profile, violating the smooth differentiability requirement.
    • Analytical Remedy: Functions with discontinuous slopes can be approximated using Fourier series expansions, which synthesize non-smooth waveforms into infinite linear combinations of continuous sine and cosine basis functions.

• Primary Engineering Applications of Taylor Series:

  • Derivation of power series representations for transcendental functions.
  • Formulation of finite difference algebraic approximations for derivatives within differential equations.

Taylor Series Formulation and Truncation Error

• General Formulation Around an Anchor Point:

  • Let $x_0$ represent an anchor (base) point where the function value and all higher-order derivatives are known.
  • The function value at an arbitrary neighboring target point $x$ is given by the infinite series:
    • $f(x) = f(x_0) + f'(x_0)(x - x_0) + \frac{f''(x_0)}{2!}(x - x_0)^2 + \frac{f'''(x_0)}{3!}(x - x_0)^3 + \dots + \frac{f^{(n)}(x_0)}{n!}(x - x_0)^n + \dots$

• Power Series Expansion of $f(x) = e^x$ Around $x_0 = 0$:

  • Base function value: $f(0) = e^0 = 1$.
  • Derivatives: $f^{(n)}(x) = e^x \implies f^{(n)}(0) = e^0 = 1$ for all integers $n \ge 1$.
  • Substituting these derivatives into the Taylor series yields the standard exponential power series:
    • $e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \dots$

• Convergence and Error Mechanics:

  • Approximating $e^1$ (Exact analytical value: $e^1 \approx 2.7182818\dots$):
    • One-term approximation: $e^1 \approx 1$ (high percentage truncation error).
    • Two-term approximation: $e^1 \approx 1 + 1 = 2$.
    • Three-term approximation: $e^1 \approx 1 + 1 + \frac{1}{2} = 2.5$.
    • Four-term approximation: $e^1 \approx 1 + 1 + 0.5 + \frac{1}{6} \approx 2.6667$.
    • Incorporating successive higher-order terms systematically decreases the percentage error toward machine precision.
  • Effect of Step Size on Truncation Error:
    • Evaluating the series at $x = 0.1$ yields significantly higher accuracy with fewer polynomial terms compared to evaluation at $x = 1.0$.
    • Proximity principle: The closer the target evaluation point $x$ is to the anchor point $x_0$ (i.e., the smaller the displacement $\Delta x = x - x_0$), the fewer terms are required to achieve a specified error tolerance.
  • Series Truncation: Numerical implementations cannot sum infinite terms. Series are truncated after a designated number of terms (e.g., retaining the first four terms), discarding all higher-order derivatives as the truncation error residue.

Spatial Domain Discretization and Grid Indexing

• One-Dimensional Discretization Mechanics:

  • A continuous spatial domain of length $L$ bounded between coordinates $x_0$ and $x_N$ is partitioned into discrete sub-intervals.
  • By convention, the domain origin can be defined such that $x_0 = 0$.
  • Partitioning the physical interval into $N$ discrete segments requires exactly $N + 1$ discrete boundary and interior grid points.

• Grid Point Numbering and Coordinate Mapping:

  • Grid points are indexed using integer subscripts: $x_0, x_1, x_2, \dots, x_i, \dots, x_N$.
  • Index ranges: $i \in \{0, 1, 2, \dots, N\}$.
  • In higher-dimensional domains, spatial coordinates utilize distinct spatial indices:
    • $i$ for the $x$-coordinate.
    • $j$ for the $y$-coordinate.
    • $k$ for the $z$-coordinate.
    • $n$ is typically reserved as a superscript or subscript for the temporal index ($t$).

• Uniform Grid Step Size ($h$):

  • On a uniform (equidistant) mesh, the step size $h$ is constant throughout:
    • $h = x_1 - x_0 = x_2 - x_1 = x_{i+1} - x_i$
  • For a total of $N$ uniform segments over span $x_N - x_0$:
    • $h = \frac{x_N - x_0}{N}$
  • The exact physical coordinate of the $i$-th node is determined algebraically:
    • $x_i = x_0 + i \cdot h$
    • When $x_0 = 0$, this simplifies to $x_i = i \cdot h$.
  • Non-Uniform Grids: Equidistant spacing simplifies algebraic formulations, but non-uniform grids are deployed when localized physical gradients demand localized spatial resolution.

• Compact Discrete Function Notation:

  • The value of the continuous function $f(x)$ evaluated at a discrete grid point $x_i$ is denoted in compact subscript form:
    • $f(x_i) \equiv f_i$
    • $f(x_{i+1}) \equiv f_{i+1}$
    • $f(x_{i-1}) \equiv f_{i-1}$
    • $f(x_{i+2}) \equiv f_{i+2}$
    • $f(x_{i-2}) \equiv f_{i-2}$

Discrete Grid Taylor Series Expansions

• Forward Taylor Series Expansion ($x_{i+1}$ about $x_i$):

  • Anchor point: $x_i$.
  • Target forward point: $x_{i+1} = x_i + h$.
  • Displacement: $x_{i+1} - x_i = h$.
  • Expansion:
    • $f_{i+1} = f_i + h f'_i + \frac{h^2}{2!} f''_i + \frac{h^3}{3!} f'''_i + \frac{h^4}{4!} f^{(4)}_i + \dots$

• Backward Taylor Series Expansion ($x_{i-1}$ about $x_i$):

  • Anchor point: $x_i$.
  • Target backward point: $x_{i-1} = x_i - h$.
  • Displacement: $x_{i-1} - x_i = -h$.
  • Because odd powers of $-h$ produce negative signs and even powers produce positive signs, the series alternates:
    • $f_{i-1} = f_i - h f'_i + \frac{h^2}{2!} f''_i - \frac{h^3}{3!} f'''_i + \frac{h^4}{4!} f^{(4)}_i - \dots$

• Extended Forward and Backward Multi-Step Expansions:

  • Double-step forward ($x_{i+2}$ about $x_i$):
    • Target point: $x_{i+2} = x_i + 2h$.
    • Displacement: $x_{i+2} - x_i = 2h$.
    • Expansion:
      • $f_{i+2} = f_i + 2h f'_i + \frac{(2h)^2}{2!} f''_i + \frac{(2h)^3}{3!} f'''_i + \dots$
      • $f_{i+2} = f_i + 2h f'_i + 2h^2 f''_i + \frac{4h^3}{3} f'''_i + \dots$
  • Double-step backward ($x_{i-2}$ about $x_i$):
    • Target point: $x_{i-2} = x_i - 2h$.
    • Displacement: $x_{i-2} - x_i = -2h$.
    • Expansion:
      • $f_{i-2} = f_i - 2h f'_i + \frac{(-2h)^2}{2!} f''_i + \frac{(-2h)^3}{3!} f'''_i + \dots$
      • $f_{i-2} = f_i - 2h f'_i + 2h^2 f''_i - \frac{4h^3}{3} f'''_i + \dots$

Finite Difference Approximations of Derivatives

• Algorithmic Objective:

  • To replace continuous derivative operators ($f'_i$, $f''_i$, etc.) with algebraic expressions involving only the discrete function values ($f_i$, $f_{i+1}$, $f_{i-1}$, etc.), eliminating higher-order analytical derivatives from the computational scheme.

• Derivation of the First-Order Forward Difference Formula:

  • Truncate the forward Taylor series expansion after the linear derivative term:
    • $f_{i+1} = f_i + h f'_i + \mathcal{O}(h^2)$
  • Isolate the derivative term $f'_i$:
    • $h f'_i = f_{i+1} - f_i - \mathcal{O}(h^2)$
  • Divide through by the grid step size $h$:
    • $f'_i = \frac{f_{i+1} - f_i}{h} + \mathcal{O}(h)$
  • The leading truncation error term is proportional to the first power of $h$, classifying this as a first-order accurate forward difference approximation.

• Geometric and Physical Interpretation:

  • The continuous derivative $f'(x_i)$ represents the exact tangent slope to the function curve at coordinate $x_i$.
  • The finite difference quotient $\frac{f_{i+1} - f_i}{h}$ represents the slope of the secant line intersecting the two discrete coordinate points $(x_i, f_i)$ and $(x_{i+1}, f_{i+1})$.
  • Secant slope formula corresponds directly to the "rise over run" formulation:
    • $\text{Rise} = f_{i+1} - f_i$
    • $\text{Run} = x_{i+1} - x_i = h$
    • $\text{Secant Slope} = \frac{\text{Rise}}{\text{Run}} = \frac{f_{i+1} - f_i}{h}$

• Impact of Stencil Width on Approximation Accuracy:

  • Using a two-point stencil ($x_i$, $x_{i+1}$) yields a basic first-order estimate.
  • Incorporating additional information—such as two points on either side (a four-point stencil) or symmetric three-point and five-point stencils—cancels lower-order error terms in the Taylor expansions, yielding higher-order accurate algebraic formulas for the derivative.