Mathematical Analysis: Functions of Several Variables - Encyclopedic Study Guide

PREFACE AND COURSE INTRODUCTION

• Mathematics 1b: Functions of Several Variables (2026 Edition)

• Authors: Ole Christensen and Jakob Lemvig, DTU Compute.

• Built on the foundations of Mat 1a (Logic and Linear Algebra).

• Core Objectives:

  • Analyzing systems in multiple dimensions.

  • Differentiation in several variables: Determining rates of change in multiple directions (e.g., biology: plant growth vs. light/water).

  • Integration: Aggregating effects over regions (volume, energy, cumulative rainfall).

  • Modern Applications: Aerodynamic structures, electromagnetic fields, and optimizing machine learning/artificial intelligence.

PRELIMINARIES (CHAPTER 0)

Scalars and Fields

• Fields ($F$): Typically denote the real numbers ($R$) or complex numbers ($C$).

• Complex Numbers ($C$): Of the form $c = c<em>1 + ic</em>2$.

  • $cc = c<em>1^2 + c</em>2^2 = |c|^2$

  • $\overline{c+d} = \overline{c} + \overline{d}$

  • $c + \overline{c} = 2 \text{Re } c$

• Real Numbers ($R$): Defined as a unique complete ordered field.

  • Supremum (Least Upper Bound): $L = \sup(A)$ if $L$ is an upper bound and \forall M < L, $M$ is not an upper bound.

  • Infimum (Greatest Lower Bound): $L = \inf(A)$ if $L$ is a lower bound and \forall M > L, $M$ is not a lower bound.

  • Supremum Axiom: Any non-empty subset of $R$ bounded above has a supremum in $R$ (distinguishes $R$ from $Q$).

Sets and Functions

• Function Definition: A correspondence associating each element $x \in A$ to a unique $y \in B$.

  • Domain: $A$ ($\text{dom}(f)$).

  • Codomain: $B$ ($\text{co-dom}(f)$).

  • Range (Image): $\text{im}(f) = {f(x) | x \in A}$.

• Function Types:

  • Injective (One-to-one): $x<em>1 \neq x</em>2 \implies f(x<em>1) \neq f(x</em>2)$.

  • Surjective (Onto): $\text{im}(f) = B$.

  • Bijective: Both injective and surjective.

Matrix Theory Essentials

• Matrix Types:

  • Hermitian: $A^* = A$ (where $A^* = \overline{A}^T$).

  • Unitary: $A^*A = I$.

  • Orthogonal (Real): $A^T A = I$.

  • Normal: $A^<em>A = AA^</em>$.

  • Idempotent: $A^2 = A$ (e.g., projection matrices).

  • Trace: $\text{tr } A = \sum<em>{i=1}^n a</em>{ii}$.

FUNCTIONS OF SEVERAL VARIABLES (CHAPTER 1)

Scalar vs. Vector Functions

• Scalar Fields: $f: R^n \to R$. Maps vectors to real numbers.

• Vector Fields: $f: R^n \to R^n$. Maps vectors to vectors.

• Quadratic Forms: $q(x) = x^T Ax + x^T b + c$.

  • Represents multivariate polynomials of degree at most two.

Machine Learning Building Blocks

• Activation Functions:

  • ReLU (Rectified Linear Unit): $\text{ReLU}(x) = \max(x, 0)$. Vectorized, it is applied coordinate-wise.

  • Sigmoid: $x \mapsto (1 + e^{-x})^{-1}$.

  • Softmax: Mapping $R^n \to R^n$. Acts as a "soft" argmax to create probability vectors.

    • $\text{softmax}(x)<em>i = \frac{e^{x</em>i}}{\sum<em>{j=1}^n e^{x</em>j}}$.

• Neural Networks (Feed-forward):

  • A composition of $L$ layers ($\Phi = f<em>L \circ f</em>{L-1} \circ \dots \circ f_1$).

  • Each layer is an affine transformation followed by a non-linear activation function: $f<em>\ell(z) = \sigma</em>\ell(A<em>\ell z + b</em>\ell)$.

  • Deep Learning: Defined as networks with L > 3.

Visualization Tools

• Graphs: Subset of $R^{n+1}$ defined by ${(x, f(x)) | x \in \text{dom}(f)}$.

• Level Sets: $\text{constant} = f(x, y)$. Useful for contours (circles, ellipses, hyperbolas).

INNER PRODUCT SPACES AND SPECTRAL THEOREM (CHAPTER 2)

Norms and Inner Products

• Euclidean Norm ($L<em>2$): $||x|| = \sqrt{\sum |x</em>i|^2}$.

• Inner Product ($\langle x, y \rangle$): In $F^n$, defined as $y^* x$.

  • Cauchy-Schwarz Inequality: $|\langle x, y \rangle| \leq ||x|| \cdot ||y||$.

  • Triangle Inequality: $||x + y|| \leq ||x|| + ||y||$.

  • Pythagorean Theorem: If $x \perp y$, then $||x+y||^2 = ||x||^2 + ||y||^2$.

The Spectral Theorem

• Statement: Every real symmetric matrix $A$ can be decomposed as $A = Q\Lambda Q^T$.

  • $Q$ is a real orthogonal matrix containing eigenvectors as columns.

  • $\Lambda$ is a diagonal matrix of real eigenvalues.

• Gram-Schmidt Process: Algorithm to turn a linearly independent set into an orthonormal basis through projection subtraction: $w<em>k = v</em>k - \sum<em>{j=1}^{k-1} \text{proj}</em>{u<em>j}(v</em>k)$.

CONTINUITY AND DIFFERENTIABILITY (CHAPTER 3)

Partial Derivatives and Gradients

• Partial Derivative: $\frac{\partial f}{\partial x<em>j}(x) = \lim</em>{h \to 0} \frac{f(x + h e_j) - f(x)}{h}$.

• Gradient Vector ($\nabla f$): A vector containing all first-order partial derivatives.

  • Directional Derivative: $\nabla_u f(x) = \langle u, \nabla f(x) \rangle$ for a unit vector $u$.

  • Gradient points in the direction of steepest ascent.

Total Differentiability

• Higher-dimensional analog to the derivative. Requires a linear map $L$ (Jacobian) such that the error goes to zero faster than the step size:

  • $f(x<em>0 + h) - f(x</em>0) - \nabla f(x_0)^T h - \epsilon(h)||h|| = 0$.

• Jacobian Matrix ($J_f$): The matrix of first-order partial derivatives for vector functions.

• Hessian Matrix ($H_f$): The square matrix of second-order partial derivatives.

  • If $f$ is $C^2$, the Hessian is symmetric ($\frac{\partial^2 f}{\partial x<em>i \partial x</em>j} = \frac{\partial^2 f}{\partial x<em>j \partial x</em>i}$).

TAYLOR APPROXIMATIONS (CHAPTER 4)

• First-degree (Tangent Plane): $P<em>1(x) = f(x</em>0) + \nabla f(x<em>0)^T (x - x</em>0)$.

• Second-degree (Quadratic Approximation):

  • $P<em>2(x) = f(x</em>0) + \nabla f(x<em>0)^T (x - x</em>0) + \frac{1}{2}(x - x<em>0)^T H</em>f(x<em>0)(x - x</em>0)$.

• Taylor's Formula: Quantifies error $R<em>k(x) = f(x) - P</em>k(x)$.

OPTIMIZATION (CHAPTER 5)

• Stationary Points: Where $\nabla f(x) = 0$.

• Second Derivative Test:

  • Local Minimum: $H<em>f(x</em>0)$ is positive definite (all eigenvalues > 0).

  • Local Maximum: $H<em>f(x</em>0)$ is negative definite (all eigenvalues < 0).

  • Saddle Point: $H<em>f(x</em>0)$ has mixed signs in eigenvalues.

INTEGRATION (CHAPTER 6)

• Riemann Integral: Limit of Riemann sums $S<em>J = \sum f(\xi</em>i) \text{vol}(Q_i)$.

• Fubini's Theorem: Allows iterated integration over rectangles.

• Change of Variables: $\iint<em>{r(\Gamma)} f(x,y) dx dy = \iint</em>{\Gamma} f(r(u,v)) |\text{det}(J_r(u,v))| du dv$.

  • Polar Coordinates Conversion: $dx dy = r dr d\theta$.

  • Spherical Coordinates Conversion: $dx dy dz = r^2 \sin(\theta) dr d\theta d\phi$.

VECTOR FIELDS (CHAPTER 7)

• Line Integrals: $\int<em>C V \cdot ds = \int</em>a^b \langle V(r(u)), r'(u) \rangle du$.

• Gradient (Conservative) Fields: $V = \nabla f$.

  • Fundamental Theorem for Line Integrals: $\int_C \nabla f \cdot ds = f(r(b)) - f(r(a))$.

  • Path independence: The integral only depends on endpoints.

• Flux: $\iint<em>F V \cdot dS = \iint</em>\Gamma \langle V(r), n_F \rangle du dv$. Measures fluid flow through a surface.