Directional Limits: Multivariate functions can approach a point from infinitely many directions. A function is only continuous if the limit is identical regardless of the direction chosen.
Example 8.10: The function f(x,y)=x2+y2x2 with f(0,0)=0 is discontinuous at (0,0).
Limit along x-axis: limx→0f(x,0)=x2x2=1.
Limit along y-axis: limy→0f(0,y)=y20=0.
Limit along diagonal: limx→0f(x,x)=2x2x2=21.
Since limits differ by direction, the function is discontinuous at (0,0).
Partial and Directional Derivatives
Definition 8.11: Partial Derivative and Gradient:
Partial derivative involves differentiating with respect to one variable while holding others constant.
Let gk(t)=f(x1,…,xk−1,xk+t,xk+1,…,xn).
The k-th partial derivative: ∂xk∂f(x)=fxk(x)=gk′(0).
Gradient: If all partial derivatives exist, the vector containing them is the gradient: ∇f(x)=grad f(x)=∂x1∂f(x)…∂xn∂f(x).
Example 8.12: Critical Analysis of ∣x∣y:
f(x,y)=∣x∣y for x=0 and f(0,y)=0 for y≥0.
Partial derivative fx(x,y)=yx∣x∣y−2 for x=0 or y>1.
Partial derivative fy(x,y)=∣x∣yln(∣x∣) for x=0 or y>0.
At (0,0), the limit along the x-axis is 1 while along the y-axis it is 0, showing jump discontinuity.
Example 8.13: Gradients of Norm-based Functions:
g(x)=2∥x∥2⟹∇g(x)=x.
h(x)=∥x∥⟹∇h(x)=∥x∥x for x=0. Note that it is not differentiable at 0, similar to ∣x∣ in 1D.
Definition 8.14: Directional Derivative:
Calculates change along a unit direction vector h (∥h∥2=1).
Formula: Dhf(x)=∂h∂f(x)=limt→0tf(x+th)−f(x).
The k-th partial derivative is a directional derivative where h is the k-th unit vector ek.
Theorem 8.16: Calculating with Gradient: If partial derivatives are continuous, then the directional derivative is the dot product of the gradient and the direction vector:
fh(x)=(∇f(x))Th=⟨∇f(x),h⟩=∑k=1nhk∂xk∂f(x).
Multivariate Differentiation and Linear Approximation
Total Differentiability: Unlike single-variable calculus, division by a vector is undefined, so the derivative is defined using a linear approximation.
Definition 8.19: f:Rn→R is differentiable at x0 if there exists a unique linear function f′(x0):Rn→R such that:
f(x)−f(x0)=f′(x0)(x−x0)+r(x)
Where limx→x0∥x−x0∥r(x)=0.
Theorem 8.21: Properties and Connections:
If all partial derivatives exist and are continuous at x0, then f is differentiable.
If f is differentiable, then all partial derivatives exist, and f′(x0)(x−x0)=⟨∇f(x0),x−x0⟩.
Differentiability implies continuity.
Example 8.23: Counterexamples:
Case 1: Inverse of Theorem 8.21 — Function with a gradient that is NOT differentiable:
f(x,y)=x2+y2xy at (0,0). Gradient is 0, but linear approximation error r(x,y) does not decrease fast enough.
Case 2: Differentiable function with non-continuous partial derivatives:
g(x,y)=(x2+y2)sin(x2+y21) at (0,0).
Theorem 8.24: Operational Rules:
Sum: (f+g)′=f′+g′
Product: (f⋅g)′(x0)=f(x0)g′(x0)+g(x0)f′(x0)
Quotient: (gf)′=g2gf′−fg′
Theorem 8.26: Multivariate Chain Rule: If f:R→R and g:Rn→R, then:
(f∘g)′(x0)=f′(g(x0))⋅g′(x0)
∇(f∘g)(x0)=f′(g(x0))∇g(x0)
Extra 8.27: Jacobi Matrix: For a map f:Rn→Rp, the derivative is an n×p matrix called the Jacobi matrix, where entry (j,k) is the k-th partial derivative of the j-th output.
Radial Functions and Mean Value Theorem
Definition 8.29: Radial Functions: Functions that depend only on the distance from the origin: f(x)=g(∥x∥) or f(x)=g(∥x∥2).
Gradient Formula: ∇f(x)=g′(∥x∥)∥x∥x or ∇f(x)=2g′(∥x∥2)x.
Theorem 8.30: Multivariate Mean Value Theorem: If f is differentiable on the segment from a to b, there exists ω∈(0,1) such that:
f(b)−f(a)=(∇f(a+ω(b−a)))T(b−a).
Lipschitz Continuity (Example 8.31): A function is Lipschitz continuous (∣f(x)−f(y)∣≤C∥x−y∥) if its gradient is bounded.
f(x)=∥x∥ is Lipschitz continuous because ∥∇f∥=1.
g(x)=arcsin(∥x∥) is not Lipschitz continuous because the gradient is unbounded as ∥x∥→1.
Higher Order Derivatives and Schwarz's Theorem
Definition 8.32: Higher Order Partial Derivatives:
∂xi∂xj∂2f(x)=fxixj(x)=∂xj∂(∂xi∂f).
Multi-derivatives are read from left to right.
f∈Ck(Rn) means the function is k-times continuously partially differentiable.
Theorem 8.34: Schwarz's Theorem: If f∈Ck(Rn), the order of partial derivatives does not matter:
∂xi∂xj∂2f=∂xj∂xi∂2f.
Counterexample 8.35: Failure of Schwarz's Theorem: For f(x,y)=x2+y2xy(x2−y2) at (0,0), fxy=−1 and fyx=1. The second order partial derivatives are not continuous at the origin.
Definition 8.36: Multi-index Notation:
A multi-index is a vector k=(k1,…,kn) where kj∈N0.
Order: ∣k∣=∑j=1nkj.
Factorial: k!=k1!⋅k2!⋅⋯⋅kn!.
Derivative: ∂kf=∂x1k1…∂xnkn∂∣k∣f.
Multivariate Taylor Polynomials and Hessian Matrices
Theorem 8.37: Multivariate Taylor Polynomial: For f∈Cm(Rn), there exists y0 on the line between x0 and x such that:
Every multivariate quadratic polynomial can be written as 21xTHx+bTx+c.
Optimization: Local Minima and Maxima
Definition 8.42: Local Extrema:
Local Maximum: f(x)≤f(x0) for ∥x−x0∥<ϵ.
Local Minimum: f(x)≥f(x0) for ∥x−x0∥<ϵ.
Extremum is strict if the inequality is strict for x=x0.
Theorem 8.43: Necessary Condition: If x0 is a local extremum, then ∇f(x0)=0.
Points where the gradient is zero are candidates for extrema.
Gradient Descent Application: To find a minimum, move in direction −∇f: xk=xk−1−ω∇f(xk−1).
Matrix Definiteness (Definition 8.44):
Positive Definite: xTHx>0 for all x=0.
Positive Semidefinite: xTHx≥0.
Negative Definite: xTHx<0 for all x=0.
Indefinite: Matrix satisfies none of the above.
Test 8.46: Eigenvalues: A symmetric matrix is:
Positive definite if all eigenvalues ωk>0.
Negative definite if all eigenvalues ωk<0.
Indefinite if it has both positive and negative eigenvalues.
Test 8.48: Sylvester's Criterion: Let Hk be the upper left k×k sub-matrix.
Positive definite if det(Hk)>0 for all k.
Negative definite if det(Hk)<0 for odd k and det(Hk)>0 for even k.
Theorem 8.50: Sufficient Conditions for Extrema:
If Hf(x0) is positive definite, x0 is a strict local minimum.
If Hf(x0) is negative definite, x0 is a strict local maximum.
If Hf(x0) is indefinite, x0 is an inflection point (saddle point).
If semidefinite, the case is inconclusive.
Convex and Concave Multivariate Functions
Definition 8.53: Convexity: f(wx1+(1−w)x2)≤wf(x1)+(1−w)f(x2) for all w∈(0,1).
Theorem 8.54: Hessian and Convexity:
f is convex ⟺Hf is positive semidefinite everywhere.
f is strictly convex if Hf is positive definite everywhere.
f is concave ⟺−f is convex.
Basics of Multivariate Integration
Definition 9.1: n-dimensional interval: An interval I⊂Rn is defined as I=[a1,b1]×[a2,b2]×⋯×[an,bn].
Area/Volume: ∣I∣=∏k=1n(bk−ak).
Multivariate Riemann Integral (Definition 9.2): Defined via Riemann sums over hyperrectangles. f is integrable if the sum RS(f, Xn,Tn) converges to a constant c as the partition size tends to zero.
Example 9.4: Riemann Sum of xy: For f(x,y)=xy on [0,1]×[0,1], the integral is 41.
Theorem 9.6: Fubini's Theorem: If f is Riemann integrable on [a,b]×[c,d], then the double integral can be calculated as iterated single integrals:
∬If(x,y)dxdy=∫cd(∫abf(x,y)dx)dy.
Counterexample 9.5: f(x,y)=(x2+y2)2x2−y2. Iterating in different orders yields −4π and 4π. This function is not Riemann integrable.
Null Sets (Definition 9.11): A set X⊂Rn has measure 0 if it can be covered by intervals whose total size is less than any arbitrary ϵ>0.
Examples: Point sets, lines in R2, planes in R3, and graphs of continuous functions.
Theorem 9.14: Lebesgue Integrability Condition: A bounded function is Riemann integrable if and only if it is continuous almost everywhere (i.e., its discontinuities form a null set).
Integration on Normal Domains
Domain Boundaries (Definition 9.16): Boundary points ∂X are points where any neighborhood contains points both inside and outside X. If ∂X is a null set, and f is continuous on X, f is integrable.
Simple Regions (Definition 9.19):
x-simple: X={(x,y)∣a≤x≤b,ρ1(x)≤y≤ρ2(x)}.
y-simple: X={(x,y)∣a≤y≤b,ρ1(y)≤x≤ρ2(y)}.
Theorem 9.20: Iterated Integral on Simple Regions:
For x-simple: ∬Xf(x,y)dA=∫ab(∫ρ1(x)ρ2(x)f(x,y)dy)dx.
Example 9.18: Ellipse Integration: Integrating x3y+12yx2+4 over x2+2y2≤1 result in 22π. It requires splitting the constraint into y=±21−x2.
Example 9.21: trigonometric bounds: For f(x,y)=x+1 on 0≤x≤π, cos(x)≤y≤sin(x)+2, the volume is π2+3π+4.