Chapter 1: Introduction to Multivariable Functions and Differential Calculus

Multivariable Functions and Differential Calculus

Introduction

• Essential for modeling phenomena in physics, computer science, and economics.
• Goal: To understand how and why things work and how to apply them without falling into traps.

Foundations

• RN Space: The space in which these functions exist.
• Norms: Used to measure distances.
• Neighborhoods: Defined by open and closed sets.
• Continuity: The idea that a function does not have abrupt jumps.
• Differentiability: A more advanced form of derivative.
• Partial Derivatives: Derivatives taken with respect to one variable while holding others constant.
• Taylor Expansions: Used to zoom in on local behavior.
• Extrema: Finding the highest and lowest points of a function.

The Space RN

• Set of points or vectors with n real coordinates, e.g., $x = (x<em>1, …, x</em>n)$.
• Need a way to measure distances and sizes of vectors.

Norms

• A function that associates a non-negative number to a vector, representing its length.
• Notation: $\lVert x \rVert$
• Three rules:
  • Separation: Only the zero vector has a norm of zero.
    • $x = 0 \iff \lVert x \rVert = 0$
  • Homogeneity: Scaling a vector by a factor (\lambda) multiplies its norm by the absolute value of (\lambda).
    • $\lVert \lambda x \rVert = |\lambda| \lVert x \rVert$
  • Triangle Inequality: The direct path is always shorter.
    • $\lVert x + y \rVert \leq \lVert x \rVert + \lVert y \rVert$

Types of Norms on RN

• Norm 1 (Manhattan Distance): Sum of the absolute values of the coordinates.
  • $\lVert x \rVert<em>1 = \sum |x</em>i|$
• Norm 2 (Euclidean Norm): Standard distance formula.
  • $\lVert x \rVert<em>2 = \sqrt{\sum x</em>i^2}$
• Norm Infinity: Maximum of the absolute values of the coordinates.
  • $\lVert x \rVert<em>{\infty} = \max |x</em>i|$
• In finite dimensions, all norms are equivalent, meaning they define the same notion of proximity.

Open and Closed Balls

• Open Ball: All points within a certain distance (r) of a point (a).
  • B(a, r) = {x : \lVert x - a \rVert < r}
• Closed Ball: All points within a distance (r) of a point (a), including the boundary.
  • $\overline{B}(a, r) = {x : \lVert x - a \rVert \leq r}$
• The shape of the ball depends on the norm used.
  • In R2, the unit ball looks different for norm 1 (a diamond), norm 2 (a disk), and norm infinity (a square).

Open and Closed Sets

• Open Set: Around each point, there exists a small open ball that is entirely contained within the set (no boundary points included).
• Closed Set: A set whose complement is open; equivalently, it contains all its limit points.
  • Example: R is both open and closed.
  • Example: In R, the interval (0, 1) is open, while [0, 1] is closed.

Continuity

• A function without sudden jumps.
• A function (f) is continuous at a point (a) if, as (x) approaches (a), (f(x)) approaches (f(a)).
  • $\lim_{x \to a} f(x) = f(a)$

Functions Defined Piecewise

• Check if the limit of the function as (x) and (y) approach (0, 0) exists and equals (f(0, 0)).
• Technique: Convert to polar coordinates.
  • $x = r \cos(\theta)$
  • $y = r \sin(\theta)$
• If the result depends on (\theta), the limit does not exist.
• If the limit exists and is independent of (\theta) and equals (f(0, 0)), the function is continuous at (0, 0).

Uniform Convergence

• If a sequence of continuous functions (fn) converges uniformly to (f) (i.e., the maximum difference between (fn) and (f) tends to 0 over the entire domain), then the limit (f) is also continuous.
• Useful for series of functions and justifying the continuity of parameter integrals.
  • Example: If $F(t, s)$ is continuous in (s) and there exists a dominating function, then $f(s) = \int_{A}^{B} F(t, s) dt$ is continuous.

Differential Calculus

Partial Derivatives

• $\frac{\partial f}{\partial x<em>i}$ measures how (f) changes when only (xi) is varied, keeping other variables constant.
• Represents the slope in a single direction.
• Higher-order partial derivatives can be calculated by differentiating multiple times.

Class CK Functions

• A function is of class (C^k) on an open set if all its partial derivatives up to order (k) exist and are continuous on that set.
• Continuity of derivatives is crucial for regularity.

Differentiability

• A stronger condition than having partial derivatives.
• A function is differentiable at a point (a) if, when zoomed in at (a), its graph increasingly resembles a plane.
• Can approximate $f(a + h)$ by $f(a) + L(h)$, where $L(h)$ is a linear term in $h$ and the error term is negligible compared to $h$ as $h$ approaches 0.
  • $f(a + h) = f(a) + L(h) + o(h)$

Gradient

• If (f) is differentiable, its partial derivatives exist at (a).

• For a function $f : R^n \to R$, the linear part $L(h)$, called the differential $df_a$, is given by the dot product with the gradient.

  • $L(h) = \nabla f(a) \cdot h = \sum<em>{i=1}^{n} \frac{\partial f}{\partial x</em>i}(a) h_i$

• The gradient is the vector of partial derivatives and indicates the direction of the greatest slope.

  • $\nabla f(a) = \left(\frac{\partial f}{\partial x<em>1}(a), …, \frac{\partial f}{\partial x</em>n}(a)\right)$

• A function can have all its partial derivatives at a point without being differentiable or even continuous at that point.

Sufficient Condition for Differentiability

• If (f) is of class $C^1$, i.e., its first-order partial derivatives exist and are continuous in a neighborhood of the point, then (f) is differentiable.
  • $C^1 \implies \text{differentiable} \implies \text{partial derivatives exist and } f \text{ is continuous}$
  • The reverse implications are not generally true.

Jacobian Matrix

• For a function $f : R^n \to R^p$ with p > 1, the Jacobian matrix is used.
• If $f = (f<em>1, …, f</em>p)$, the Jacobian matrix $J<em>f(a)$ is a $p \times n$ matrix where the element in the i-th row and j-th column is the partial derivative of the i-th component function $f</em>i$ with respect to the j-th variable $x_j$. It's evaluated at a.
  • $J<em>f(a) = \begin{pmatrix} \frac{\partial f</em>1}{\partial x<em>1}(a) & \cdots & \frac{\partial f</em>1}{\partial x<em>n}(a) \ \vdots & \ddots & \vdots \ \frac{\partial f</em>p}{\partial x<em>1}(a) & \cdots & \frac{\partial f</em>p}{\partial x_n}(a) \end{pmatrix}$

Chain Rule

• If (f) is differentiable at (c) and (g) is differentiable at (d = f(c)$, then the composite function$h = g \circ f is differentiable at (c).
• The differential of the composite function is the composite of the differentials: dhc = dg{f(c)} \circ df_c.
• In terms of Jacobian matrices:
  • J{g \circ f}(c) = Jg(f(c)) J_f(c)
• If (f) and (g) are of class C^k$, then$g \circ f$is also of class$C^k.

Taylor Expansion

• A generalization of Taylor series to multiple variables.
• A Taylor expansion of order (k) of (f) around (a) is a polynomial P(h)$of total degree less than or equal to (k) in the components of$h = x - a$such that$f = P + o(\lVert h \rVert^k).
• If (f) is of class C^k, Taylor's formula can be used.

Order 2 Taylor Expansion

• f(a + h) \approx f(a) + dfa(h) + \frac{1}{2} d^2fa(h, h)
• Where d^2f_a(h, h) is the second-order term, a quadratic form involving second derivatives.

Hessian Matrix

• The Hessian matrix Hf(a)$is the matrix of second derivatives, where the (i, j)-th entry is$\frac{\partial^2 f}{\partial xi \partial x_j}(a).
• The second-order term can be written as: \frac{1}{2} h^T H_f(a) h.

Local Extrema

• Points where the gradient is zero are candidates for local extrema (critical points).
  • \nabla f = 0
• The tangent plane is horizontal at these points.
• To determine the nature of a critical point (a), examine the second-order Taylor expansion:
  • f(a + h) = f(a) + \frac{1}{2} h^T H_f(a) h + o(\lVert h \rVert^2)
• The local behavior is determined by the sign of the quadratic term Q(h) = \frac{1}{2} h^T H_f(a) h.

Nature of Critical Points Based on Quadratic Form

• If Q(h) > 0$for all$h \neq 0 (positive definite), it is a strict local minimum (bottom of a bowl).
• If Q(h) < 0$for all$h \neq 0 (negative definite), it is a strict local maximum (top of a hill).
• If Q(h)$changes sign depending on$h, it is a saddle point (neither a max nor a min).

Lagrange Multipliers

• Used to find extrema on a subset, i.e., extrema under constraints.
• To optimize f(x)$subject to the constraint$g(x) = 0$, introduce a new variable (\lambda) (the Lagrange multiplier) and form the Lagrangian:$L(x, \lambda) = f(x) + \lambda g(x).
• The candidate points x$for constrained extrema are those for which there exists a (\lambda) such that the gradient of$L$with respect to all variables (the$x_i$and (\lambda$) is zero.
• Solve the system:
  • $\nabla<em>x f = \lambda \nabla</em>x g$
  • $g(x) = 0$

Laplacian

• The Laplacian $\Delta f$ is given by: $\Delta f = \sum<em>{i=1}^{n} \frac{\partial^2 f}{\partial x</em>i^2}$, the sum of the pure second partial derivatives.

Exercise Examples

Domain of Definition

• For $f_1(x, y) = \sqrt{1 - x^2 - y^2}$, the domain is $x^2 + y^2 \leq 1$, the closed unit disk.
• For $f_3(x, y) = \arctan(\frac{y}{x})$, the domain is $R^2$ excluding the y-axis (where $x = 0$).

Level Curves

• For $f_1(x, y) = x + y = c$, the level curves are lines $y = -x + c$, a family of parallel lines with slope -1.
• For $f_2(x, y) = \sqrt{x^2 + y^2} = c$, the level curves are circles centered at the origin with radius $c$ when $c \geq 0$.

Limits

• For $f(x, y) = \frac{x^2 y^2}{x^2 + xy + y^2}$, to find the limit at (0, 0), test different paths. Using $y = mx$, the limit depends on $m$, so the limit does not exist.

Differentiability Test

• For $f(x, y) = \frac{x^3 y^3}{\sqrt{x^2 + y^2}}$ if $(x, y) \neq (0, 0)$ and $f(0, 0) = 0$, follow these steps:
  • Check continuity at (0, 0) using polar coordinates.
  • Calculate partial derivatives at (0, 0) using the definition.
  • Test differentiability by checking if the function can be approximated by a linear term plus a little-o term.

Jacobian and Composition

• Given $f(x, y) = x + y$ and $g(t) = (t^2, t^3)$, find the Jacobian of $h = f \circ g$.
• First calculate $h(t) = f(g(t)) = t^2 + t^3$ and compute its derivative.
• Verify using the chain rule.

Taylor Expansion

• To find the Taylor expansion of order 2 for $f(x, y) = e^{2x + 3y}$
  • Substitute $u = 2x + 3y$ into the Taylor expansion of $e^u$ and keep terms up to order 2.

Recurring Themes in Exams

• Testing continuity, existence of partial derivatives, and differentiability for functions defined piecewise at (0, 0).
• Radial Laplacian (calculating $\Delta f$ for $f(x, y) = f(r)$, where $r = \sqrt{x^2 + y^2}$).
• Free or constrained extrema (using Lagrange multipliers).
• Relating Taylor expansions to extrema.

General Strategies

• Think in polar coordinates for limits, continuity, and differentiability.
• Be cautious about differentiability; existence of partial derivatives does not imply differentiability.
• Use the chain rule for composition (Jacobian of $g \circ f$).
• Use Taylor expansions to approximate function behavior.
• Always check the hypotheses of theorems (open domains, class $C^k$).