Calculus and Linear Algebra Notes

Preliminaries

Real Numbers and the Real Line

• Real numbers can be visualized as points on a horizontal line called the real line, denoted by $\mathbb{R}$.

• If $x$ is a point on the real line, we write $x \in \mathbb{R}$.

• For two points $x$ and $y$ in $\mathbb{R}$, x < y or y > x if $x$ lies to the left of $y$.

• The number 0 divides the real line into positive (right of 0) and negative (left of 0) numbers.

• Examples: 0 < 3.333… and -2.5 < 0.

• Real numbers are closed under algebraic operations: for any $a, b \in \mathbb{R}$, $a + b, a - b, ab \in \mathbb{R}$, and $\frac{a}{b} \in \mathbb{R}$ if $b \neq 0$.

• Special subsets of $\mathbb{R}$:

  • $\mathbb{N} = {1, 2, 3, …}$ (positive integers)

  • $\mathbb{Z} = {\dots, -3, -2, -1, 0, 1, 2, 3, \dots}$ (integers)

  • $\mathbb{Q} = {\frac{p}{q} \mid p, q \in \mathbb{Z}, q \neq 0}$ (rational numbers)

Axioms and Properties of Real Numbers

• Axiom 1 (Trichotomy): For every $a \in \mathbb{R}$, either a < 0 or $a = 0$ or a > 0.

• Axiom 2 (Translation): Given $a, b, c \in \mathbb{R}$, if a < b then a + c < b + c. Also, a - c < b - c.

• Axiom 3 (Magnification): Let $a, b, c \in \mathbb{R}$. If a < b, then

  • ac < bc if c > 0.

  • ac > bc if c < 0.

• Axiom 4 (Inversion of a positive number): If a > 0, then \frac{1}{a} > 0.

• Axiom 5 (Order reversal): If 0 < a < b, then \frac{1}{b} < \frac{1}{a}.

• Axiom 6 (Non-zero square): If $a \in \mathbb{R}$, then a^2 > 0.

• The notation a < b excludes $a = b$.

• The notation $a \leq b$ or $b \geq a$ means a < b or $a = b$.

Intervals

• An interval is a section of the real line.

• Example: all numbers strictly between 1 and 2 are denoted by $(1, 2)$.

• (1, 2) = {x \in \mathbb{R} \mid 1 < x < 2}, where 1 < x < 2 means 1 < x and x < 2 simultaneously.

• $x \in (1, 2)$ if and only if 1 < x < 2.

• 1 and 2 are endpoints or boundary points of $(1, 2)$.

• If we include both 1 and 2, we get $[1, 2] = {x \in \mathbb{R} \mid 1 \leq x \leq 2}$.

• $x \in [1, 2]$ if and only if $1 \leq x \leq 2$.

Types of Intervals

• (a, b) = {x ∈ R | a < x < b} (open interval)

• [a, b) = {x ∈ R | a ≤ x < b} (left-closed or half-closed interval)

• (a, b] = {x ∈ R | a < x ≤ b} (right-closed or half-closed interval)

• [a, b] = {x ∈ R | a ≤ x ≤ b} (closed interval)

Unbounded Intervals

• (a, +∞) = {x ∈ R | x > a}

• [a, +∞) = {x ∈ R | x ≥ a}

• (−∞, b) = {x ∈ R | x < b}

• (−∞, b] = {x ∈ R | x ≤ b}

• (−∞, +∞) = R *Remark:

  • If a=b, then [a,b] = {a}.

  • If a=b, then (a,b) = (a,b] = [a,b) = ∅.

  • If b1}$</p></li><li><p>$f(-x) = -f(x)$if$f$is odd, then the graph is symmetric with respect to the origin$(0, 0)$.</p></li></ul></li><li><p>Example:$f(x) = x^2 - 1$is even, and$g(x) = \frac{1}{6}x^3$is odd.</p></li></ul><h3 id="c2d3d8ce-5041-4083-a06e-e277519ec9dd" data-toc-id="c2d3d8ce-5041-4083-a06e-e277519ec9dd" collapsed="false" seolevelmigrated="true">Limit of a Function</h3><h4 id="45dcb13a-b52f-4262-b4b0-f0f8cb544655" data-toc-id="45dcb13a-b52f-4262-b4b0-f0f8cb544655" collapsed="false" seolevelmigrated="true">Formal Definition</h4><ul><li><p>Example: Consider the function$f(x) = 2x$and the number$c = 3$. We know that$\lim{x \to c} f(x) = \lim{x \to 3} 2x = 6$.</p></li><li><p>The formal definition involves an inverse problem on proximity.</p></li><li><p><strong>Definition 2.1:</strong>$\lim_{x \to c} f(x) = l$if for any error bound$\epsilon > 0$, a deviation bound$\delta > 0$can be found such that if$|x - c| < \delta$then$|f(x) - l| < \epsilon$.</p></li><li><p>Briefly:$(\forall \epsilon > 0)(\exists \delta > 0), |x - c| < \delta \implies |f(x) - l| < \epsilon$.</p></li></ul><h5 id="e552fca5-6cb9-4b6b-9faa-366e3f7d0f65" data-toc-id="e552fca5-6cb9-4b6b-9faa-366e3f7d0f65" collapsed="false" seolevelmigrated="true">Example 2.2</h5><ul><li><p>Show that$\lim_{x \to 3} 2x = 6$.</p><ul><li><p><strong>Step 0:</strong> Write the formal definition: For any$\epsilon > 0$, there is a$\delta > 0$such that$|2x - 6| < \epsilon$whenever$|x - 3| < \delta$.</p></li><li><p><strong>Step 1:</strong> Write$|2x - 6|$as a sub-multiple of$|x - 3|$, that is find$M$such that$|2x - 6| \leq M|x - 3|$:$|2x - 6| = |2(x - 3)| = |2||x - 3| = 2|x - 3|$</p></li><li><p><strong>Step 2:</strong>$\delta = \frac{\epsilon}{M} = \frac{\epsilon}{2}$. If$|x - 3| < \delta$, then$|2x - 6| = 2|x - 3| < 2\delta = 2 \cdot \frac{\epsilon}{2} = \epsilon$. Therefore$\lim_{x \to 3} 2x = 6$.</p></li></ul></li></ul><h5 id="eba72b23-3643-45f5-a24b-2fe3b7f2168f" data-toc-id="eba72b23-3643-45f5-a24b-2fe3b7f2168f" collapsed="false" seolevelmigrated="true">Example 2.3</h5><ul><li><p>Show that$\lim_{x \to 2} x^2 = 4$.</p><ul><li><p><strong>Step 0:</strong> For any$\epsilon > 0$, we shall find$\delta > 0$such that whenever$|x - 2| < \delta$, then$|x^2 - 4| < \epsilon$.</p></li><li><p><strong>Step 1:</strong>$|x^2 - 4| = |(x - 2)(x + 2)| = |x - 2||x + 2|$. Now since$|x + 2|$is not a constant, we also need to bound it. For that, we assume that$|x - 2|$is bounded by some number, say 1. Thus$-1 < x - 2 < 1 \implies 1 < x < 3 \implies 3 < x + 2 < 5$. And so$ -5 < 3 < x + 2 < 5$so that$|x + 2| < 5$. Hence$|x^2 - 4| = |x - 2||x + 2| < 5|x - 2|$</p></li><li><p><strong>Step 2:</strong> We can choose$\frac{\epsilon}{5}$for$\delta$but with the condition that$|x - 2| < 1$. Hence$\delta = \min{1, \frac{\epsilon}{5}}$. If$|x - 2| < \delta$, then$|x - 2| < 1$as well as$|x - 2| < \frac{\epsilon}{5}$and so$|x^2 - 4| = |x - 2||x + 2| < 5|x - 2| < 5 \cdot \frac{\epsilon}{5} = \epsilon$.</p></li></ul></li></ul><h4 id="ead89820-3a0a-41fb-89cf-81f55e921367" data-toc-id="ead89820-3a0a-41fb-89cf-81f55e921367" collapsed="false" seolevelmigrated="true">Algebra of Limits</h4><ul><li><p><strong>Definition 2.4:</strong> A function$f$is continuous at$c \in Df$if$\lim{x \to c} f(x) = f(c)$, i.e., for any error bound$\epsilon > 0$, there is a deviation bound$\delta$such that$|x - c| < \delta \implies |f(x) - f(c)| < \epsilon$.</p></li><li><p><strong>Remark 2.5:</strong> In order to discuss continuity of$f$at$c$,$f$must be defined at$c$, that is we must have$c \in Df$. If$c \notin Df$, then we only discuss limit of$f$at$c$and sometimes we write$\lim{x \to c \atop x \neq c} f(x)$to indicate that$x$approaches$c$while staying in$Df$.</p></li><li><p><strong>Definition 2.6:</strong> A function$f$is continuous if it is continuous at every point$c \in D_f$.</p></li><li><p><strong>Example 2.7:</strong> The function defined by$f(x) = \sin(x)$is continuous at$c = \pi$.$f$is continuous at$\pi$if$\lim_{x \to \pi} \sin(x) = \sin(\pi) = 0$. Given an error bound$\epsilon > 0$we must find$\delta > 0$such that$|\sin(x) - 0| < \epsilon$whenever$|x - \pi| < \delta$. Using the contraction property of sin (Proposition ??), we must find$M$such that$|\sin(x) - 0| \leq M|x - \pi|$. But we have$|\sin(x) - 0| \leq |x - \pi|$, so$M = 1$.$ \delta = \frac{\epsilon}{1} = \epsilon$. If$|x - \pi| < \delta = \epsilon$, then$|\sin(x) - 0| \leq |x - \pi| < \epsilon$.</p></li><li><p><strong>Theorem 2.8:</strong> Arithmetic functions and circular (or trigonometric) functions are continuous, that is they are continuous at each and every point of their respective domains.</p></li><li><p><strong>Proposition 2.9 (Algebra of limits):</strong> Suppose that$f$and$g$are two continuous functions that are continuous at$c \in Df \cap Dg$.</p><ul><li><p>$\lim{x \to c} (f(x) + g(x)) = \lim{x \to c} f(x) + \lim_{x \to c} g(x) = f(c) + g(c)$;</p></li><li><p>$\lim{x \to c} (f(x).g(x)) = (\lim{x \to c} f(x)).(\lim_{x \to c} g(x)) = f(c).g(c)$;</p></li><li><p>$\lim{x \to c} f(x)^m = (\lim{x \to c} f(x))^m = f(c)^m$, where$m \in \mathbb{R}$.</p></li><li><p>If$g(x) \neq 0$and$g(c) \neq 0$, then$\lim{x \to c} \frac{f(x)}{g(x)} = \frac{\lim{x \to c} f(x)}{\lim_{x \to c} g(x)} = \frac{f(c)}{g(c)}$.</p></li></ul></li></ul><h5 id="2a7aada7-d0d2-447c-92c2-ea7ec3b6580e" data-toc-id="2a7aada7-d0d2-447c-92c2-ea7ec3b6580e" collapsed="false" seolevelmigrated="true">Examples 2.10</h5><ul><li><p>$\lim{x \to 2} \sqrt{2x + 1} = \lim{x \to 2} (2x + 1)^{\frac{1}{2}} = (\lim_{x \to 2} (2x + 1))^{\frac{1}{2}} = \sqrt{5}$.</p></li><li><p>$\lim{x \to 0} \frac{x^7 + 2x^6 + x^5}{3x^7 + 3x + 1} = \frac{\lim{x \to 0} (x^7 + 2x^6 + x^5)}{\lim_{x \to 0} (3x^7 + 3x + 1)} = \frac{0 + 0 + 0}{0 + 0 + 1} = \frac{0}{1} = 0$</p></li><li><p>$\lim{x \to \pi} \frac{2 \cos{\left(x + \frac{\pi}{2} \right)}}{\sin(x)} = \frac{\lim{x \to \pi} 2 \cos{\left(x + \frac{\pi}{2} \right)}}{\lim_{x \to \pi} 2 \sin (x)} = (-1)(1) = -1$</p></li></ul><h4 id="4ff91560-2ecc-46b2-b838-44cc008be9dd" data-toc-id="4ff91560-2ecc-46b2-b838-44cc008be9dd" collapsed="false" seolevelmigrated="true">Rational Functions</h4><ul><li><p><strong>Theorem 2.11:</strong> Polynomials and rational functions, i.e. functions of the form$\frac{P(x)}{Q(x)}$where$P(x)$and$Q(x)$are polynomials, are continuous.</p></li><li><p>In certain case rational functions require some algebraic manipulations. It is worth keeping in mind that when approaches the point , the behaviour of the function at the point$c$is less important than its behaviour at every$x$approximating$c$.</p></li></ul><h5 id="1aa52c89-3511-4f13-a655-873eda468cb5" data-toc-id="1aa52c89-3511-4f13-a655-873eda468cb5" collapsed="false" seolevelmigrated="true">Example 2.12</h5><ul><li><p>Consider the limit$\lim_{s \to 0 \atop s \neq 0} \frac{(s + 1)^2 - (s - 1)^2}{s}$. Here we have$f(s) = \frac{(s + 1)^2 - (s - 1)^2}{s}$and$f(s)$is not defined at$c = 0$. It is then necessary to “manipulate”$f(s)$at points$s \neq 0$:$f(s) = \frac{(s + 1)^2 - (s - 1)^2}{s} = \frac{(s^2 + 2s + 1) - (s^2 - 2s + 1)}{s} = \frac{s^2 + 2s + 1 - s^2 + 2s - 1}{s} = \frac{4s}{s} (s \neq 0)$</p></li></ul><h4 id="bf244539-16ab-429c-831e-3def00b0e011" data-toc-id="bf244539-16ab-429c-831e-3def00b0e011" collapsed="false" seolevelmigrated="true">Composition and Limits</h4><ul><li><p><strong>Theorem 2.13:</strong><br>Suppose that we have a composition$f \circ g$where$f$is continuous. We have$\lim{x \to c} (f \circ g)(x) = \lim{x \to c} (f(g(x)) = f(\lim_{x \to c} g(x))$</p></li></ul><h5 id="a161b982-152c-4135-8f4e-8e6e6b59d964" data-toc-id="a161b982-152c-4135-8f4e-8e6e6b59d964" collapsed="false" seolevelmigrated="true">Example 2.14</h5><ul><li><p>$\lim{x \to \pi/2} \sin(\sqrt{x}) = \sin(\lim{x \to \pi/2} \sqrt{x}) = \sin(|\pi|) = \sin(\pi) = 0$. Note that$\pi/2$is positive so$\sqrt{x}$is positive.</p></li></ul><h3 id="59be6304-c51a-4c36-9afe-0ce4c6d1afc2" data-toc-id="59be6304-c51a-4c36-9afe-0ce4c6d1afc2" collapsed="false" seolevelmigrated="true">Discontinuity</h3><ul><li><p>A function$f$is continuous at$c \in Df$, if$\lim{x \to c} f(x) = f(c)$.</p></li><li><p>Note that if$c \notin D_f$, then$f$is not continuous at$c$because the concept of continuity at$c$requires that the function be defined at$c$.</p></li><li><p>In cases where$c \notin D_f$, we are in the presence of certain types of discontinuities that can be classified using the following test:</p><ul><li><p>Case 1:$\lim_{x \to c \atop x \neq c} f(x)$exists. it is possible to extend ”f” such that it is continuos, so it's a removable discontinuity at c.</p></li><li><p>Case 2:$\lim_{x \to c \atop x \neq c} f(x)$does not exist. This extension will not be possible and we have either a jump discontinuity or an essential discontinuity at c.</p></li></ul></li></ul><h5 id="198855f5-6cb9-4777-a452-bb7b53a1e7b4" data-toc-id="198855f5-6cb9-4777-a452-bb7b53a1e7b4" collapsed="false" seolevelmigrated="true">Examples 2.15</h5><ul><li><p>Consider the function$f(x) = |\frac{x - 2}{x - 2}|$. We have$Df = R \backslash {2}$so that$f$is discontinuous at 2. To compute$\lim{x \to 2 \atop x \neq 2} f(x)$requires that we consider$x < 2$and$x > 2$. When$x < 2$we have$|x - 2| = -(x - 2)$, and when$x > 2$we have$|x - 2| = x - 2$. Therefore the limits from the right and from the left are respectively$ \lim{x \to 2^+} f(x) = \lim{x \to 2^+} \frac{x - 2}{x - 2} = 1$and$\lim{x \to 2^-} f(x) = \lim{x \to 2^-} \frac{-(x - 2)}{x - 2} = -1$This shows that$\lim_{x \to 2 \atop x \neq c} f(x)$does not exist. We have a jump discontinuity.</p></li><li><p>Consider the function$f(x) = \frac{x^2 - 1}{x - 1}$.$Df = R \backslash {1}$and$f$is discontinuous at 1. As opposed to the previous example$\lim{x \to 1 \atop x \neq 1} f(x)$can be found:$\lim{x \to 1 \atop x \neq 1} f(x) = \lim{x \to 1 \atop x \neq 1} \frac{(x - 1)(x + 1)}{x - 1} = \lim_{x \to 1 \atop x \neq 1} (x + 1) = 2$In this case we have a removable discontinuity.</p></li><li><p>Consider the function$f(x) = \frac{1}{x}$with$Df = R \backslash {0}$. Then$\lim{x \to 0 \atop x \neq 0} f(x)$does not exist and we have an essential discontinuity:</p></li></ul><h4 id="92800706-5319-42c3-88ac-692928583037" data-toc-id="92800706-5319-42c3-88ac-692928583037" collapsed="false" seolevelmigrated="true">Squeeze Theorem</h4><ul><li><p><strong>Lemma 2.16. (Squeeze play)</strong><br>Suppose that$f(x) \leq g(x)$for all$x \in Df \cap Dg$. If both$\lim{x \to c} f(x)$and$\lim{x \to c} g(x)$exist, then$\lim{x \to c} f(x) \leq \lim{x \to c} g(x)$.</p></li><li><p><strong>Theorem 2.17. (Squeeze Theorem)</strong><br>Suppose that$h \leq f \leq g$and let$c \in R$. If$\lim{x \to c} h(x) = \lim{x \to c} g(x) = L$, then also$\lim_{x \to c} f(x) = L$</p></li></ul><h5 id="63395518-0f0d-4443-b755-de31ec571efb" data-toc-id="63395518-0f0d-4443-b755-de31ec571efb" collapsed="false" seolevelmigrated="true">Example 2.18</h5><ul><li><p>Consider the example$f(x) = x^2 \sin^2{\left(\frac{1}{x} \right)}$with$Df = R \backslash {0}$. The function has a removable discontinuity at 0. We shall show this by using the Squeeze Theorem. First note that for all$x \neq 0$. Since$\sin(a) \leq 1$, also$\sin^2(a) \leq 1$for all$a \in R$. Therefore$\sin^2{\left(\frac{1}{x} \right)} \leq 1$and by multiplying by$x^2 \geq 0$$$f(x) = x^2 \sin^2{\left(\frac{1}{x} \right)} \leq x^2$. We then have$0 \leq f(x) \leq x^2$for all$x \neq 0$. By Theorem 2.17,$\lim{x \to 0 \atop x \neq 0} f(x) = 0$.</p></li></ul><h4 id="b3cb67b2-b5c5-4398-96ec-81b4946aceb2" data-toc-id="b3cb67b2-b5c5-4398-96ec-81b4946aceb2" collapsed="false" seolevelmigrated="true">Continuous extension</h4><ul><li><p>If$f(x)$has a removable discontinuity at$c \notin Df$then it is possible to extend the domain of definition of$f$to include$c$. We then obtain a new continuous function$f^(x)$such that$f^(x) = f(x)$for all$x \in Df$and$f^*(c) = \lim_{x \to c \atop x \neq c} f(x)$.</p></li><li><p><strong>Proposition 2.19. (Extension by continuity)</strong><br>Suppose that$f$is continuous and has a removable discontinuity at$c \notin Df$. Then the function$f^* : Df \cup {c} \to R$defined by$f^(x) = \begin{cases} f(x) & \text{if } x \in Df \ \lim{x \to c \atop x \neq c} f(x) & \text{if } x = c \end{cases}$<br>is continuous at every point in$D_f \cup {c}$. We say that$f^$is a continuous extension of$f$at c.</p></li></ul><h5 id="f61321ab-dff3-4605-8378-69b12b8d5d1c" data-toc-id="f61321ab-dff3-4605-8378-69b12b8d5d1c" collapsed="false" seolevelmigrated="true">Examples 2.20</h5><ul><li><p>$f(x) = \frac{\sin(x)}{x}$has a removable discontinuity at 0 with$\lim_{x \to 0 \atop x \neq 0} f(x) = 1$. We have the continuous extension$f^* : R \to R$where$f^*(x) = \begin{cases} \frac{\sin(x)}{x} & \text{if } x \neq 0 \ 1 & \text{if } x = 0 \end{cases}$</p></li><li><p>$f(x) = \frac{\cos(x) - 1}{x}$has a removable discontinuity at 0 with$\lim_{x \to 0 \atop x \neq 0} f(x) = 0$. We have the continuous extension$f^* : R \to R$where$f^*(x) = \begin{cases} \frac{\cos(x) - 1}{x} & \text{if } x \neq 0 \ 0 & \text{if } x = 0 \end{cases}$</p></li><li><p>$f(x) = \frac{\tan(x)}{x}$has a removable discontinuity at 0 with$\lim_{x \to 0 \atop x \neq 0} f(x) = 1$. We have the continuous extension$f^* : R \to R$where$f^*(x) = \begin{cases} \frac{\tan(x)}{x} & \text{if } x \neq 0 \ 1 & \text{if } x = 0 \end{cases}$</p></li></ul><h4 id="b0381662-757f-45ee-8393-6850a27b1ec7" data-toc-id="b0381662-757f-45ee-8393-6850a27b1ec7" collapsed="false" seolevelmigrated="true">Intermediate Value Theorem</h4><ul><li><p>One of the most important applications of the Intermediate Value Theorem is the possibility of claiming that certain equations have at least one root or solution.</p></li><li><p><strong>Definition 2.24:</strong> A compact interval is an interval that is closed and contains its endpoints, i.e., an interval of the form$[a; b]$where$a, b \in R$and$a \leq b$.</p></li><li><p>Functions that are defined on compact intervals and are continuous have numerous nice properties. One of them is the fact that their graphs mimic the compact intervals on two points: they contain the endpoints, and the graph is an uncut line.</p></li><li><p><strong>Theorem 2.26:</strong>(Intermediate Value Theorem) Let$f : [a; b] \to R$be a continuous function and let$x1, x2 \in [a; b]$with$x1 < x2$. For any value$k$such that$f(x1) 0$or$$f(x2) < 0 < f(x1)$, then there is$c \in (x1; x_2)$such that$f(c) = 0$.</p></li><li><p>The expression$f(x1)f(x2) < 0$indicates that the function$f$changes sign from$x1$to$x2$.</p></li><li><p><strong>Example 2.29:</strong><br>Consider the equation$\frac{x^3}{10} - \frac{3x}{2} - 1 = 0$. We want to find a solution to this equation between -3 and 3. The function$f(x) = \frac{x^3}{10} - \frac{3x}{2} - 1$, being a polynomial, is continuous and hence continuous on$[-3; 3]$. Now we choose$x1 = -2$and$x2 = 2$and compute$f(2) = \frac{8}{10} - 3 - 1 < 0$and$f(-2) = \frac{-8}{10} + 3 - 1 > 0$. By Theorem 2.28, there is$c \in (-2; 2)$such that$f(c) = 0$, that is$\frac{c^3}{10} - \frac{3c}{2} - 1 = 0$$.