Introduction to Numerical Methods and Error Analysis

Flashcards covering the fundamental concepts of numerical methods, types of errors, accuracy and precision, specific error measurements, and algorithmic stability.

Numerical Methods

Algorithms and techniques used to solve mathematical problems numerically using arithmetic operations ($+$, $-$, $\times$, $\div$) rather than symbolically using exact formulas.

Analytical Solution

An exact solution expressed as a closed-form formula, such as solving $x^2 - 5x + 6 = 0$ to find $x = 2, 3$.

Numerical Solution

An approximate solution obtained via iteration when no exact formula exists, such as finding $x \approx 0.739085$ for the equation $x = \cos(x)$.

Abel–Ruffini theorem

A theorem stating that a general polynomial of degree $\ge 5$ has no closed-form solution by radicals, making numerical root-finding the only general option.

Modeling Errors

Errors introduced when mathematical equations are used to represent a physical system, such as neglecting air resistance in a projectile model.

Blunders

Human errors that occur at any stage of the modeling process; they can be avoided by sound knowledge of fundamental principles and proper care.

Mistakes

Errors in the modeling process specifically due to programming errors.

Inherent Errors

Errors that preexist in the problem statement itself, such as uncertain measurements or empirical constants from handbooks.

Round-off Errors

Errors that are inevitable when using finite-precision floating-point arithmetic on computers or calculators, occurring through rounding or chopping.

Truncation Errors

Also known as mathematical approximation errors, these arise when an infinite process is approximated by a finite one, such as keeping only finitely many terms of a Taylor series.

Accuracy

How closely a computed or measured value agrees with the true value.

Precision

How closely individual computed or measured values agree with each other.

Inaccuracy

Also known as bias, this refers to a systematic deviation from the truth.

Imprecision

Also known as uncertainty, this refers to the magnitude of the scatter in measurements.

Absolute Error ($E_a$)

The absolute value of the difference between the exact value ($X_E$) and the approximate value ($X_A$), defined as $|X_E - X_A|$.

Relative Error ($E_r$)

The ratio of the absolute error to the exact value, defined as $\left| \frac{X_E - X_A}{X_E} \right|$ provided $X_E \ne 0$.

Percentage Relative Error ($E_p$)

The relative error expressed as a percentage, calculated as $100 \times E_r$.

Approximate Relative Error ($\varepsilon_a$)

Used in iterative methods where the true value is unknown, defined as $\left| \frac{\text{current approximation} - \text{previous approximation}}{\text{current approximation}} \right| \times 100\%$.

Stopping Criterion (Scarborough)

An iterative rule to stop when $\varepsilon_a < \varepsilon_s = (0.5 \times 10^{2-n})\%$, which guarantees the result is correct to at least $n$ significant figures.

Significant Digits

The digits in a numerical value that can be used with confidence and represent how many digits are meaningful and reliable.

Machine Epsilon ($\varepsilon_{mach}$)

The smallest number such that $1 + \varepsilon_{mach} \ne 1$; for double precision (64-bit), it is approximately $2.22 \times 10^{-16}$.

Chopping

The process of dropping extra digits in a number to fit a finite representation, which always creates a biased error in one direction.

Rounding

Adjusting the last kept digit of a number; the error is at most half a unit in the last place, and errors tend to cancel out.

Subtractive Cancellation

A phenomenon where subtracting two nearly equal numbers destroys significant digits, representing the most dangerous effect of round-off error.

Taylor Series

A series expansion used to analyze truncation error, formulated as $f(x_{i+1}) = f(x_i) + f'(x_i)h + \frac{f''(x_i)}{2!}h^2 + \dots + R_n$.

Error Propagation

The study of how uncertainties in input variables affect the uncertainty of a function’s output calculation.

Conditioning

A property of a mathematical problem describing how sensitive the exact answer is to small changes in the input data.

Stability

A property of a numerical algorithm describing whether the method amplifies small errors (round-off or truncation) made during the process.

Condition Number

The relative error magnification factor, calculated for a function $f(x)$ at point $a$ as $\frac{af'(a)}{f(a)}$.

Ill-conditioned Problem

A problem where the condition number is much greater than 1 ($\gg 1$), meaning input uncertainties are significantly magnified in the result.