Numerical Basics and Data Fundamentals

Flashcards covering numerical basics, floating point representation, array layout, and tensor operations.

Floating Point Numbers

A number in [1.0, 2.0) with a scaling/stretching factor (exponent, varies in steps of powers of 2).

Sign (Floating Point)

Single bit indicating positive or negative number.

Exponent (Floating Point)

Signed integer indicating how much to shift the mantissa.

Mantissa (Floating Point)

Unsigned integer representing the fractional part of the number, following the 1.

IEEE 754

Dominant standard for floating point numbers, specifying representation and operations.

float32

32 bits, or 4 bytes per number.

float64

64 bits, or 8 bytes per number.

Binary Representation of Floats

Fixed-length sequence of bits representing a float in memory, split into sign, exponent, and mantissa.

Zero (+0.0 and -0.0)

IEEE 754 has both positive and negative representations.

Infinity (+∞ and -∞)

All ones for the exponent, all zeros for the mantissa. Sign bit indicates positive or negative.

NaN (Not A Number)

Used to represent values that are invalid; result of 0.0 / 0.0.

Roundoff Error

Floating point operations introduce this, because operations quantize computation results. Precision is variable according to magnitude.

ulps (units in the last place)

Measure of the difference between successive floating point numbers.

np.nextafter

Next representable number in the direction of the second argument.

Relative Error

Absolute difference between a floating point number and its real counterpart, normalized by the magnitude of the real number.

ndarrays

An efficient data structure in memory and computational terms, having fixed size, rectangular shape, uniform type, and multidimensional elements.

Strides

Memory offset constants specifying how to index into the array, one per axis.

Dope Vector

Information about striding.

C Ordering (Row-Major)

The last index changes first; default in C-based languages.

Fortran Ordering (Column-Major)

First index changes fastest; sometimes used in older software.

Reshape

Rearranges array axes by writing a new set of strides. Requirements: element count unchanged and order preserved.

Rank-Preserving Operations

Number of dimensions is preserved (e.g. maps, slices, transpose).

Rank-Reducing Operations

Number of dimensions is reduced (e.g. indexing, reductions, ravel()).

Rank-Promoting Operations

Add new dimensions to an array (e.g. broadcasting, np.tile).

np.squeeze()

Removes all singleton dimensions in one go.

Elided Axes

Elides repeated : (“all” slices) in an indexing expression, using three dots.

np.swapaxes(a, axis1, axis2)

Swaps any pair of axes; rewrites the strides.

Einstein Summation Notation

Letter names for dimensions with dimension rearrangement as a string.