Comprehensive Notes on Matrix Algebra and Calculus: Module 01 - Matrices

Foundations and Scope of Linear Algebra

• Linear algebra is the branch of mathematics devoted to the study of vector spaces and linear mappings between them, providing analytical and algorithmic tools to formulate, analyze, and resolve complex mathematical problems across science, engineering, and data disciplines.

• Core algebraic constructs include:

  • Vectors: Ordered lists of numbers representing quantities defined by magnitude and spatial direction. For example, a three-dimensional column vector is expressed as:     $\mathbf{v} = \begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix}$

  • Matrices: Rectangular arrays of numerical elements or algebraic expressions organized along horizontal rows and vertical columns. A general $3 \times 3$ matrix is written as:     $A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}$

  • Linear Equations: Mathematical statements wherein every variable appears exclusively to the first power without products, roots, or non-linear functional compositions. A representative system of linear equations is:     $\begin{cases} 2x + y - z = 3 \\ x - 2y + 3z = 1 \end{cases}$

  • Linear Combinations: The synthesis of a new vector through the scalar weighting and sequential addition of a set of existing vectors:     $\mathbf{u} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \dots + c_k\mathbf{v}_k$     For example, with scalars $3$, $-2$, and $1$: $\mathbf{u} = 3\mathbf{v}_1 - 2\mathbf{v}_2 + \mathbf{v}_3$.

  • Matrix Operations: Algebraic processes including entry-wise matrix addition, scalar multiplication, matrix multiplication ($A \times B = C$), transposition ($A^T$), and matrix inversion ($A^{-1}$, when $A$ is non-singular).

  • Linear Transformations: Vector mappings $T: V \rightarrow W$ between vector spaces preserving the operations of vector addition and scalar multiplication:     $T(\mathbf{v} + \mathbf{w}) = T(\mathbf{v}) + T(\mathbf{w})$     $T(c\mathbf{v}) = cT(\mathbf{v})$

• Foundational significance of linear algebra:

  • Transforms intricate multidimensional physical and computational challenges into structured, computationally solvable matrix equations.

  • Provides a universal mathematical language across diverse branches of physical science, computational science, and quantitative economics.

  • Serves as the mathematical bedrock for advanced analytical methods, including spectral decomposition, singular value decomposition, convex optimization, and machine learning pipelines.

  • Gilbert Strang characterizes the discipline: "Linear algebra is concerned with questions of line and continuity, of equivalence and similarity, of stretching and compressions."

• The progressive linear algebra learning sequence moves through seven interconnected milestones:

  1. Start with Vectors.

  2. Organize vectors into structured Matrices.

  3. Solve systems of Linear Equations.

  4. Execute foundational Matrix Operations.

  5. Discover Eigenvalues & Eigenvectors.

  6. Apply geometric Linear Transformations.

  7. Solve complex Real-World Problems.

• Primary interdisciplinary applications:

  • Computer Graphics & Visualization: Renders, scales, shears, rotates, and projects geometric models in two- and three-dimensional spaces for animation, virtual reality, and computational geometry ($[T][\mathbf{v}] \rightarrow [T][\mathbf{v}_{\text{sky}}]$).

  • Machine Learning & Artificial Intelligence: Underpins high-dimensional feature representation, weight optimization in deep neural networks, and covariance matrix manipulations ($[X]^T[X]$, $[W][\mathbf{x}]$).

  • Electrical Engineering: Determines node voltages and branch loop currents in complex networks via matrix formulations of Ohm's Law and Kirchhoff's Laws ($[A][I] = [V]$).

  • Structural Engineering: Analyzes equilibrium states, deflections, and load distributions across civil infrastructures, bridges, and airframes using stiffness matrices ($[K]\{\mathbf{d}\} = \{\mathbf{F}\}$, governed locally by Hooke's Law $F = k\Delta x$).

  • Signal Processing: Formulates transforms, frequency filters, and data compression schemes across audio, imagery, and communications via Discrete Fourier Transforms ($[W] \rightarrow \text{Filter}$).

  • Robotics & Automated Control: Models kinematic chains, joint coordinate frames, and multi-link robotic trajectories via sequential transformation matrices ($[T_e] = [A_1][A_2]\dots[A_n]$).

  • Physics & Quantum Mechanics: Represents physical observables through Hermitian matrices, where quantum states are state vectors and physical measurement outcomes correspond to eigenvalues.

  • Economics & Quantitative Data Science: Implements multivariate regression, economic input-output forecasting, and linear programming optimizations.

Historical Evolution of Matrix Theory

• Matrix concepts developed over centuries from spatial problem-solving into a formal branch of modern mathematics.

• Contributions from ancient India:

  • Vedic Roots and Geometry ($800\text{ BC} - 500\text{ BC}$): The Śulba Sūtras (authored by Baudhāyana, Āpastamba, and others) introduced grid-based spatial arrays and geometric algebra to plan, construct, and align complex sacrificial altars according to exact geometrical constraints.

  • The Bakhshali Manuscript ($3^{\text{rd}} - 4^{\text{th}}\text{ Century CE}$): Documented early multi-variable linear equation representations using systematic rectangular tabular layouts termed Sthāpanā or Nyāsa, employing dot notation (śūnya-bindu) to mark unknowns within structured arrays.

  • Classical Innovations & Tabular Methods ($5^{\text{th}} - 9^{\text{th}}\text{ Century CE}$):

  • Āryabhaṭa I ($499\text{ CE}$) formulated foundational algorithmic rules for linear indeterminate equations.

  • Mahāvīra ($850\text{ CE}$) developed elimination algorithms for $n$-variable systems using array reduction principles analogous to Gaussian elimination.

  • Systematic Algebra & Indeterminate Systems ($12^{\text{th}} - 14^{\text{th}}\text{ Century CE}$):

  • Bhāskara II ($1150\text{ CE}$) advanced coefficient elimination methods by aligning unknown variables within column matrices.

  • Nārāyaṇa Paṇḍita ($1356\text{ CE}$) investigated combinatorial matrix layouts, higher-order coefficient reductions, and the construction of magic squares.

• Broader historical timeline:

  • Ancient Chinese Tabular Methods ($300\text{ BC} - 200\text{ CE}$): The historical treatise Jiuzhang Suanshu (The Nine Chapters on the Mathematical Art) utilized bamboo rod arrays on counting boards to solve systems of linear equations via column operations equivalent to matrix transformations.

  • Gottfried Wilhelm Leibniz ($1646 - 1716$): Introduced systematic two-dimensional indexing schemes in the late $17^{\text{th}}$ century to record coefficients of linear systems, pioneering determinant theory.

  • Carl Friedrich Gauss ($1777 - 1855$): Established algorithmic row reduction (Gaussian elimination) while studying celestial mechanics and geodesic surveys, formalizing early concepts of matrix multiplication and matrix inversion.

  • James Joseph Sylvester ($1814 - 1897$): Coined the term "matrix" in $1850$ (derived from the Latin word for "womb"), defining it as an oblong arrangement of terms out of which determinants can be born:

  • "A matrix is a womb in which we may generate determinants."

  • Arthur Cayley ($1821 - 1895$): Published the definitive Memoir on the Theory of Matrices ($1858$), establishing matrices as an independent abstract mathematical discipline by defining matrix addition, scalar multiplication, matrix multiplication, inverses, and the Cayley-Hamilton theorem.

Elementary Transformations of a Matrix

• Elementary transformations are row or column operations that modify a matrix without altering its fundamental algebraic rank or linear solution space.

• Three elementary row transformations:

  1. Interchange of any two rows: Denoted $R_i \leftrightarrow R_j$.

  2. Multiplication of any row by a non-zero scalar: Denoted $R_i \rightarrow kR_i$ or $kR_i$ where $k \neq 0$.

  3. Addition of a scalar multiple of one row to another row: Denoted $R_j \rightarrow R_j + kR_i$.

• Corresponding column operations ($C_i \leftrightarrow C_j$, $C_i \rightarrow kC_i$, $C_j \rightarrow C_j + kC_i$) are termed elementary column transformations.

• Demonstration of elementary row operations on a general $3 \times 3$ matrix:   $A = \begin{bmatrix} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ c_1 & c_2 & c_3 \end{bmatrix}$

  • Row Interchange ($R_1 \leftrightarrow R_2$):     $A \sim \begin{bmatrix} b_1 & b_2 & b_3 \\ a_1 & a_2 & a_3 \\ c_1 & c_2 & c_3 \end{bmatrix}$

  • Scalar Multiplication ($R_3 \rightarrow kR_3$):     $A \sim \begin{bmatrix} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ kc_1 & kc_2 & kc_3 \end{bmatrix}$

  • Row Addition of a Multiple ($R_2 \rightarrow R_2 + kR_1$):     $A \sim \begin{bmatrix} a_1 & a_2 & a_3 \\ ka_1 + b_1 & ka_2 + b_2 & ka_3 + b_3 \\ c_1 & c_2 & c_3 \end{bmatrix}$

• Invariance properties under elementary transformations:

  • Elementary row and column transformations do not alter the order (dimensions $m \times n$) of a matrix.

  • Elementary row and column transformations do not change the rank of a matrix.

Row Echelon Form and Rank of a Matrix

• Definition of Row Echelon Form:

  • A non-zero matrix $A$ is in row echelon form if it satisfies two conditions:

  1. All zero rows (rows whose entries are entirely zero) are positioned below every non-zero row.

  2. The leading entry (the first non-zero entry from the left) in each non-zero row is $1$, and all entries directly beneath this leading entry in the same column are zero.

  • Canonical examples:     $A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 1 \end{bmatrix}, \quad B = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

• Algorithmic rules for echelon reduction:

  • Prefer a leading non-zero entry (ideally $1$) at position $(1,1)$. If the leading entry is zero, perform an interchange ($R_1 \leftrightarrow R_i$) with a lower row having a non-zero leading entry.

  • Use the leading pivot element of a given row to systematically eliminate all entries below it within the same column via elementary row operations of the type $R_j \rightarrow R_j - cR_i$.

  • Row transformations must be applied to every entry across the entire row.

  • Fractions should be avoided during hand computation wherever possible to prevent arithmetic errors.

• Definition of Rank of a Matrix:

  • The rank of a matrix $A$, denoted by $\rho(A)$, is the total number of non-zero rows present in its row echelon form.

• Fundamental properties of rank:

  • Rank Equivalence: For any matrix $A$, $\text{Rank}(A) = \text{Row Rank} = \text{Column Rank}$, where row rank equals the maximum number of linearly independent row vectors, and column rank equals the maximum number of linearly independent column vectors.

  • Dimension Bounds: For any real matrix $A \in \mathbb{R}^{m \times n}$:     $0 \le \rho(A) \le \min(m, n)$

  • Null Matrix: $\rho(A) = 0$ if and only if $A$ is the zero matrix ($A = 0$).

  • Transpose Invariance: The rank of a matrix equals the rank of its transpose:     $\rho(A) = \rho(A^T)$

  • Linear Independence: If $\rho(A) = r$ for an $m \times n$ matrix, exactly $r$ rows (and $r$ columns) of $A$ are linearly independent.

• Engineering and computational utility of rank:

  • Linear Systems Analysis: Determines whether a system of linear equations $A\mathbf{x} = \mathbf{b}$ admits a unique solution, infinitely many solutions, or no solution.

  • Linear Dependence Detection: Identifies whether data channels, database attributes, or structural variables are redundant combinations of other features (e.g., in database normalization).

  • Dimensionality Reduction: Serves as the operating foundation for Principal Component Analysis (PCA) to extract truly informative dimensions from high-dimensional datasets.

  • Low-Rank Approximations via SVD: Powers Singular Value Decomposition (SVD), utilized in image compression, streaming recommender engines (e.g., Netflix, Spotify), audio noise reduction, and signal feature extraction.

Worked Problems: Matrix Rank via Echelon Reduction

• Problem 1 (a): Determine the rank of $A = \begin{bmatrix} 1 & 2 \\ 3 & 6 \end{bmatrix}$.

  • Perform row operation $R_2 \rightarrow R_2 - 3R_1$:     $A \sim \begin{bmatrix} 1 & 2 \\ 3 - 3(1) & 6 - 3(2) \end{bmatrix} = \begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix}$

  • The row echelon form contains $1$ non-zero row.

  • Therefore, $\rho(A) = 1$.

• Problem 1 (b): Determine the rank of $A = \begin{bmatrix} 0 & 2 & 3 & 4 \\ 2 & 3 & 5 & 4 \\ 4 & 8 & 13 & 12 \end{bmatrix}$.

  • Step 1: Interchange row 1 and row 2 ($R_1 \leftrightarrow R_2$) to place a non-zero entry at the top-left pivot:     $A \sim \begin{bmatrix} 2 & 3 & 5 & 4 \\ 0 & 2 & 3 & 4 \\ 4 & 8 & 13 & 12 \end{bmatrix}$

  • Step 2: Eliminate the first entry of row 3 via $R_3 \rightarrow R_3 - 2R_1$:     $R_3 = \begin{bmatrix} 4 - 2(2) & 8 - 2(3) & 13 - 2(5) & 12 - 2(4) \end{bmatrix} = \begin{bmatrix} 0 & 2 & 3 & 4 \end{bmatrix}$     $A \sim \begin{bmatrix} 2 & 3 & 5 & 4 \\ 0 & 2 & 3 & 4 \\ 0 & 2 & 3 & 4 \end{bmatrix}$

  • Step 3: Eliminate row 3 using row 2 via $R_3 \rightarrow R_3 - R_2$:     $A \sim \begin{bmatrix} 2 & 3 & 5 & 4 \\ 0 & 2 & 3 & 4 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

  • Step 4: Normalize pivot elements to $1$ via $R_1 \rightarrow \frac{1}{2}R_1$ and $R_2 \rightarrow \frac{1}{2}R_2$:     $A \sim \begin{bmatrix} 1 & 3/2 & 5/2 & 2 \\ 0 & 1 & 3/2 & 2 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

  • The number of non-zero rows is $2$.

  • Therefore, $\rho(A) = 2$.

• Problem 2: Determine the rank of $A = \begin{bmatrix} 1 & 3 & -2 \\ 2 & -1 & 4 \\ 1 & -11 & 14 \end{bmatrix}$.

  • Step 1: Eliminate the first column entries in rows 2 and 3 via $R_2 \rightarrow R_2 - 2R_1$ and $R_3 \rightarrow R_3 - R_1$:     $R_2 = \begin{bmatrix} 2 - 2(1) & -1 - 2(3) & 4 - 2(-2) \end{bmatrix} = \begin{bmatrix} 0 & -7 & 8 \end{bmatrix}$     $R_3 = \begin{bmatrix} 1 - 1 & -11 - 3 & 14 - (-2) \end{bmatrix} = \begin{bmatrix} 0 & -14 & 16 \end{bmatrix}$     $A \sim \begin{bmatrix} 1 & 3 & -2 \\ 0 & -7 & 8 \\ 0 & -14 & 16 \end{bmatrix}$

  • Step 2: Eliminate row 3 via $R_3 \rightarrow R_3 - 2R_2$:     $R_3 = \begin{bmatrix} 0 - 0 & -14 - 2(-7) & 16 - 2(8) \end{bmatrix} = \begin{bmatrix} 0 & 0 & 0 \end{bmatrix}$     $A \sim \begin{bmatrix} 1 & 3 & -2 \\ 0 & -7 & 8 \\ 0 & 0 & 0 \end{bmatrix}$

  • The matrix has $2$ non-zero rows.

  • Therefore, $\rho(A) = 2$.

• Problem 3: Determine the rank of $A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 2 & 3 & 4 & 5 \\ 3 & 4 & 5 & 6 \\ 4 & 5 & 6 & 7 \end{bmatrix}$.

  • Step 1: Clear the first column beneath the leading pivot via $R_2 \rightarrow R_2 - 2R_1$, $R_3 \rightarrow R_3 - 3R_1$, and $R_4 \rightarrow R_4 - 4R_1$:     $R_2 = \begin{bmatrix} 0 & -1 & -2 & -3 \end{bmatrix}$     $R_3 = \begin{bmatrix} 0 & -2 & -4 & -6 \end{bmatrix}$     $R_4 = \begin{bmatrix} 0 & -3 & -6 & -9 \end{bmatrix}$     $A \sim \begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & -1 & -2 & -3 \\ 0 & -2 & -4 & -6 \\ 0 & -3 & -6 & -9 \end{bmatrix}$

  • Step 2: Multiply $R_2$ by $-1$ ($R_2 \rightarrow -R_2$), then perform $R_3 \rightarrow R_3 + 2R_2$ and $R_4 \rightarrow R_4 + 3R_2$:     $A \sim \begin{bmatrix} 1 & 2 & 3 & 4 \\ 0 & 1 & 2 & 3 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

  • The reduced matrix displays exactly $2$ non-zero rows.

  • Therefore, $\rho(A) = 2$.

• Problem 4: Determine the rank of $A = \begin{bmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 4 & 5 & 1 & 2 \\ 3 & 4 & 2 & 5 \end{bmatrix}$.

  • Step 1: Interchange row 1 and row 4 ($R_1 \leftrightarrow R_4$):     $A \sim \begin{bmatrix} 3 & 4 & 2 & 5 \\ 0 & 0 & 0 & 0 \\ 4 & 5 & 1 & 2 \\ 0 & 1 & 0 & 0 \end{bmatrix}$

  • Step 2: Eliminate the leading element in row 3 via $R_3 \rightarrow 3R_3 - 4R_1$:     $3R_3 - 4R_1 = 3\begin{bmatrix} 4 & 5 & 1 & 2 \end{bmatrix} - 4\begin{bmatrix} 3 & 4 & 2 & 5 \end{bmatrix} = \begin{bmatrix} 0 & -1 & -5 & -14 \end{bmatrix}$     $A \sim \begin{bmatrix} 3 & 4 & 2 & 5 \\ 0 & 0 & 0 & 0 \\ 0 & -1 & -5 & -14 \\ 0 & 1 & 0 & 0 \end{bmatrix}$

  • Step 3: Interchange row 2 with row 4 ($R_2 \leftrightarrow R_4$) to bring non-zero rows up:     $A \sim \begin{bmatrix} 3 & 4 & 2 & 5 \\ 0 & 1 & 0 & 0 \\ 0 & -1 & -5 & -14 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

  • Step 4: Eliminate the second element of row 3 via $R_3 \rightarrow R_3 + R_2$:     $R_3 = \begin{bmatrix} 0 & 0 & -5 & -14 \end{bmatrix}$     $A \sim \begin{bmatrix} 3 & 4 & 2 & 5 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & -5 & -14 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

  • The number of non-zero rows is $3$.

  • Therefore, $\rho(A) = 3$.

• Problem 5: Determine the rank of $A = \begin{bmatrix} -2 & -1 & -3 & -1 \\ 1 & 2 & 3 & 1 \\ 1 & 0 & 1 & 1 \\ 0 & 1 & 1 & 1 \end{bmatrix}$.

  • Step 1: Interchange row 1 and row 2 ($R_1 \leftrightarrow R_2$):     $A \sim \begin{bmatrix} 1 & 2 & 3 & 1 \\ -2 & -1 & -3 & -1 \\ 1 & 0 & 1 & 1 \\ 0 & 1 & 1 & 1 \end{bmatrix}$

  • Step 2: Eliminate entries in column 1 via $R_2 \rightarrow R_2 + 2R_1$ and $R_3 \rightarrow R_3 - R_1$:     $R_2 = \begin{bmatrix} 0 & 3 & 3 & 1 \end{bmatrix}$     $R_3 = \begin{bmatrix} 0 & -2 & -2 & 0 \end{bmatrix}$     $A \sim \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 3 & 3 & 1 \\ 0 & -2 & -2 & 0 \\ 0 & 1 & 1 & 1 \end{bmatrix}$

  • Step 3: Interchange row 2 and row 4 ($R_2 \leftrightarrow R_4$) for simpler pivot arithmetic:     $A \sim \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 1 & 1 & 1 \\ 0 & -2 & -2 & 0 \\ 0 & 3 & 3 & 1 \end{bmatrix}$

  • Step 4: Eliminate column 2 entries via $R_3 \rightarrow R_3 + 2R_2$ and $R_4 \rightarrow R_4 - 3R_2$:     $R_3 = \begin{bmatrix} 0 & 0 & 0 & 2 \end{bmatrix}$     $R_4 = \begin{bmatrix} 0 & 0 & 0 & -2 \end{bmatrix}$     $A \sim \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 0 & 2 \\ 0 & 0 & 0 & -2 \end{bmatrix}$

  • Step 5: Eliminate row 4 via $R_4 \rightarrow R_4 + R_3$ and normalize row 3 via $R_3 \rightarrow \frac{1}{2}R_3$:     $A \sim \begin{bmatrix} 1 & 2 & 3 & 1 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

  • The number of non-zero rows is $3$.

  • Therefore, $\rho(A) = 3$.

Applied Case Studies in Matrix Rank

• Case Study 1: Packet Loss Analysis in a Data Center:

  • Problem context: A network administrator tracks average packet loss percentages between four critical servers (Server A, B, C, D) and four routers (Router 1, 2, 3, 4) in a data center over one month. The packet loss data matrix $P$ is:     $P = \begin{bmatrix} 5 & 10 & 15 & 20 \\ 10 & 20 & 30 & 40 \\ 5 & 12 & 18 & 25 \\ 0 & 5 & 7 & 10 \end{bmatrix}$

  • Mathematical reduction of matrix $P$:

  • Apply $R_2 \rightarrow R_2 - 2R_1$:       $P \sim \begin{bmatrix} 5 & 10 & 15 & 20 \\ 0 & 0 & 0 & 0 \\ 5 & 12 & 18 & 25 \\ 0 & 5 & 7 & 10 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 - R_1$:       $P \sim \begin{bmatrix} 5 & 10 & 15 & 20 \\ 0 & 0 & 0 & 0 \\ 0 & 2 & 3 & 5 \\ 0 & 5 & 7 & 10 \end{bmatrix}$

  • Eliminate row 4 via $R_4 \rightarrow 2R_4 - 5R_3$:       $2R_4 - 5R_3 = 2\begin{bmatrix} 0 & 5 & 7 & 10 \end{bmatrix} - 5\begin{bmatrix} 0 & 2 & 3 & 5 \end{bmatrix} = \begin{bmatrix} 0 & 0 & -1 & -5 \end{bmatrix}$       $P \sim \begin{bmatrix} 5 & 10 & 15 & 20 \\ 0 & 0 & 0 & 0 \\ 0 & 2 & 3 & 5 \\ 0 & 0 & -1 & -5 \end{bmatrix}$

  • Rearranging non-zero rows yields $3$ non-zero rows, so $\rho(P) = 3$.

  • MATLAB Code: matlab % Packet Loss Matrix P = [5 10 15 20; 10 20 30 40; 5 12 18 25; 0 5 7 10]; % Find the rank r = rank(P); disp(['Rank of matrix P = ', num2str(r)]);     

• Operational Insights:

  • Independent Contribution: Because $\rho(P) = 3$, only $3$ out of $4$ routers contribute uniquely and independently to packet loss variation across the servers.

  • Redundancy/Correlation: Because $\rho(P) < 4$, at least one router's behavior is a linear combination of the other three. Packet loss patterns among routers contain redundancy.

  • Maintenance & Upgrade Prioritization: Maintenance, testing, and hardware upgrades should be prioritized on the $3$ independent, critical routers. The redundant router can serve as a secondary backup, optimizing operational budgets while preserving network reliability.

• Case Study 2: Detecting Redundant Features in Smart City Traffic Monitoring:

  • Problem context: Three sensors monitor a busy municipal intersection:

  • Sensor A counts cars per minute.

  • Sensor B counts buses per minute.

  • Sensor C counts total vehicles per minute.

  • Data matrix $A$ collected over $3$ minutes:     $A = \begin{bmatrix} 2 & 1 & 3 \\ 4 & 2 & 6 \\ 3 & 1 & 4 \end{bmatrix}$     where Column 1 = Cars (A), Column 2 = Buses (B), Column 3 = Total Vehicles (C).

  • Analytical reduction:

  • The underlying relationship across columns is:       $\text{Column 3} = \text{Column 1} + \text{Column 2}$       $(\text{Sensor C} = \text{Sensor A} + \text{Sensor B})$

  • Applying elementary row reduction transforms $A$ to:       $A \sim \begin{bmatrix} 1 & 1/2 & 3/2 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}$

  • The row echelon form yields $\rho(A) = 2$.

  • Architectural conclusions:

  • Independent Features: There are exactly $2$ independent features in the dataset.

  • Sensor Removal: Sensor C provides entirely redundant information because its values can be reconstructed at any instant via $\text{Sensor A} + \text{Sensor B}$. Sensor C can be removed.

  • Optimization Utility: Rank analysis identifies feature dependency ($\rho(A) = 2 < 3$ columns), enabling engineers to eliminate redundant sensing hardware, streamline data storage, and lower bandwidth costs without information loss.

Systems of Linear Equations and Consistency Analysis

• Formulation of an $m \times n$ linear system:   $\begin{cases} a_{11}x_1 + a_{12}x_2 + \dots + a_{1n}x_n = b_1 \\ a_{21}x_1 + a_{22}x_2 + \dots + a_{2n}x_n = b_2 \\ \quad \vdots \\ a_{m1}x_1 + a_{m2}x_2 + \dots + a_{mn}x_n = b_m \end{cases}$

• Matrix representation:   $AX = B$   where:   $A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix}, \quad X = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \quad B = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix}$

  • $A$ is the coefficient matrix ($m \times n$).

  • $X$ is the column vector of unknowns ($n \times 1$).

  • $B$ is the column vector of constants ($m \times 1$).

  • The augmented matrix is denoted by $[A : B]$:     $[A : B] = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} & : & b_1 \\ a_{21} & a_{22} & \dots & a_{2n} & : & b_2 \\ \vdots & \vdots & \ddots & \vdots & : & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} & : & b_m \end{bmatrix}$

• Homogeneous vs. Non-Homogeneous Systems:

  • If $b_1 = b_2 = \dots = b_m = 0$ ($B = \mathbf{0}$), the system is homogeneous ($AX = \mathbf{0}$).

  • If at least one $b_i \neq 0$, the system is non-homogeneous ($AX = B$).

• For homogeneous systems, $x_1 = x_2 = \dots = x_n = 0$ is always a solution, termed the trivial solution.

• A solution containing at least one non-zero component ($x_i \neq 0$) is a non-trivial solution.

• Consistency criteria (Rouché–Capelli Theorem):

  • A linear system is consistent if and only if the rank of the coefficient matrix equals the rank of the augmented matrix:     $\rho(A) = \rho([A : B])$

  • Let $\rho(A) = \rho([A : B]) = r$, and let $n$ denote the total number of unknowns:

  1. Unique Solution: $r = n$. The system is consistent and possesses a unique solution vector.

  2. Infinitely Many Solutions: $r < n$. The system is consistent and possesses infinitely many solutions. Exactly $n - r$ variables can be assigned arbitrary parameter values.

  • Inconsistent (No Solution): If $\rho(A) \neq \rho([A : B])$, the system is inconsistent and admits no solution.

• Systematic procedure for solving linear systems:

  1. Assemble the augmented matrix $[A : B]$.

  2. Reduce $[A : B]$ into row echelon form via elementary row operations.

  3. Read off $\rho(A)$ and $\rho([A : B])$ to evaluate consistency.

  4. Re-express the reduced echelon matrix as a system of algebraic equations and solve for the unknowns using back substitution.

• Problem 6: Unique Solution Verification:

  • System: $2x + y + z = 5$, $x + y + z = 4$, $x - y + 2z = 1$.

  • Augmented matrix:     $[A : B] = \begin{bmatrix} 2 & 1 & 1 & : & 5 \\ 1 & 1 & 1 & : & 4 \\ 1 & -1 & 2 & : & 1 \end{bmatrix}$

  • Apply $R_1 \leftrightarrow R_2$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & 1 & : & 4 \\ 2 & 1 & 1 & : & 5 \\ 1 & -1 & 2 & : & 1 \end{bmatrix}$

  • Apply $R_2 \rightarrow R_2 - 2R_1$ and $R_3 \rightarrow R_3 - R_1$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & 1 & : & 4 \\ 0 & -1 & -1 & : & -3 \\ 0 & -2 & 1 & : & -3 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 - 2R_2$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & 1 & : & 4 \\ 0 & -1 & -1 & : & -3 \\ 0 & 0 & 3 & : & 3 \end{bmatrix}$

  • Evaluation: $\rho(A) = 3$, $\rho([A : B]) = 3$, and $n = 3$. Since $\rho(A) = \rho([A : B]) = 3 = n$, the system has a unique solution.

  • Back substitution:

  • From equation 3: $3z = 3 \implies z = 1$.

  • From equation 2: $-y - (1) = -3 \implies y = 2$.

  • From equation 1: $x + 2 + 1 = 4 \implies x = 1$.

  • Final solution: $x = 1$, $y = 2$, $z = 1$.

• Problem 7: Infinitely Many Solutions:

  • System: $4x - 2y + 6z = 8$, $x + y - 3z = -1$, $15x - 3y + 9z = 21$.

  • Augmented matrix:     $[A : B] = \begin{bmatrix} 4 & -2 & 6 & : & 8 \\ 1 & 1 & -3 & : & -1 \\ 15 & -3 & 9 & : & 21 \end{bmatrix}$

  • Apply $R_1 \leftrightarrow R_2$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & -3 & : & -1 \\ 4 & -2 & 6 & : & 8 \\ 15 & -3 & 9 & : & 21 \end{bmatrix}$

  • Apply $R_2 \rightarrow R_2 - 4R_1$ and $R_3 \rightarrow R_3 - 15R_1$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & -3 & : & -1 \\ 0 & -6 & 18 & : & 12 \\ 0 & -18 & 54 & : & 36 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 - 3R_2$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & -3 & : & -1 \\ 0 & -6 & 18 & : & 12 \\ 0 & 0 & 0 & : & 0 \end{bmatrix}$

  • Normalize row 2 via $R_2 \rightarrow -\frac{1}{6}R_2$:     $[A : B] \sim \begin{bmatrix} 1 & 1 & -3 & : & -1 \\ 0 & 1 & -3 & : & -2 \\ 0 & 0 & 0 & : & 0 \end{bmatrix}$

  • Evaluation: $\rho(A) = 2$, $\rho([A : B]) = 2$, and $n = 3$. Because $r = 2 < 3$, the system has infinitely many solutions with $n - r = 3 - 2 = 1$ arbitrary parameter.

  • Algebraic parameterization:

  • Set $z = k$ (arbitrary parameter).

  • From equation 2: $y - 3k = -2 \implies y = 3k - 2$.

  • From equation 1: $x + (3k - 2) - 3k = -1 \implies x - 2 = -1 \implies x = 1$.

  • Solution set: $x = 1$, $y = 3k - 2$, $z = k$.

• Problem 10: Inconsistent System (No Solution):

  • System: $5x + 3y + 7z = 5$, $3x + 26y + 2z = 9$, $7x + 2y + 10z = 5$.

  • Augmented matrix:     $[A : B] = \begin{bmatrix} 5 & 3 & 7 & : & 5 \\ 3 & 26 & 2 & : & 9 \\ 7 & 2 & 10 & : & 5 \end{bmatrix}$

  • Apply $R_2 \rightarrow 5R_2 - 3R_1$ and $R_3 \rightarrow 5R_3 - 7R_1$:     $5R_2 - 3R_1 = \begin{bmatrix} 0 & 130 - 9 & 10 - 21 & : & 45 - 15 \end{bmatrix} = \begin{bmatrix} 0 & 121 & -11 & : & 30 \end{bmatrix}$     $5R_3 - 7R_1 = \begin{bmatrix} 0 & 10 - 21 & 50 - 49 & : & 25 - 35 \end{bmatrix} = \begin{bmatrix} 0 & -11 & 1 & : & -10 \end{bmatrix}$     $[A : B] \sim \begin{bmatrix} 5 & 3 & 7 & : & 5 \\ 0 & 121 & -11 & : & 30 \\ 0 & -11 & 1 & : & -10 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_2 + 11R_3$:     $R_2 + 11R_3 = \begin{bmatrix} 0 & 121 - 121 & -11 + 11 & : & 30 + 11(-10) \end{bmatrix} = \begin{bmatrix} 0 & 0 & 0 & : & -80 \end{bmatrix}$     $[A : B] \sim \begin{bmatrix} 5 & 3 & 7 & : & 5 \\ 0 & 121 & -11 & : & 30 \\ 0 & 0 & 0 & : & -80 \end{bmatrix}$

  • Evaluation: $\rho(A) = 2$, but $\rho([A : B]) = 3$.

  • Because $\rho(A) \neq \rho([A : B])$, the system is inconsistent and admits no solution.

• Problem 11: Parametric Investigation with $\lambda$ and $\mu$:

  • System: $x + y + z = 6$, $x + 2y + 3z = 10$, $x + 2y + \lambda z = \mu$.

  • Augmented matrix reduction:     $[A : B] = \begin{bmatrix} 1 & 1 & 1 & : & 6 \\ 1 & 2 & 3 & : & 10 \\ 1 & 2 & \lambda & : & \mu \end{bmatrix}$

  • Apply $R_2 \rightarrow R_2 - R_1$ and $R_3 \rightarrow R_3 - R_1$:       $[A : B] \sim \begin{bmatrix} 1 & 1 & 1 & : & 6 \\ 0 & 1 & 2 & : & 4 \\ 0 & 1 & \lambda - 1 & : & \mu - 6 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 - R_2$:       $[A : B] \sim \begin{bmatrix} 1 & 1 & 1 & : & 6 \\ 0 & 1 & 2 & : & 4 \\ 0 & 0 & \lambda - 3 & : & \mu - 10 \end{bmatrix}$

  • Comprehensive parameter analysis:

  • Unique Solution: Requires $\rho(A) = \rho([A : B]) = 3$. This occurs when $\lambda - 3 \neq 0 \implies \lambda \neq 3$, for any real value of $\mu$.

• No Solution (Inconsistent): Requires $\rho(A) \neq \rho([A : B])$. This occurs when $\lambda - 3 = 0$ and $\mu - 10 \neq 0 \implies \lambda = 3$ and $\mu \neq 10$, giving $\rho(A) = 2$ and $\rho([A : B]) = 3$.

  • Infinite Solutions: Requires $\rho(A) = \rho([A : B]) = 2 < 3$. This occurs when $\lambda - 3 = 0$ and $\mu - 10 = 0 \implies \lambda = 3$ and $\mu = 10$.

• Problem 12: Parametric Investigation with $\lambda$ and $\mu$:

  • System: $2x + 3y + 5z = 9$, $7x + 3y - 2z = 8$, $2x + 3y + \lambda z = \mu$.

  • Augmented matrix reduction:     $[A : B] = \begin{bmatrix} 2 & 3 & 5 & : & 9 \\ 7 & 3 & -2 & : & 8 \\ 2 & 3 & \lambda & : & \mu \end{bmatrix}$

  • Apply $R_2 \rightarrow 2R_2 - 7R_1$ and $R_3 \rightarrow R_3 - R_1$:       $2R_2 - 7R_1 = \begin{bmatrix} 14 - 14 & 6 - 21 & -4 - 35 & : & 16 - 63 \end{bmatrix} = \begin{bmatrix} 0 & -15 & -39 & : & -47 \end{bmatrix}$       $R_3 - R_1 = \begin{bmatrix} 0 & 0 & \lambda - 5 & : & \mu - 9 \end{bmatrix}$       $[A : B] \sim \begin{bmatrix} 2 & 3 & 5 & : & 9 \\ 0 & -15 & -39 & : & -47 \\ 0 & 0 & \lambda - 5 & : & \mu - 9 \end{bmatrix}$

  • Consistency cases:

  • Unique Solution: $\rho(A) = \rho([A : B]) = 3$, requiring $\lambda - 5 \neq 0 \implies \lambda \neq 5$ for any value of $\mu$.

• No Solution: $\rho(A) \neq \rho([A : B])$, requiring $\lambda = 5$ (so $\rho(A) = 2$) and $\mu \neq 9$ (so $\rho([A : B]) = 3$).

  • Infinite Solutions: $\rho(A) = \rho([A : B]) = 2 < 3$, requiring $\lambda = 5$ and $\mu = 9$.

• Problem 13: Homogeneous System Analysis:

  • System: $2x - y + 3z = 0$, $3x + 2y + z = 0$, $x - 4y + 5z = 0$.

  • Augmented matrix reduction:     $[A : 0] = \begin{bmatrix} 2 & -1 & 3 & : & 0 \\ 3 & 2 & 1 & : & 0 \\ 1 & -4 & 5 & : & 0 \end{bmatrix}$

  • Apply $R_1 \leftrightarrow R_3$:       $[A : 0] \sim \begin{bmatrix} 1 & -4 & 5 & : & 0 \\ 3 & 2 & 1 & : & 0 \\ 2 & -1 & 3 & : & 0 \end{bmatrix}$

  • Apply $R_2 \rightarrow R_2 - 3R_1$ and $R_3 \rightarrow R_3 - 2R_1$:       $R_2 = \begin{bmatrix} 0 & 2 - 3(-4) & 1 - 3(5) & : & 0 \end{bmatrix} = \begin{bmatrix} 0 & 14 & -14 & : & 0 \end{bmatrix}$       $R_3 = \begin{bmatrix} 0 & -1 - 2(-4) & 3 - 2(5) & : & 0 \end{bmatrix} = \begin{bmatrix} 0 & 7 & -7 & : & 0 \end{bmatrix}$       $[A : 0] \sim \begin{bmatrix} 1 & -4 & 5 & : & 0 \\ 0 & 14 & -14 & : & 0 \\ 0 & 7 & -7 & : & 0 \end{bmatrix}$

  • Apply $R_3 \rightarrow 2R_3 - R_2$:       $[A : 0] \sim \begin{bmatrix} 1 & -4 & 5 & : & 0 \\ 0 & 14 & -14 & : & 0 \\ 0 & 0 & 0 & : & 0 \end{bmatrix}$

  • Rank analysis: $\rho(A) = 2 < n = 3$. Because the rank is strictly less than the number of unknowns, non-trivial solutions exist.

  • Back substitution:

  • From equation 2: $14y - 14z = 0 \implies y = z$.

  • Let $z = k$ (arbitrary constant), then $y = k$.

  • From equation 1: $x - 4k + 5k = 0 \implies x + k = 0 \implies x = -k$.

  • Complete solution: $x = -k$, $y = k$, $z = k$.

• Problem 14: Homogeneous System with Trivial Solution:

  • System: $x + 2y + 3z = 0$, $3x + 4y + 4z = 0$, $7x + 10y + 12z = 0$.

  • Augmented matrix reduction:     $[A : 0] = \begin{bmatrix} 1 & 2 & 3 & : & 0 \\ 3 & 4 & 4 & : & 0 \\ 7 & 10 & 12 & : & 0 \end{bmatrix}$

  • Apply $R_2 \rightarrow R_2 - 3R_1$ and $R_3 \rightarrow R_3 - 7R_1$:       $R_2 = \begin{bmatrix} 0 & 4 - 6 & 4 - 9 & : & 0 \end{bmatrix} = \begin{bmatrix} 0 & -2 & -5 & : & 0 \end{bmatrix}$       $R_3 = \begin{bmatrix} 0 & 10 - 14 & 12 - 21 & : & 0 \end{bmatrix} = \begin{bmatrix} 0 & -4 & -9 & : & 0 \end{bmatrix}$       $[A : 0] \sim \begin{bmatrix} 1 & 2 & 3 & : & 0 \\ 0 & -2 & -5 & : & 0 \\ 0 & -4 & -9 & : & 0 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 - 2R_2$:       $R_3 = \begin{bmatrix} 0 & -4 - 2(-2) & -9 - 2(-5) & : & 0 \end{bmatrix} = \begin{bmatrix} 0 & 0 & 1 & : & 0 \end{bmatrix}$       $[A : 0] \sim \begin{bmatrix} 1 & 2 & 3 & : & 0 \\ 0 & -2 & -5 & : & 0 \\ 0 & 0 & 1 & : & 0 \end{bmatrix}$

  • Back substitution:

  • Equation 3 yields $z = 0$.

  • Equation 2 yields $-2y - 5(0) = 0 \implies y = 0$.

  • Equation 1 yields $x + 2(0) + 3(0) = 0 \implies x = 0$.

  • The system admits only the trivial solution: $x = 0$, $y = 0$, $z = 0$.