Linear Algebra: Section 1.7 - Linear Independence

Indexed Set of Vectors: An indexed set of vectors $\{v_1, \dots, v_{p}\}$ in $\mathbb{R}^n$ is defined to be linearly independent if the vector equation $x_1 v_1 + x_2 v_2 + \dots + x_p v_p = 0$ has only the trivial solution.
Linearly Dependent Set: The set $\text{S} = \{v_1, \dots, v_{p}\}$ is said to be linearly dependent if there exist weights $c_1, \dots, c_p$ , not all zero, such that: $c_1 v_1 + c_2 v_2 + \dots + c_p v_p = 0$
Linear Dependence Relation: The equation $c_1 v_1 + c_2 v_2 + \dots + c_p v_p = 0$ is called a linear dependence relation among the vectors $v_1, \dots, v_p$ specifically when the weights are not all zero.
Mutual Exclusivity: An indexed set is linearly dependent if and only if it is not linearly independent.

Objective: Determine if the set $\text{S} = \{v_1, v_2, v_3\}$ is linearly independent and find a linear dependence relation if one exists.
Methodology: To determine independence, one must evaluate if there is a nontrivial solution to the vector equation: $x_1 v_1 + x_2 v_2 + x_3 v_3 = 0$
Matrix Analysis: Row operations on the associated augmented matrix for the system reveal the structure of the solution set.
- In this specific case, calculations show that $x_1$ and $x_2$ are basic variables.
- The variable $x_3$ is a free variable.
- Because each nonzero value assigned to $x_3$ determines a nontrivial solution, the set $\text{S}$ is concluded to be linearly dependent.
Finding a Specific Relation: By row reducing the augmented matrix to reveal the system, the relationship between variables is established:
- $x_3$ is free.
- A specific nonzero value for $x_3$ is selected (e.g., $x_3 = 10$ ).
- Substituting this into the reduced system yields $x_1 = 3$ and $x_2 = -2$ .
- Substituting these weights into the original vector equation produces the linear dependence relation: $3v_1 - 2v_2 + 10v_3 = 0$
- This is only one of infinitely many possible relations among these vectors.

Relation to Matrix Equations: If a set of vectors is treated as the columns of a matrix $A$ , the linear independence of those columns is tied directly to the equation $Ax = 0$ .
The Equation Break Down: The equation $Ax = 0$ can be expressed as: $x_1 a_1 + x_2 a_2 + \dots + x_n a_n = 0$
Correspondence: Every linear dependence relation among the columns of matrix $A$ corresponds to a nontrivial solution of the matrix equation $Ax = 0$ .
The Rule: The columns of matrix $A$ are linearly independent if and only if the equation $Ax = 0$ has only the trivial solution.

Single-Vector Sets: A set containing only one vector $\{v\}$ is linearly independent if and only if $v \neq 0$ .
- The equation $x_1 v = 0$ has only the trivial solution $x_1 = 0$ when $v \neq 0$ .
- The Zero Vector: A set consisting solely of the zero vector $\{0\}$ is linearly dependent because the equation $x_1 0 = 0$ has many nontrivial solutions (any value of $x_1$ works).
Two-Vector Sets: A set of two vectors $\text{S} = \{v_1, v_2\}$ is linearly dependent if and only if at least one of the vectors is a multiple of the other.
- Conversely, the set is linearly independent if and only if neither vector is a multiple of the other.

Theorem Statement: An indexed set $S = \{v_1, \dots, v_p\}$ of two or more vectors is linearly dependent if and only if at least one of the vectors in $S$ is a linear combination of the others.
Sequence-Based Dependence: If $S$ is linearly dependent and $v_1 \neq 0$ , then there exists some $v_j$ (where j > 1) that is a linear combination of the preceding vectors $v_1, \dots, v_{j-1}$ .
Proof Logic (Subset - Linear Combination to Dependence):
- If some $v_j$ is a linear combination of other vectors (e.g., $v_j = c_1 v_1 + \dots + c_{j-1} v_{j-1}$ ), then $v_j$ can be subtracted from both sides to produce a linear dependence relation with a non-zero weight (specifically $-1$ ) on $v_j$ , thus proving the set is linearly dependent.
Proof Logic (Subset - Dependence to Linear Combination):
- If $S$ is linearly dependent, weights exist (not all zero) where $c_1 v_1 + \dots + c_p v_p = 0$ .
- If $v_1 = 0$ , it is already a trivial linear combination of other vectors.
- If $v_1 \neq 0$ , let $j$ be the largest subscript such that $c_j \neq 0$ . If j > 1, we can solve for $v_j$ : $v_j = (\frac{-c_1}{c_j})v_1 + \dots + (\frac{-c_{j-1}}{c_j})v_{j-1}$
Important Distinction: Theorem 7 does not state that every vector in a linearly dependent set must be a linear combination of the others. It only requires that at least one such vector exists.

Scenario: Let vectors $u, v, w$ be in $\mathbb{R}^3$ , where $u$ and $v$ are linearly independent.
Spanning and Planes: Because $u$ and $v$ are independent (neither is a multiple of the other), they span a plane in $\mathbb{R}^3$ (e.g., the $x_1 x_2$ -plane if the third coordinates are zero).
Vector Inclusion: A vector $w$ is in $\text{Span}\{u, v\}$ if and only if the set $\{u, v, w\}$ is linearly dependent.
- By Theorem 7, if the set is dependent, one vector must be a linear combination of preceding vectors. Since $v$ is not a multiple of $u$ , that vector must be $w$ , placing $w$ in the plane spanned by $u$ and $v$ .

Theorem Statement: If a set contains more vectors than there are entries in each vector, then the set is linearly dependent. Specifically, any set in $\mathbb{R}^n$ is linearly dependent if the number of vectors $p$ is greater than the number of entries $n$ . (p > n).
Proof by Matrix Properties:
- Let $A = [v_1, v_2, \dots, v_p]$ .
- The matrix $A$ is of size $n \times p$ .
- The equation $Ax = 0$ corresponds to a system of $n$ equations in $p$ unknowns.
- If p > n, there are more variables than equations, guaranteed to result in at least one free variable.
- The existence of a free variable implies the existence of a nontrivial solution, proving the columns (the vectors in the set) are linearly dependent.
Caveat: Theorem 8 provides no information regarding cases where the number of vectors is less than or equal to the number of entries ( $p \leq n$ ).

Theorem Statement: If a set $S = \{v_1, \dots, v_p\}$ in $\mathbb{R}^n$ contains the zero vector ( $0$ ), then the set is linearly dependent.
Proof: By ordering the vectors such that $v_1 = 0$ , we can establish the following linear dependence relation: $1v_1 + 0v_2 + \dots + 0v_p = 0$ Because the weight for $v_1$ is nonzero (it is $1$ ), the set satisfies the definition of linear dependence.