least squares

motivation of least squares

when Ax = b is inconsistent, it allows us to solve Ax = b as closely as possible

definition of least squares

least squares solution to Ax = b is a vector xhat in R^n so that:

the length of b - Axhat ≤ the length of b-Ax for all x in R^n

Ax = b(col A)

True or False: If xhat is the least square solution of Ax = b, then Axhat is the point in the column space of A closest to b

True. This is essentially the definition of a least squares solution.

True or False: A least squares solution of Ax = b is a vector x hat such that the length of:

b - Ax is less than or equal to the length of b - Axhat for all x in R^n

False. This inequality should be the other way around:

the length of b - Axhat is less than or equal to the length of b-Ax.

This means that Axhat is the best approximate solution to Ax = b

True or False: The general least squares problem is to find an x that makes Ax as close as possible to b

True

True or False: If b is in the column space of a, the every solution set of Ax = b is a least squares solution

True. If the equation Ax = b can be solved exactly, then any solution is the best possible approximate solution. Ax cannot be made any closer to b

how to find least squares solution xhat when given matrix A and a vector b

Goal is to set A^TA = A^Tb

1. Find A^TA by multiplying A^T by the original matrix A

2. Find A^Tb by multiplying A^T by the vector b

3. Set up an augmented matrix with A^TA on the left and A^Tb on the right

4. Row reduce into RREF

5. xhat is the vector on the right hand side of the matrix

how to find the best fit line using least squares given a set of data points

1. plug in each x and y values into the given function

2. turn the equations into a matrix A and vector b

3. Solve for A^TA

4. Solve for A^Tb

5. Set up an augmented matrix with A^TA on the left and A^Tb on the right

6. Row reduce into RREF

7. xhat is the vector on the right hand side of the matrix

8. Plug in the entries of xhat into the given function