emth211 exam

What is the goal of least squares?

To find the vector of coefficients that minimises the length of the residual vector squared

How is the residual component ri defined for a linear fit?

yi minus the quantity c0 plus c1 times xi

What makes a linear system overdetermined?

Having more rows than columns meaning more equations than unknowns

In the matrix equation Ac is approximately y what does matrix A contain for a linear fit?

A column of ones and a column of x values

Which NumPy function finds the least squares solution?

np.linalg.lstsq

How do you modify matrix A to fit a quadratic relationship?

Add a third column containing the x values squared

Why does Ax equals b usually have no exact solution in an overdetermined system?

Because vector b is unlikely to be in the column space of A

What equation guarantees a least squares solution x hat?

A times x hat equals the projection of b onto the column space of A

How is the residual vector r defined algebraically?

b minus A times x hat

In terms of projections what is the residual vector r equivalent to?

The perpendicular component of b relative to the column space of A

What condition guarantees a unique least squares solution?

The nullity of matrix A must be exactly zero

What are the normal equations for an overdetermined system A x approximately b?

A transpose A x hat equals A transpose b

How is the norm of a vector defined in a general inner product space?

The square root of the inner product of the vector with itself

How is the distance between vectors u and v defined in an inner product space?

The norm of the vector u minus v

When are two vectors considered orthogonal in an inner product space?

When their inner product equals zero

What does the Pythagorean Theorem state for an inner product space?

The squared norm of u plus v equals the squared norm of u plus the squared norm of v if and only if u and v are orthogonal

How is the Gram-Schmidt process adapted for general inner product spaces?

By replacing the standard dot product with the general inner product

How do you compute the orthogonal projection of vector x onto a subspace W with an orthogonal basis?

Take the sum of the inner product of x and each basis vector divided by the inner product of that basis vector with itself multiplied by that basis vector

How is the orthogonal component of a vector x relative to a subspace W calculated?

The original vector x minus the orthogonal projection of x onto W

What mathematically defines the best approximation of a vector v in a subspace W? (for v not in W)

It is the vector in W whose distance to v is strictly less than the distance from v to any other vector in W

According to the Best Approximation Theorem what specific vector provides the best approximation to a vector v in a finite dimensional subspace W?

The orthogonal projection of v onto W

What happens when a matrix A multiplies one of its eigenvectors x?

It simply scales the vector x by a factor called the eigenvalue

How can you efficiently compute a matrix A to a high power times an eigenvector x?

By calculating the eigenvalue to that same power times the vector x

Why is it useful to represent an arbitrary vector in a basis of eigenvectors? And what conditions are required to use this representation?

It makes computing matrix powers much simpler because each eigenvector component only gets scaled by its eigenvalue (this works as a full basis only when the matrix has enough linearly independent eigenvectors; only when diagonalisable)

What is the equation determinant of A minus lambda times the identity matrix equals zero called?

The characteristic equation for matrix A

After finding the eigenvalues how do you calculate the corresponding eigenvectors?

By solving the linear system A minus lambda times the identity matrix times v equals the zero vector for each eigenvalue

If v is an eigenvector of A with eigenvalue lambda what is a solution to the differential equation system y prime equals A y?

The vector z equals e to the power of lambda t times v

What is the characteristic polynomial of an n by n matrix A?

A polynomial of degree n defined as the determinant of A minus lambda times the identity matrix

What do the roots of the characteristic polynomial represent?

The eigenvalues of the matrix A

What is the trace of a matrix?

The sum of its diagonal elements

What are the coordinates of a vector relative to a basis

The coefficients used to express the vector as a linear combination of the basis vectors

What does numpy.linalg.eig return

A vector of eigenvalues and a matrix whose columns are the corresponding eigenvectors

How are eigenvectors returned by NumPy usually scaled

They are normalised to have unit 2 norm

Why can NumPy return a tiny nonzero value when the exact answer is zero

Numerical calculations are subject to rounding error

What is true about eigenvectors associated with distinct eigenvalues

They are linearly independent

What is the algebraic multiplicity of an eigenvalue

The number of times its factor is repeated in the characteristic polynomial

What is the geometric multiplicity of an eigenvalue

The dimension of its eigenspace

How can geometric multiplicity be found using row reduction

Count the free parameters in the eigenvector system

How are geometric and algebraic multiplicity related

Geometric multiplicity is at least 1 and no greater than algebraic multiplicity

What is a defective matrix

A matrix with an eigenvalue whose geometric multiplicity is less than its algebraic multiplicity

Does a repeated eigenvalue always make a matrix defective

No because its geometric multiplicity can equal its algebraic multiplicity

What is the eigenspace associated with an eigenvalue

The solution space of A minus lambda I times v equals zero

Is every vector in an eigenspace an eigenvector

No because the zero vector belongs to the eigenspace but is not an eigenvector

How can the eigenvalues of a triangular matrix be found

They are the entries on the main diagonal

Can eigenvalues of a real matrix be complex

Yes some roots of the characteristic polynomial may be complex

What does the matrix P contain in a diagonalisation

P has the eigenvectors of A as its columns

What does the matrix D contain in a diagonalisation

D has the eigenvalues matching the columns of P on its diagonal

What relationship connects A with P and D in a diagonalisation

A equals P D P inverse

What must be true of P for diagonalisation to work

P must be invertible

Does the order of eigenvalues in D matter

No as long as each eigenvalue stays matched with its eigenvector column in P

When is a matrix diagonalisable

When it has enough linearly independent eigenvectors to form a basis

Can a real matrix require complex matrices P and D for diagonalisation

Yes

How do complex eigenvalues occur for a real matrix

They occur in complex conjugate pairs

How are eigenvectors related for conjugate eigenvalues of a real matrix

The corresponding eigenvectors can be chosen as complex conjugates

What change of variables decouples the system y prime equals A y when A is diagonalisable

Use z equals P inverse y

What system results after changing to eigenvector coordinates

z prime equals D z

What does it mean for a system of ODEs to be decoupled

Each component equation can be solved independently

What is the solution form for one decoupled component

z i equals c i times e to the power lambda i t

How can the initial coefficient vector c be found

Solve P c equals y at time zero

Why is solving P c equals y at time zero preferred over computing P inverse explicitly

It avoids unnecessary computation of the inverse

What does e to the power D t look like when D is diagonal

It is diagonal with e to the power lambda i t on each diagonal entry

How is the original ODE solution recovered from the decoupled variables

Multiply the decoupled solution by P

If A is invertible how does an eigenvalue change for A inverse

lambda becomes one divided by lambda with the same eigenvector

When is zero an eigenvalue of A

Exactly when A is singular

If p is the characteristic polynomial of A what matrix does p of A equal

The zero matrix

Does the Cayley Hamilton Theorem only apply to diagonalizable matrices

No it also applies to defective matrices

How is the matrix exponential defined

By the Taylor series I plus At plus one half A squared t squared and continuing

If A is diagonalizable how is its matrix exponential computed

Use P times exponential of Dt times P inverse

What identity allows powers of a diagonalizable matrix to be simplified

A to the kth power equals P times D to the kth power times P inverse

Can matrix exponentials still be computed when A is defective

Yes methods such as scipy.linalg.expm do not require diagonalization

How are the characteristic polynomials of A and A transpose related

They are identical

Must A and A transpose have the same eigenvectors

No their eigenvectors can differ

When comparing fits to the same data using the same residual definition what does a smaller squared residual norm mean?

A better fit according to that least squares objective

When does the reduced QR equation R times x hat equals Q transpose times b give a unique least squares solution?

When A has full column rank and Q has orthonormal columns with A equals Q R

How are all least squares solutions related when A has a nonzero null space?

Every solution equals one least squares solution plus an arbitrary vector in the null space of A

For full column rank A how does forming the normal equations affect its two norm condition number?

The condition number of A transpose A is the square of the condition number of A

What design matrix is used for a degree n polynomial fit in the monomial basis?

A Vandermonde matrix with columns of ones and x values and successive powers up to x to the power n

Why can a monomial Vandermonde matrix cause trouble in polynomial fitting?

It can be badly conditioned especially for high degree fits or poorly scaled x values

How can y equals a times e to the power b x be linearised using real logarithms?

For positive a and positive y fit ln y equals ln a plus b x and recover a by exponentiating the intercept

How can y equals a times x to the power k be linearised using real logarithms?

For positive a and positive x and positive y fit ln y equals ln a plus k times ln x

What does linear least squares fitting of a circle minimise?

Squared residuals of the expanded algebraic circle equation rather than squared radial distances to the circle

What are the four requirements for a real inner product?

Symmetry and additivity and scalar homogeneity and positive definiteness meaning the self inner product is nonnegative and is zero exactly for the zero vector

When is a weighted coordinate dot product an inner product on all of Rn?

When every coordinate weight is strictly positive

What does a Taylor polynomial prioritise when approximating a smooth function?

Matching derivatives at its expansion point to give local accuracy near that point

What error does projection onto a polynomial subspace minimise when the inner product is the integral of f times g?

The integral of the squared error across the interval rather than the error at each individual point

For an eigenvalue lambda why must A minus lambda times I be singular?

Its null space must contain a nonzero eigenvector

How is the determinant related to eigenvalues when some are repeated?

It is their product with every eigenvalue counted according to its algebraic multiplicity

How is the trace related to eigenvalues when some are repeated?

It is their sum with every eigenvalue counted according to its algebraic multiplicity

If a real n by n matrix has n distinct eigenvalues when do their eigenvectors form a basis for Rn?

When all the eigenvalues are real otherwise the eigenvectors form a basis for complex n dimensional space

How can a proposed diagonalisation be checked without finding P inverse?

Check that P is invertible and D is diagonal and A P equals P D

If A times v equals lambda times v what happens when a polynomial q is applied to A?

The same nonzero vector v is an eigenvector of q of A with eigenvalue q of lambda

How does Cayley Hamilton reduce a high matrix power?

Use the characteristic equation to replace powers of degree n and above by combinations of lower powers

How can Cayley Hamilton be used to find the inverse of an invertible matrix?

Multiply its characteristic equation by A inverse and isolate A inverse using the nonzero constant coefficient

For constant A what solves y prime equals A y with y at zero equal to y0?

The matrix exponential of A times t multiplied by y0

If A equals P D P inverse how can A transpose be diagonalised?

Use P inverse transpose as the eigenvector matrix and the same diagonal matrix D with P transpose at the end

How can you check a least squares candidate without knowing the exact solution?

Verify that A transpose times the residual is zero

When is a least squares residual zero?

Exactly when the original system is consistent and the fitted vector equals b

Why is a polynomial fit still a linear least squares problem?

It is linear in the unknown coefficients even when it is nonlinear in the input variable

When is a straight line least squares fit unique?

When the x values are not all identical so the columns of ones and x values are independent

How do you recover a circle from x squared plus y squared equals c0 plus c1 x plus c2 y?

Its centre is c1 divided by two and c2 divided by two and its radius is the square root of c0 plus one quarter c1 squared plus one quarter c2 squared