AMATH Quiz 7

General Optimization

Recognize and define the “objective function” in an optimization problem.
- The function that you are trying to maximize or minimize.
- It can be a function of one variable, or it can be a function of several variables.

Differentiate between argmin, minimum, argmax, and maximum.
- argmin
  - The x value at which the minimum occurs.
- minimum
  - The minimum value of the output of the function.
- argmax
  - The x value at which the maximum occurs.
- maximum
  - The maximum value of the output of the function.

Solve maximization problems by recasting as a minimization problem and using a minimization technique.
- $\max\left(g\left(x\right)\right)=-\min\left(-h\left(x\right)\right)$

1D Optimization

Minimize functions using scipy.optimize.fsolve to find the root of the derivative of the function.
- Steps
  - Take the derivative of the function analytically, just as you did in Calculus.
  - Find the root numerically using scipy.optimize.fsolve.
  - Check to see if it’s a max or min by checking a nearby point or looking at a graph.
- Example:

import numpy as np
import matplotlib.pyplot as plt
f = lambda x: x**2*np.sin(x)
f_prime_factor = lambda x: x*np.cos(x) + 2*np.sin(x)
from scipy.optimize import fsolve
crit_point = fsolve(f_prime_factor, -2)[0]
print("Minimizer = ", crit_point)
print("Minimum =", f(crit_point))

Minimize functions using scipy.optimize.fminbound.
- Syntax:
  - fminbound(function, a, b)
- Example:

import numpy as np
import matplotlib.pyplot as plt
f = lambda x: x**2*np.sin(x)
xplot = np.linspace(-5, 5, 200)
fig, ax = plt.subplots() 
ax.plot(xplot, f(xplot), linewidth=3, label=r"$f(x)$")
ax.legend()
plt.xlim([-3, -1]) # Restrict the x range for the plot
plt.ylim([-5, 0])  # Restrict the y range for the plot
from scipy.optimize import fminbound
argmin = fminbound(f, -3, -1) # Syntax is fminbound(function, left_bound, right_bound)
print("Minimizer = ", argmin)
print("Minimum =", f(argmin))
plt.show()

Multivariable Optimization

Package variables to create functions with array inputs and outputs.

Recognize critical points of multi-dimensional functions from a surface plot and/or a contour plot.

Recognize and explain the steps of the gradient descent algorithm.
- Steps
  - Calculate the gradient, $\Delta f\left(p_0\right)$ .
  - Define the line in the negative direction of the gradient, $\Phi\left(t\right)=p_0-t\cdot\Delta f\left(p_0\right)$ .
  - Define the function of heights along the line, $f\left(\Phi\left(t\right)\right)$ .
  - Use fminbound to find t*, the minimizer of the heights along the line $f\left(\Phi\left(t\right)\right)$ .
  - Define $p_1=f\left(\Phi\left(t^{\ast}\right)\right)$ .

Use break statements to exit loops when an iterative process converges and when it is not converging.

Implement the 2-norm as a tolerance check for multidimensional algorithms.

Use scipy.optimize.fmin to find minima of 2D functions.

2D Plotting

Create meshes from two arrays using np.meshgrid.

import numpy as np
x = np.linspace(-2, 1, 4)
y = np.linspace(-4, 4, 3)

X_grid, Y_grid = np.meshgrid(x,y)

Create surface and contour plots of 2-dimensional functions using contour and plot_surface.

Contour Plot

# Create our function
f = lambda x, y: (x-2)**2 + (y+1)**2 + 5*np.sin(x)*np.sin(y) + 100 

fig, ax = plt.subplots()

# 🚨 Create the contour plot
ax.contour(X_grid, Y_grid, f(X_grid, Y_grid))

Surface Plot

# Create our function
f = lambda x, y: (x-2)**2 + (y+1)**2 + 5*np.sin(x)*np.sin(y) + 100 

# 🚨 Tell matplotlib we are making a 3D plot
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
# 🚨 Plot the surface by evaluating f on the grid.
ax.plot_surface(X_grid, Y_grid, f(X_grid, Y_grid))

Basic Python Functions

for / while
- For Format
```
for val in sequence:
# some action
```
- While Format

while condition:
# some action

NumPy

linalg.norm
- numpy.linalg.norm(x)
- Calculates vector and matrix norms

Scipy

optimize.fmin
- argmin = fmin(fp, [0, 0])