Lecture 10 Local Search and Optimization

This week's focus: Diving into local search and optimization algorithms, which help computers find the best solutions to problems by making small, local changes.
Next week's topic: We'll explore adversarial search and games, where algorithms compete against each other.
Example algorithm: Genetic algorithm. It's inspired by the process of evolution, where the best traits are passed down through generations to find optimal solutions.
Goal: Solving optimization problems using computer functions. These problems require finding the "best" solution from many possibilities using computer programs.

Need a function to optimize, algorithms to optimize functions in the general case. In essence, we need a way to measure how "good" a solution is and a method to improve it.

Hill Climbing

Basic Idea: Imagine you're climbing a hill in dense fog and want to reach the highest point. You take the current state, evaluate the function (check the steepness), and move in the direction of the steepest ascent (for maximization) or descent (for minimization).
Finding a Maximum: Look around and go in the direction where the function increases. Keep doing this until you can't go any higher.
Finding a Minimum: Imagine you're trying to find the lowest point in a valley. Walk in the direction of the steepest descent until you can't go any lower.

Key Topics

Local vs. Global Optima: Think of local optima as small hills in a mountain range and global optima as the highest peak. Hill climbing can get stuck on a local hill, mistaking it for the highest point overall.
Variants of Hill Climbing: Different strategies on how to explore the neighborhood and escape local optima, like random restarts.
Simulated Annealing: Inspired by how metals cool down. It allows the algorithm to sometimes make "bad" moves to escape local optima. It can theoretically guarantee the best outcome, but may take a very long time.
Local Beam Search: Instead of one "climber," use multiple climbers exploring the landscape in parallel.

Previous Search Problems: Concerned with finding a solution and how to get there (the actions required). We cared about the path to the goal.
Function Optimization: Focus may solely be on the final state, not the changes made to reach it. We only care about the best result, not how we got there.

Formula One Example

Optimization Goal: Optimize the aerodynamics of a race car to minimize air resistance, allowing it to go faster.
State: Current configuration of the race car, including the shape and angles of its various parts.
Changes: Local changes to the car (e.g., adjusting the spoiler by a small amount).
Goal: Minimize resistance to airflow to increase speed.
Local Optimum: A configuration where no small changes improve airflow further. The car is well-optimized, but maybe not perfectly.
Computer Simulation: Use wind tunnel simulations with CAD models to evaluate wind resistance. This helps to test different configurations quickly and efficiently.

Problem: Distribute items of different weights into two backpacks to minimize the difference in weight between them. This is often encountered when dividing tasks or resources fairly.
Optimization Goal: Minimize the weight difference for fairness, ensuring that one person isn't carrying significantly more than the other.
Complexity: NP-hard problem, meaning there's no known algorithm that can solve it quickly for large numbers of items.
State: Distribution of items in the two backpacks (which item is in which backpack).
Objective Function: Difference in weight between the backpacks, which we want to minimize.
Minimization vs. Maximization: Can convert between minimization and maximization by taking the additive inverse (e.g., multiplying by -1). If minimizing $x$ is the goal, then maximizing $-x$ achieves the same result.
Objective Function: The function to maximize in optimization problems. It could represent profit, efficiency, or any other desirable outcome.

State: Unsorted list of numbers.
Goal: Sort the numbers in ascending or descending order.
Local Search Approach: Swap elements to reduce the number of out-of-order elements. This can be very slow, especially for large lists.
Efficient Solution: Use specialized algorithms like Quick Sort or Merge Sort, which guarantee the optimal solution in polynomial time. These algorithms are designed specifically for sorting and are much faster.

Problem: Place eight queens on a chessboard such that no two queens attack each other. Queens can attack horizontally, vertically, and diagonally.
Objective Function: Number of pairs of queens attacking each other (to be minimized). The goal is to get this number down to zero.
State: Position of queens on the chessboard describing where each queen is placed.
Local Changes: Move one queen along its column to a different row. This keeps the number of queens in each column constant.

Example Board

Greedy Hill Climbing

Select the move that gives the biggest change in the objective function. Always make the most significant improvement possible at each step.
Iterate until no further improvement can be made. Keep going until you reach a point where no move reduces the number of conflicts.

Approach: Make local changes to the state without fully exploring the state space. Focus on small modifications to the current solution.
Contrast: Systematic algorithms from previous lectures explored options in a search tree. They looked at all possible solutions, which can take a very long time.
Memory: Simplest algorithms keep only the current best state. They don't remember previous states or paths.
More Complex Algorithms: May keep a fixed set of states (e.g., genetic algorithms with a population of states). This allows for more diverse exploration.

Start with an initial state. Begin with a random or pre-selected starting point.
Iterate until no improvement is possible. Keep going as long as you can make the solution better.
Find the highest-value successor of the current state by performing all possible actions. Look at every possible move and choose the one that improves the solution the most.

Chessboard example: move each queen to a different location in its column. Each queen will have 7 possible actions.
Problem definition must define possible actions to allow local changes.

Issue: May get stuck in a local minimum or maximum. The algorithm can find a solution that's good but not the best.
Optimization: Finding the global best state may not be guaranteed, especially in complex problems.

Analogy: Training Deep Neural Networks

Training: Can be viewed as an optimization problem to minimize error on training data. The neural network learns by adjusting its parameters to reduce errors.
Issue: Not guaranteed to find the neural network that globally minimizes error; it only finds a good solution.