Lec01_Algorithms

Algorithms Overview

• Instructor: Mehrdad Nojoumian, Department of Computer & Electrical Engineering and Computer Science, Florida Atlantic University.

• Course: COP 3530: Data Structures and Algorithm Analysis.

Section Summary

• Topics Covered:

  • Properties of Algorithms

  • Algorithms for Searching and Sorting

    • Linear Search

    • Binary Search

    • Bubble Sort

    • Insertion Sort

    • Greedy Algorithms

Definition of Algorithms

• Definition: An algorithm is a finite set of precise instructions for performing a computation or solving a problem.

• Example Algorithm: To find the maximum value in a sequence of integers:

  • Solution: perform the steps below

    1. Set a temporary maximum to the first integer.

    2. Compare each subsequent integer to the temporary maximum.

    3. Update temporary maximum if a larger integer is found. set it to the largest integer

    4. Continue until all integers are processed. otherwise, stop

       Once the algorithm terminates, the temporary maximum represents the largest integer in the sequence.

Specifying Algorithms

• Algorithms can be specified in various ways:

  • English Descriptions

  • Pseudocode: An intermediate representation that combines natural language with coding structures from languages like C++ and Java.

  • Importance of Pseudocode: Facilitates analysis of problem-solving time independently of programming language, aiding in programming.

Properties of Algorithms

• Input: Accepts values from a specified set.

• Output: Produces output values that represent a solution from the input set.

• Correctness: Must consistently generate correct outputs for given inputs.

• Finiteness: Outputs must be delivered after a finite number of steps, regardless of input.

• Effectiveness: Every step must be possible to perform correctly and within a finite timeframe.

• Generality: Should be applicable to all problems of the defined type.

Finding the Maximum Element

• Pseudocode for Maximum Element Algorithm:

  procedure max (a1, a2, …, an: integers)
      max := a1
      for i := 2 to n
          if max < ai then max := ai
      return max

Example Algorithm Problems

• Classes of Problems:

  1. Searching Problems: Finding the position of a specific element in a list.

  2. Sorting Problems: Ordering list elements in increasing order.

  3. Optimization Problems: Determining the optimal value for a quantity across all possible inputs. (Optimal value means maximum or minimum)

Linear Search Algorithm

• Process: Compares target element with each list item one by one:

  1. Compare x with a_¹. If equal, return position 1.

  2. If not, continue comparing with subsequent items. (i.e. try a₂. if x = a₂, return to position 2)

  3. If no match after scanning the list, return 0.

• Pseudocode:

  procedure linear search (x: integer, a1, a2, …, an: distinct integers)
      i := 1
      while (i ≤ n and x ≠ ai)
          i := i + 1
      if i ≤ n then location := i else location := 0
      return location

Binary Search

• Condition: The input list must be sorted in increasing order.

• Process:

  1. Compare the desired element with the middle element of the list.

  2. If smaller, search in the upper half, if larger, search in the lower half.

  3. Repeat until the list size is 1.

  4. Return the position if found, else return 0 if element not found.

Binary Search Pseudocode

procedure binary search (x: integer, a1, a2, …, an: increasing integers)
    i := 1  // left endpoint of interval
    j := n  // right endpoint of interval
    while i < j
        m := ⌊(i + j)/2⌋
        if x > am then i := m + 1
        else j := m
    if x = ai then location := i else location := 0
    return location

Example of Binary Search

• Example List: 1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22

• Procedure Demonstration: Midpoints are calculated until the desired element is located based on comparisons. Final return indicates the found position.

  The list has 16 elements, therefore the midpoint is 8. The value in the 8th position is 10. Since 19 > 10, further search is restricted to positions 9 through 16.

  1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22

  The midpoint of the list (positions 9 through 16) is now the 12th position with a value of 16. Since 19 > 16, further search is restricted to the 13th position and above.

  1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22

  The midpoint of the current list is now the 14th position with a value of 19. Since 19 ≯ 19, further search is restricted to the portion from the 13th through the 14th positions

  1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22

  The midpoint of the current list is now the 13th position with a value of 18. Since 19>18, search is restricted to the portion from the 14th position through the 14th.

  1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22

  Now the list has one element and the loop ends. Since 19=19, the location 14 is returned

Bubble Sort

• Pseudocode:

  procedure bubblesort (a1,…,an: real numbers with n ≥ 2)
      for i := 1 to n− 1
          for j := 1 to n − i
              if aj > aj+1 then interchange aj and aj+1

Insertion Sort

• Process: Starts with the 2nd element, placing each in the correct position.

• Pseudocode:

  procedure insertion sort (a1,…,an: real numbers with n ≥ 2)
      for j := 2 to n
          i := 1
          while aj > ai
              i := i + 1
          temp := aj
          for k := 0 to j − i − 1
              aj-k := aj-k-1
          ai := temp

Greedy Algorithms

• Optimization Problems: Aim to minimize or maximize parameters over potential inputs:

  • Example Problems:

  • Finding a route between two cities with the smallest total mileage.

  • Finding the fiber links among nodes using the least amount of fiber.

• Greedy Approach: Chooses the “best immediate option” at each step, which may not always lead to the optimal overall solution, but often does.

• The greedy approach is an algorithmic paradigm, which is a general

  approach for designing an algorithm.

Greedy Change-Making Change

• Problem: design a greedy algorithm for making change of n cents with the

  following coins: quarters (25 cents), dimes (10 cents), nickels (5 cents),

  and pennies (1 cent), using the least number of coins

• Solution Process: at each step choose the coin with the largest possible value that does not exceed the amount of change left. For instance:

• Example: For n = 67 cents:

  1. Use two quarters, one dime, one nickel, and two pennies.

  2. first choose a quarter leaving 67−25 = 42 cents. Then choose

     another quarter leaving 42 −25 = 17 cents.

     Then choose a dime, leaving 17 − 10 = 7 cents.

     Choose a nickel, leaving 7 – 5 = 2 cents.

     Choose a penny, leaving 1 cent. Choose another penny leaving 0 cents.

  3. Total coins used: 6.

Greedy Change-Making Algorithm

• Pseudocode for Change-Making:

  • Solution: greedy change-making algorithm for n cents. The algorithm works with any coin denominations c_¹, c₂, …,c_r

    procedure change (c1, c2, …, cr: values of coins, where c1 > c2 > … > cr; n: positive integer)
        for i := 1 to r
            di := 0
            while n ≥ ci
                di := di + 1
                n = n - ci

• For the example of US currency, we may have quarters, dimes,

  nickels and pennies, with c₁ = 25, c₂ = 10, c₃ = 5, and c₄ = 1

Proving Optimality for US Coins

• Key Observations (Lemma 1):

  • if n is a positive integer, n cents in change using quarters, dimes,

    nickels, and pennies, using the fewest coins possible has

  • Maximum combinations include at most 2 dimes (2×10c), 1 nickel (1×5c), and 4 pennies (4×1c).

  • cannot have 2 dimes and a nickel (2x10c + 5c), and total amount of change in dimes, nickels, pennies must not exceed 24c

  • Proof: by contradiction

    If we had 3 dimes, we could replace them with a quarter and a nickel.

    If we had 2 nickels, we could replace them with 1 dime.

    If we had 5 pennies, we could replace them with a nickel (5c).

    If we had 2 dimes and 1 nickel, we could replace them with a quarter

  • Any alternative combination can be converted into a configuration with fewer coins. as all allowable combos have a maximum value of 24 cents

Proving Optimality Theorem

• Theorem Statement: The greedy algorithm for US coins minimizes the number of coins used.

• Proof by Contradiction: Assuming an optimal solution and a positive integer n such that change can be made for n cents using quarters, dimes, nickels, and pennies, with a fewer total number of coins than the greedy algorithm.

• Then, q’ ≤ q where q’ is the number of quarters used in this optimal way

  and q is the number of quarters in the greedy algorithm, i.e., less number

  of quarters in the optimal way. But this is not possible by Lemma 1 since

  the value of coins other than quarters can not be > 24c

• Similarly, by Lemma 1, the two algorithms must have the same number of dimes, nickels, and pennies

Limitations of Greedy Algorithms

• Change-Making Limitations: When certain coins are omitted (e.g., no nickels), the greedy approach may no longer yield the least coins (demonstrated in failure case for 30 cents).

• Optimality depends on the denominations available.

  • US coins, optimality still holds if we add half dollar coins (50 cents) and dollar coins (100 cents).

  • But if we allow only quarters (25 cents), dimes (10 cents), and pennies (1 cent) but not nickels (5 cent), the algorithm no longer produces the minimum number of coins

  • Consider 30 cents when we only have 25c, 10c, and 1c:

    30 = 25 + 1 + 1 + 1 + 1 + 1 (6 coins!)

    But we could use 30 = 10 + 10 + 10 (3 coins)

Greedy Scheduling Problem

• Objective: construct a greedy algorithm that schedules as many talks as possible based on start and end conditions.

• Assumptions: Talks occupy time without overlapping, can begin as one ends, and can only add compatible talks.

• How should we make the “best choice” at each step of the algorithm? That is, which talk should we pick?

  

Greedy Scheduling Algorithm

• Solution Steps:

  1. Choose talks based on earliest ending time among compatible options of those selected.

• Pseudocode:

  procedure schedule (s1 ≤ … ≤ sn: start times, e1 ≤ … ≤ en: end times)
      sort talks by end times
      S := ∅
      for j := 1 to n
          if talk j is compatible with S
              S := S ∪ {talk j}
      return S

Practice Problems for Algorithms Overview:

1. Searching Problem: Given the sorted list of integers A = {3, 6, 8, 10, 15, 20, 24}, use both linear search and binary search to find the position of the integer 10 in the list. What are the steps involved in each search method?

2. Sorting Problem: Given the unsorted list of real numbers B = {5.1, 2.3, 9.6, 4.4, 3.2}, implement the bubble sort and insertion sort algorithms to sort the list in increasing order. Show the state of the list after each pass of both algorithms.

3. Optimization Problem: Suppose you need to make change for 45 cents using the least number of coins possible. You have coins of denominations 25 cents, 10 cents, and 5 cents. Write a greedy algorithm to find the solution and explain its validity.

4. Greedy Scheduling: You have the following talks scheduled with their start and end times:

   • Talk 1: (1, 3)

   • Talk 2: (2, 5)

   • Talk 3: (4, 6)

   • Talk 4: (5, 7) Using the greedy scheduling approach, determine which talks can be scheduled without overlapping, and list them.

5. Finding Maximum Element: Given the list of integers C = {7, 3, 9, 1, 5}, write pseudocode to find the maximum element using the algorithm described in the notes. Then implement the pseudocode in a programming language of your choice.

Algorithms Overview (CLIFF NOTES)

Instructor and Course Information

• Instructor: Mehrdad Nojoumian, Department of Computer & Electrical Engineering and Computer Science, Florida Atlantic University

• Course: COP 3530: Data Structures and Algorithm Analysis

Definition of Algorithms

An algorithm is a finite set of precise instructions for performing a computation or solving a problem.

Example: To find the maximum value in a sequence of integers.

1. Set a temporary maximum to the first integer.

2. Compare each subsequent integer with the temporary maximum.

3. Update temporary maximum if a larger integer is found.

4. Continue until all integers are processed.

5. The temporary maximum represents the largest integer.

Properties of Algorithms

• Input: Accepts values from a defined set.

• Output: Produces output values solving the input set.

• Correctness: Must generate correct outputs consistently.

• Finiteness: Outputs should be delivered in a finite number of steps.

• Effectiveness: Steps must be performed correctly within a finite time frame.

• Generality: Should apply to all problems of a defined type.

Algorithm Examples

• Linear Search: Sequentially compares each list item.

• Binary Search: Requires sorted input, compares desired element with the middle element to decide on the next half to search.

• Bubble Sort: Repeatedly compares and swaps adjacent elements if they are in the wrong order.

• Insertion Sort: Places each subsequent element into its correct position within a sorted subsection.

Greedy Algorithms

• Used for optimization problems (e.g., minimizing or maximizing values).

• Example: Change-making algorithm optimizes the number of coins used by always selecting the largest coin denomination without exceeding the remaining amount.

Greatest Value Determination

• Greedy Change-Making: Demonstrates that the greedy approach is not always optimal (e.g., omitting certain coins can lead to increased total coins).

• Greedy Scheduling Algorithm: Schedules talks based on start and end conditions, maximizing the number of non-overlapping talks.

Summary of Problem Types:

• Searching Problems, Sorting Problems, Optimization Problems, and Scheduling Problems.