Lec02&03-RunTime-AlgorithmicAnalysis_v1

Algorithm Analysis

Techniques Covered:

• Proving Correctness: Using induction

• Runtime Analysis: Asymptotic analysis

• Specific Problems Discussed: Comparison-sorting problems

Warm-Up Exercises

Example 1: Comparison Sorting

• Pseudo-code to introduce bubble sort logic:

  for i = 1 to length(A) {
    for j = i+1 to length(A) {
      if( A[i] > A[j] ) swap A[i] and A[j] 
    }
  }

Example 2: GCD Function

• Euclidean algorithm for calculating GCD:

  int GCD (int a, int b){
    if b is equal to zero return a;
    return GCD(b, a % b);
  }

Sorting Algorithms

• Purpose: Order sequences of values with applications in organizing data (libraries, databases).

• Applications Include: Finding medians and data compression.

Insertion Sort Overview

• Concept: Insert elements one at a time into their correct position in a growing sorted list.

• Implementation:

  insertionSort( A : array of items ) {
    int holePosition;
    int valueToInsert;
    for i = 1 to length(A) do {
      valueToInsert = A[i];
      holePosition = i;
      while holePosition > 0 and A[holePosition-1] > valueToInsert do {
        A[holePosition] = A[holePosition-1];
        holePosition = holePosition - 1;
      }
      A[holePosition] = valueToInsert;
    }
  }

Step-by-Step Illustration

1. Initial Array: [4, 3, 1, 5, 2]

2. Move elements left until the right position is found.

3. Final Sorted Array: [1, 2, 3, 4, 5]

Correctness of Insertion Sort

Proof by Induction:

• Loop Invariant: The subarray A[0] to A[i] is sorted.

• Components of Proof:

  • Base Case: A[0] is trivially sorted.

  • Inductive Hypothesis: Assume true up to index i.

  • Inductive Step: Inserting A[i+1] into the sorted subarray shows it remains sorted.

Runtime Analysis

Worst-Case Analysis:

• The worst-case scenario occurs with a reverse-ordered list leading to O(n²) complexity.

Best-Case Analysis:

• For an already sorted list, the complexity is Ω(n).

Average-Case Analysis:

• Generally not a concern unless algorithms are randomized.

Asymptotic Notation

• Big-O Notation: Represents upper bounds of function growth ignoring constants.

• Example:

  • O(n + log(n)) = O(n)

Summary of Big-O and Big-Ω Notations:

• Big-O provides an upper-bound expectation for algorithm performance.

• Big-Ω gives a lower-bound metric similarly defined as T(n) is Ω(g(n)).

Techniques to Analyze Algorithms

• Correctness: Confirm correct implementation via induction.

• Runtime: Analyze performance characteristics using asymptotic notation.