Lecture 1

Abstract Data Types

Abstract data types separate the interface from the implementation.

• The interface is what users (including yourself in the future) need to interact with the code.

• Implementation details are hidden from the user.

• Example: sorted sorts a list, but the user doesn't need to know the sorting algorithm used.

Advantages of Separation

• Users don't require a deep understanding of the code.

• Allows for changing the implementation without breaking code that depends on it.

Abstract Data Types in Python

• Implemented primarily 'by convention'.

• Variables and functions starting with an underscore _ are considered implementation details.

  • Should not be needed to use the program.

  • Should not be addressed from outside code.

• Classes and functions should be well-documented.

  • Clearly describe expected input and output in docstrings.

  • Use typing hints whenever possible.

Why Separate Declaration and Implementation?

• Easier use of the class.

  • Programmers don't need to know all implementation details to use the code.

Updates are easier to implement.

• If a more efficient algorithm is found, it can be implemented without affecting code that uses the original interface methods.

• Code using underscore methods/attributes may break if the implementation changes.

Example - math

• Users generally don't care about the implementation of mathematical functions, but need to know which functions exist and how to use them.

• Examples:

  • math.cos(x)

  • math.sin(x)

  • math.tan(x)

  • math.ceil(x)

  • math.floor(x)

  • math.exp(x)

  • math.log(x, base=e)

  • math.log2(x)

  • math.log10(x)

  • math.pow(x, y)

  • math.sqrt(x)

  • math.fabs(x)

Stacks

Stack Example

• Consider a stack of mail in a mailbox.

• New mail is added to the top.

• Mail is processed starting from the top.

• This is known as Last-In-First-Out (LIFO).

Conceptual Stack Actions

• Adding items to the top.

• Removing items from the top.

• Everything else is implementation details.

Advantages of Stacks:

• Simple and efficient for LIFO operations.

• Memory is managed efficiently.

Disadvantages of Stacks:

• Limited access; can only access the top element.

• Risk of stack overflow if the stack grows too large.

Abstract Stack (stack.py)

• The abstract Stack class defines the interface:

class Stack:
    """Creates a stack that allows for last-in-first-out (LIFO) access"""
    def pop(self):
        """Removes and returns the item on top of the stack
        :return: item on top of the stack"""
        pass
    def push(self, item):
        """Adds an item to the top of the stack
        :param item: item to add"""
        pass
    def size(self) -> int:
        """Returns the number of items on the stack
        :return: number of items on the stack"""
        pass

Creating a Stack

• Implementing the stack as a list.

class Stack:
    def __init__(self):
        self.stack = []
    def pop(self):
        return self.stack.pop()
    def push(self, item) -> None:
        self.stack.append(item)
    def size(self) -> int:
        return len(self.stack)

Memory Management

• When creating a variable, Python needs to store it in memory.

• Memory can be thought of as a long strip, with each cell representing a certain number of bytes (e.g., 4 bytes).

• The sys module allows inspecting memory sizes.

import sys
int_variable = 100
float_variable = 100.0
print(sys.getsizeof(int_variable))  # 28 bytes
print(sys.getsizeof(float_variable)) # 24 bytes

• Python stores the list and its content separately.

• Individual items can be stored anywhere in memory.

• Memory locations of the individual items are stored in a connected section of memory.

• When an item is appended:

  • The list may need to request a larger memory block from the operating system.

  • The new item's memory location is added to the list of memory locations.

Growing Lists

• When a list grows beyond its initially allocated memory, Python automatically allocates a new, larger memory block.

• The contents of the old memory block are copied to the new one.

• This reallocation process can have performance implications, especially for very large lists.

Queues

• Queues operate on a First-In-First-Out (FIFO) principle.

  • Items are added to the end (enqueue) and removed from the front (dequeue).

Queue Example

• Consider a line at a ticket counter.

• People join the line at the end.

• The person at the front is served first.

Conceptual Queue Actions

• Enqueue: Adding items to the end.

• Dequeue: Removing items from the front.

• Everything else is an implementation detail.

Advantages of Queues:

• Ensures fair processing order (FIFO).

• Handles multiple requests or tasks in an orderly manner.

Disadvantages of Queues:

• Can be inefficient if priority is needed (all items treated equally).

• May require more memory if the queue grows large.

Abstract Queue (queue.py)

• The abstract Queue class defines the interface:

class Queue:
    """Creates a queue that allows for first-in-first-out (FIFO) access"""
    def dequeue(self):
        """Removes and returns the item on top of the queue
        :return: item at front of the queue"""
        pass
    def enqueue(self, item):
        """Adds an item to the back of the queue
        :param item: item to add"""
        pass
    def size(self) -> int:
        """Returns the number of items on the queue
        :return: number of items on the queue"""
        pass

Creating a Queue

• Implementing the queue as a list.

class Queue:
    def __init__(self):
        self.queue = []

    def enqueue(self, item):
        self.queue.append(item)

    def dequeue(self):
        if not self.is_empty():
            return self.queue.pop(0)
        else:
            return None

    def is_empty(self):
        return len(self.queue) == 0

    def size(self):
        return len(self.queue)

Implementing a Queue Using Two Stacks

• Queues can also be implemented using two stacks to simulate FIFO behavior.

• One stack is used for enqueue operations (adding to the rear), and the other for dequeue operations (removing from the front).

Enqueue Operation (Using Two Stacks)

1. Push the item onto the first stack (the enqueue stack).

Dequeue Operation (Using Two Stacks)

1. If the second stack (the dequeue stack) is empty, transfer all elements from the first stack to the second stack by popping them from the first stack and pushing them onto the second stack.

2. Pop the item from the second stack (the dequeue stack). If the second stack is empty after this operation, it means the queue is empty.

Example Implementation

class QueueUsingTwoStacks:
    def __init__(self):
        self.enqueue_stack = []
        self.dequeue_stack = []

    def enqueue(self, item):
        self.enqueue_stack.append(item)

    def dequeue(self):
        if not self.dequeue_stack:
            while self.enqueue_stack:
                self.dequeue_stack.append(self.enqueue_stack.pop())
        if self.dequeue_stack:
            return self.dequeue_stack.pop()
        else:
            return None

    def is_empty(self):
        return not (self.enqueue_stack or self.dequeue_stack)

    def size(self):
        return len(self.enqueue_stack) + len(self.dequeue_stack)

Explanation

• The enqueue operation simply pushes the new item onto the enqueue_stack.

• The dequeue operation checks if the dequeue_stack is empty. If it is, it moves all elements from the enqueue_stack to the dequeue_stack.

Complexity Analysis:

Stack Operations:

• Push: $O(1)$

• Pop: $O(1)$

• Size: $O(1)$

Queue Operations:

• Enqueue: $O(1)$

• Dequeue: $O(1)$ (for list implementation, $O(n)$; for two-stack implementation, amortized $O(1)$

• Size: $O(1)$

Error Handling:

Stack:

• Popping from an empty stack: Raises an IndexError if not handled. This is known as stack underflow. Stack overflow occurs if attempting to push onto a full stack, which may result in memory errors or exceptions depending on the implementation.

Queue:

• Dequeuing from an empty queue: Returns None or raises an exception if not handled. This is known as queue underflow. Queue overflow occurs if attempting to enqueue onto a full queue, leading to similar issues as stack overflow.

Use Cases:

Stacks:

• Function call stack in programming languages.

• Undo/Redo functionality in applications.

• Expression evaluation and syntax parsing. For example, stacks are used in compilers to convert infix expressions to postfix (RPN) notation.

Queues:

• Task scheduling in operating systems.

• Message passing in distributed systems.

• Print queue management. Queues are also used to handle requests to web servers, managing incoming HTTP requests in an orderly manner.

Alternative Implementations:

Stack:

• Using linked lists instead of lists.

Queue:

• Using collections.deque for efficient 双端队列operations.

• Using linked lists for guaranteed $O(1)$ dequeue operations.

Stack Overflow and Underflow Handling:

Stack Overflow: This occurs when attempting to push an element onto a stack that is already full. Handling stack overflow typically involves checking the stack's capacity before pushing a new element. If the stack is full, an exception can be raised or the push operation can be prevented. In dynamic stacks (e.g., using linked lists), stack overflow is less of a concern as the stack can grow dynamically.

Stack Underflow: This occurs when attempting to pop an element from an empty stack. Handling stack underflow involves checking if the stack is empty before attempting to pop an element. If the stack is empty, an exception can be raised or a default value (e.g., None) can be returned.

Queue Overflow and Underflow:

Queue Overflow: This occurs when attempting to enqueue an element onto a queue that is already full. Similar to stack overflow, handling queue overflow involves checking the queue's capacity before enqueuing a new element. If the queue is full, an exception can be raised or the enqueue operation can be prevented. In dynamic queues (e.g., using linked lists), queue overflow is less of a concern.

Queue Underflow: This occurs when attempting to dequeue an element from an empty queue. Handling queue underflow involves checking if the queue is empty before attempting to dequeue an element. If the queue is empty, an exception can be raised or a default value (e.g., None) can be returned.

Real-World Examples Expanded:

Stacks:

• Compilers: Stacks are used in compilers for expression parsing, such as converting infix notation to postfix notation (Reverse Polish Notation or RPN). They are also used in managing function calls and local variables.

• Virtual Machines: Stacks are used in virtual machines (like the JVM) to manage function calls and local variables during program execution.

• Backtracking Algorithms: Stacks are crucial for implementing backtracking algorithms, such as solving mazes or N-Queens problems.

Queues:

• Web Servers: Queues are used to manage incoming HTTP requests, ensuring that requests are processed in the order they are received. This helps prevent server overload and ensures fair processing.

• Breadth-First Search (BFS): Queues are fundamental in implementing BFS algorithms for graph traversal. BFS is used in various applications, such as finding the shortest path in a graph or network.

• Operating Systems: Queues are used for job scheduling, managing processes waiting for CPU time, and handling I/O requests.

Priority Queues:

• Priority queues are a specialized type of queue where each element is associated with a priority. Elements are dequeued based on their priority, with higher priority elements being dequeued before lower priority elements. Priority queues are commonly implemented using heaps.

Circular Queues:

• Circular queues are an efficient way to implement queues using a fixed-size array. In a circular queue, the queue's head and tail pointers can wrap around the end of the array, allowing for efficient reuse of space. This avoids the need to shift elements when dequeuing, which is a common issue with list-based queue implementations.

• Example: Consider a circular queue implemented with an array of size 5. Initially, the head and tail are at index 0. When elements are enqueued, the tail moves forward. When elements are dequeued, the head moves forward. If either the head or tail reaches the end of the array, it wraps around to the beginning, provided that the space is available.

Complexity Analysis Details:

• List-Based Queue Dequeue Operation: The dequeue operation in a list-based queue has a time complexity of $O(n)$ because removing the first element (at index 0) requires shifting all subsequent elements one position to the left. This shifting operation takes linear time relative to the number of elements in the queue.

• Two-Stack Queue Implementation Dequeue Operation: The dequeue operation in a two-stack queue has an amortized time complexity of $O(1)$. Amortized analysis considers the average time complexity over a sequence of operations. While transferring elements from the enqueue stack to the dequeue stack takes $O(n)$ time, this transfer only occurs when the dequeue stack is empty. Subsequent dequeue operations can then be performed in $O(1)$ time until the dequeue stack is empty again. Thus, the cost of the transfer is spread out over a series of dequeue operations, resulting in an average (amortized) time complexity of $O(1)$.

Space Complexity:

• Stacks and Queues (Array-Based): The space complexity of array-based stacks and queues is $O(n)$, where n is the maximum number of elements the stack or queue can hold. The space is pre-allocated, and the memory usage is constant regardless of the actual number of elements stored.

• Stacks and Queues (Linked List-Based): The space complexity of linked list-based stacks and queues is also $O(n)$, but here n is the actual number of elements stored in the stack or queue. The memory is allocated dynamically as elements are added, providing more flexibility but potentially incurring overhead due to the storage of pointers.

Thread Safety:

• When using stacks and queues in multi-threaded environments, it is crucial to ensure thread safety to prevent race conditions and data corruption. Multiple threads accessing and modifying the same stack or queue concurrently can lead to unpredictable behavior.

Synchronization Mechanisms: To ensure thread safety, synchronization mechanisms such as locks (e.g., threading.Lock in Python) can be used. Before performing any operation on the stack or queue (e.g., push, pop, enqueue, dequeue), a thread must acquire the lock. Once the operation is complete, the thread releases the lock, allowing other threads to access the data structure.

Example (Python):

import threading

class ThreadSafeStack:
    def __init__(self):
        self.stack = []
        self.lock = threading.Lock()

    def push(self, item):
        with self.lock:
            self.stack.append(item)

    def pop(self):
        with self.lock:
            if not self.is_empty():
                return self.stack.pop()
            else:
                return None

    def is_empty(self):
        with self.lock:
            return len(self.stack) == 0

    def size(self):
        with self.lock:
            return len(self.stack)