System Security and Buffer Overflow

Page 3

  • Recap: Injection of Malicious Code

    • Generates compiler-hacking.o, uses to create login-hacked.o from
      login.c.

    • Understanding code verification processes.

Page 4

  • Buffer Overflow

    • Overview of system security thoughts:

    • Goals, threat model, mechanisms, policies.

    • Majority of mechanisms are software-based, which may have bugs.

    • Security bugs lead to vulnerabilities; buffer overflow is a prominent one.

Page 5

  • Buffer Overflow Context

    • Common in C/C++ due to being unsafe languages.

    • First exploit was the Morris Worm in 1988.

    • Continues to be relevant today.

    • Avoiding unsafe languages is suggested but not comprehensive; numerous changes in compilers and hardware architectures derive from it.

Page 6

  • Web Server Example

    • Threat Model: Any individual who can send messages to the server.

    • Attacker's Goals:

    • Crash server, inject code, leak, or modify data.

Page 7

  • Important Notes

    • Focus on Intel 32-bit architecture in Linux:

    • Little Endian, 4-byte addresses, 4GB virtual memory.

    • Use -m32 for 32-bit binaries on 64-bit machines.

Page 8

  • Stack Mechanism

    • Defines the process in memory with virtual memory management.

    • Each process has own stack, heap, and code/data segments.

    • Stack is managed by the OS/compiler and are not freely accessed by applications.

    • Registers:

    • $esp: Top of the stack address.

    • $ebp: Bottom of the stack address.

    • $eip: Current instruction address.

Page 9

  • Stack Mechanism

    • The stack consists of frames for each function:

    • Top stack frame holds function info, return address, previous frame pointer.

    • Stack grows to decreasing memory indices.

Page 10

  • Function Return in Stack

    • New stack pointer set to previous base pointer.

    • Pop saved base pointer and return address to new base pointer and instruction pointer.

Page 11

  • Invoking Function Example

    • int func(int a, int b) uses a stack frame for local variables and return addresses.

    • Understand position within accounts and variables in call stack.

Page 12

  • Returning from Function

    • See stack frames evolve as function returns and local variables are popped from the stack.

Page 13

  • Buffer Overflow Example

    • void vul(char *str):

    • Implements a vulnerable buffer with a limited size; using strcpy() inserts data beyond the intended limit.

Function Return in Stack

New stack pointer set to previous base pointer. This process involves:

  • Popping the saved base pointer into the new base pointer to restore the previous stack frame.

  • Popping the return address into the instruction pointer that directs the flow of execution back to the caller function.

Invoking Function Example

The function declaration int func(int a, int b) is a demonstration of how local variables and return addresses are managed. Within the call stack, every invocation of a function creates a new stack frame that holds:

  • Local variables.

  • Return addresses allowing the program to know where to return after the function execution.

Returning from Function

As functions return:

  • The stack frames evolve; local variables are removed, and the stack pointer adjusts to previous frames.

  • This popping mechanism is crucial for maintaining the integrity of the program's execution flow.

Buffer Overflow Example

The code snippet:

void vul(char *str) {
 char buffer[16];
 strcpy(buffer, str);
}


illustrates a vulnerability where buffer can overflow. The function uses strcpy, which does not check the length of the input string. If str exceeds 16 characters, it will overwrite adjacent memory, potentially influencing return addresses and thereby may grant control over the program flow, leading to application crashes or unauthorized behavior.

Page 14

  • What Happens on Buffer Overflow?

    • When str exceeds 16 characters, vulnerability can lead to control over program flow or application crash.

Page 15

  • Buffer Overflow Concepts

    • Classification: Writing exceeds buffer size, leading to crashes or overriding return addresses.

Page 16

  • Basic Exploit Mechanism

    • Return address point to a location on the stack which contains executable code, with buffer containing junk and injected code. When function return, jump to the injected code. Overflow buffer will have new return address, junk and injected code where injected code will start a shell. execute(‘/bin/sh’)

Page 17

  • Injected Code Execution

    • When returning from function, control jumps to shell initial code (execute('/bin/sh')).

Page 18

  • Crafting Buffer Overflow

    • Change the return address through memory manipulation, specifying the position of injected code.

    • Knowledge of addresses and offsets is essential.

Page 19

  • Example of Injected Shell Code

    • Code represented in hex for executing shell operations with int $0x80 and specific registers manipulation as detailed.

Page 20

  • Retrieving Object Code

    • Use of objdump for disassembly and verification of injected shell code structure.

Page 21

  • Crafting Exploit Methodology

    • Strategies involving setting the right return address and positioning of shellcode effectively in the buffer.

Page 22

  • Other Attack Vectors

    • Include return-oriented programming, heap overflow, string overflow, integer overflow.

Page 23

  • Return Oriented Programming (ROP)

    • Techniques involving stack buffer overflows without the need to inject code directly, using already loaded code to execute actions.

    • Stack buffer overflow

      • Change return address

      • Inject code to stack

      • Execute stack

      • Does not work when stack is not executable

  • Traditional Stack Buffer Overflow vs. ROP

    Traditional stack buffer overflows worked by:

    1. Overflowing a buffer on the stack

    2. Injecting malicious code into the stack

    3. Overwriting the return address to point to this injected code

    4. When the function returns, the malicious code executes

    This approach was mitigated by non-executable stack protections (NX/DEP), which prevent code on the stack from executing.

    ROP Technique

    Since code injection no longer works with these protections, ROP takes a different approach:

    1. Instead of injecting new code, ROP reuses existing code already loaded in memory

    2. It identifies small code sequences in existing libraries that end with a return instruction (called "gadgets")

    3. It chains these gadgets together by carefully constructing a fake stack

    4. Each gadget performs a small operation and then "returns" to the next gadget in the sequence, enabling the attacker to execute arbitrary code while bypassing the security mechanisms designed to prevent such attacks.

Page 24

  • Finding Code to Exploit

    • Using functions or code snippets already present in the memory to direct control to desired execution paths through manipulations.

Page 25

  • Executing Shell from libc


    • Manipulating return address to redirect flow to system() function from libc with arguments properly set in the stack.

    • Basic ROP with libc

      The first image shows a straightforward ROP attack targeting the libc library:

      1. Find the memory address of the system() function in libc

      2. Change the return address in the stack to point to this system() function

      3. Add the string "/bin/bash" to the top of the stack as an argument

      4. When the vulnerable function returns, it will execute system("/bin/bash"), spawning a shell

      The diagram shows a vulnerable function process_msg() with a buffer that can be overflowed, and how the stack is manipulated to execute code already present in memory.

Page 26

  • Constructing Arbitrary Programs with ROP

    • If no libc() - C standard library or system()

      • Construct arbitrary program, with innocent pieces of codes

        If no system(), ROP will generate it.

Page 27

  • Return-Oriented Programming Basics

    • Identification of useful instruction sequences ('gadgets') that already exist in the code, allowing complex operations to be performed.

Page 28

  • ROP Example and Stitching Gadgets

    • illustrating how gadgets are placed sequentially in the payload for execution.

Page 29

  • Constructing the Stack for ROP

    • Mapping addresses of gadgets and setup needed stack alignment for correct ROP operation flow.

Page 30

  • Further Addressing and Stack Construction

    • Completing the stack setup to ensure gadgets fall within correct sequence for execution during exploitation.

Page 31

  • Final Layout of Stack in ROP

    • Explanation of finalized stack set up post-gadget injection.

Page 32

  • Additional Attack Vectors Recap

    • Return Oriented Programming, Heap Overflow, String Overflow, Integer Overflow reiterated as key focuses.

Page 33

  • Heap Overview in Memory

    • Details on how heap memory grows and inspection through tools like GDB.

    • Heap Memory Growth: The heap memory typically grows dynamically as requested by applications, allowing for flexible memory allocation.

    • Inspection Tools: Tools like GDB (GNU Debugger) can be utilized to examine the heap's state, monitor allocations, and detect potential vulnerabilities.

Page 34

  • Heap Operations

    • Memory management calls to allocate (malloc used to allocate memory) and deallocate (free) with discussing overwriting potential memory addresses.

    • char *buf = malloc(4) allocates 4 bytes of memory. 4 bytes x 8 bits = 32 bits

    • free() = Deallocate memory. free(buf) deallocated the above

Page 35

  • Heap Data Layout

    • Memory structure of allocated chunks, with focus on pointers and metadata relating to memory management.

      Heap Data Layout (Chunks in use)

      This image shows the structure of allocated memory chunks currently in use:

      • Each chunk begins with metadata including:

        • Information about the previous chunk (size if free, or user data)

        • The current chunk's size

        • Flag bits (N, M, P) for memory management

Page 36

  • Heap Data Layout for Unused Chunks


    • Description of free chunk pointers and their configurations within heap management.

      Heap Data Layout (Chunks not in use)

      When a chunk is freed, its user data area is repurposed:

      • Forward and backward pointers are stored where user data used to be

      • These pointers maintain a circular doubly-linked list of free chunks

      • The area marked with an X shows that user data no longer exists in this space

Page 37

  • State Before Free Operation

    • Memory structure representation before calling free().

    • Three buffers have been allocated and filled:

      • buf1 with 'A' characters (0x41414141)

      • buf2 with 'B' characters (0x42424242)

      • buf3 with 'C' characters (0x43434343)

    • The memory headers for each buffer include:

      • Size information for the previous chunk

      • Size of the current chunk (highlighted as "Size of buf1 chunk")

      • The actual user data (highlighted as "Start of buf1 user data")

Page 38

  • After Free Operation

    • Show dynamic changes in chunk structures after freeing a chunk.

    • When buf2 is freed, its memory is repurposed:

      • Forward and backward pointers (marked as "Fwd Bwd pointers") are inserted into the space that previously held user data

      • These pointers (0xf7f727f8) connect this chunk to the free list data structure

    • The metadata is also updated:

      • "Previous Size updated since buf2 is free now" highlights how size flags get updated in adjacent chunks (buf3 in this case)

When a buffer is freed, the memory isn't actually cleared - instead, it's repurposed. This allows for potential buffer overflow vulnerabilities, as malicious actors can overwrite the data in these repurposed memory areas, exploiting the system's trust in previously allocated memory.


  • When you call free(buf), the memory allocator takes that chunk of memory and adds it to its "free list" (a list of available memory chunks).

  • To maintain this list, the allocator overwrites the user data area of the freed buffer with forward and backward pointers. These pointers connect this chunk to other free chunks in a doubly-linked list structure.

    An attacker can manipulate these pointers, they may be able to redirect execution flow or corrupt memory, leading to unauthorized access of program or crashes.

Page 39

  • Example with GDB

    • Practical context of inspecting memory addresses using debugger; aids in understanding overflow implications.

Page 40

  • Free Operation Exploit

    • Tie into attack possibilities if memory address structures are corrupted.

Page 41

  • Heap Overflow Exploit Details

    Heap overflow happens when you write more data into a heap-allocated buffer (memory from malloc()) than it can hold.

    When you overflow into the metadata (the management data the system uses for heap chunks), you can corrupt it.

    • Representing how overflowed chunks can be manipulated to enable black-box approach within memory.

Page 42

  • Heap Chunk Overflow Mechanics

    • Carefully overwriting size fields to create links between arbitrary memory addresses, manipulating execution.

  • Before Overflow: Memory Layout

    | Metadata for p | Data for p (1024 bytes) |

    | Metadata for q | Data for q (1024 bytes) |

    • p and q are two separate chunks.

    • Each chunk has hidden metadata (like size, pointers) managed by the allocator.

  • When strcpy(p, input) Happens with Large Input, there is a risk of overwriting the memory allocated for chunk q, which can lead to a buffer overflow. This vulnerability allows an attacker to manipulate the allocator's metadata, potentially redirecting execution flow or corrupting data.

    Normal input:

    • Fits inside q → no issue.

    Malicious large input (more than 1024 bytes):

    • Writes into q's data.

    • Keeps writing past the end of q, into p’s metadata!

Page 43

  • Manipulation of Free Procedure

    • Exploiting memory's management to create undesired functionality or behavior through overflow.

Page 44

  • Other Attack Vectors Recap

    • Return Oriented Programming, Heap Overflow, String Overflow, Integer Overflow included.

Page 45

  • String Overflow Explanation

    • Discussing various parameter types and format specifics during function calls in C, illustrating common fail points.

Page 46

  • String Overflow Parameters

    • Detailed value outputs for common formats like %s and their implications in context.


Page 47

  • Reading Memory and Format Strings

    • Understanding how stack changes affect function inputs during string manipulation.

Page 48

  • Effects of Insufficient Arguments

    • Directed note towards observing how variances in input affect outputs in printf operations.

Page 49

  • Memory Writing via Formatting

    • Theoretical construct allowing output of byte counting to a variable using %n format in printf.

Page 50

  • Testing %n Formatting

    • Questions raised about reliability and typical behavior of %n placeholder in various contexts.

    • printf(“100%nuke”); Does not always work!

    • When does %n work?

      • You must pass a real address for %n to store the result.

      printf("100%nuke", &count);

      • Now %n will safely write 3 (the number of characters printed so far) into count

Page 51

  • Final Recap of Attack Vectors

    • Reiteration engaged with possible exploits including return-oriented programming to integer overflows.

Page 52

  • Integer Overflow Mechanics

    • Examination of valid ranges and overflow points in typical operations leading to unexpected behavior.

Page 53

  • Integer Ranges Overview

    • Detailed characterizations of types and their ranges within signed and unsigned contexts in C programming.

Page 54

  • Integer Representation Nuances

    • Binary representations discussed; how unsigned and signed types deviate markedly in behavior.

Page 55

  • Overflow Mechanics in Addition/Subtraction

    • Illustrate behaviors crossing signed bounds under arithmetic operations.

Page 56

  • Integer Division Behavior

    • Explain behaviors in division loops and the logic behind counting iterations directly.

      Operation

      Result

      Why

      (2³¹-1) × 2

      Negative

      Overflow into negative due to 32-bit limits

      7 / 3

      2

      2 times 3 fits into 7

      -7 / 3

      -2

      2 times 3 fits into -7 in negative direction

Page 57

  • Critical Integer Overflow Cases

    • Significant computation leading to overflow and unexpected outcomes for max-min ranges.

Page 58

  • Real-World Example of Integer Overflow

    • Discuss an actual case study in political vote counting misreporting due to overflow errors.

Page 59

  • Character Overflows Defined

    • Describe high likelihood scenarios and how reliance on inadequate types can lead to errors.

Page 60

  • Unsigned Type Iteration on Overflow

    • Explain behavior of unsigned integer loops and the infinite iteration effect.


  • When an unsigned integer underflows (goes below zero), it wraps around to its maximum possible value due to how unsigned integers are represented in binary. Unsigned integer CANNOT GO BELOW 0/negative so it goes back to maximum value which is 2^n - 1. This creates an infinite loop.

    Unsigned integers can only represent non-negative values from 0 up to (2^n - 1), where n is the number of bits in the integer type. For example:

    • An 8-bit unsigned integer can represent values from 0 to 255 (2^8 - 1)

    • A 32-bit size_t can represent values from 0 to 4,294,967,295 (2^32 - 1)

Page 61

  • Security Principles

    • Emphasize designing with memory safety, optimizing TCB, least privilege, and various other fundamentals.

Page 62

  • Further Security Principles

    • Defense and detection mechanisms, along with fail-safes and ensuring robust threat modeling.

Page 63

  • Minimizing TCB for Security

    • TCB gives insight into how to create secure environments with emphasis on simplicity and effectiveness.

Page 64

  • Closing Questions?

    • Summary and closure to the lecture with a prompt for clarity or uncertainties.


Topic: Buffer Overflow and Security Principles

Instructor: Daisuke Mashima
Course: SUTD ISTD 50.044 System Security

Announcements

  • Tuesday sessions will be held in LEET Lab.

  • Today's TA discussion will focus on lab setup starting at 2 PM.

  • Students are encouraged to form groups for Capture the Flag (CTF) activities, consisting of 3-4 members each.

Recap: Injection of Malicious Code

  • Review of the process to generate the compiler-hacking.o, which is pivotal for creating login-hacked.o from login.c.

  • Emphasis on understanding the critical code verification processes necessary to safeguard against such injections.

Buffer Overflow

Overview of System Security Thoughts

  • Goals: Establish robust security measures that can anticipate and mitigate anticipated threats.

  • Threat Model: An applicative outlook on potential attacks targeting the system, focusing on vulnerabilities and weaknesses in the architecture.

  • Mechanisms: The various strategies employed to secure systems, mostly software-based, are examined—acknowledging that most can contain imperfections.

  • Policies: Implementation of organizational policies guiding security practices and incident response plans.

  • A significant concern is that security bugs may lead to serious vulnerabilities, with buffer overflow being a prominent example.

Buffer Overflow Context

  • Buffer overflow vulnerabilities are particularly prevalent in C/C++ languages due to their lack of inherent safety checks regarding memory bounds.

  • Historical context: The first documented exploit was the Morris Worm in 1988, marking a significant moment in cybersecurity awareness.

  • Current relevance: Buffer overflow attacks continue to pose a threat due to persistent coding practices and insufficient compiler protections within both legacy and new systems.

  • Recommendations include avoiding unsafe languages and updating compilers and hardware architectures to address such vulnerabilities.

Web Server Example

Threat Model

  • The model posits that any user capable of sending messages to a server can potentially exploit vulnerabilities directly.

  • Attacker's Goals:

    • Crash server: Aiming to use the overflow to cause denial of service, stopping legitimate access.

    • Inject code: Altering program behavior through unauthorized code execution.

    • Leak data: Extracting sensitive information improperly.

    • Modify data: Tampering with the data integrity of the server’s database or files.

Important Notes

  • Primarily focus on Intel's 32-bit architecture when working with Linux, characterized by:

    • Little Endian format, which indicates how byte values are stored.

    • 4-byte addresses capable of addressing up to 4GB of virtual memory.

  • Use the -m32 flag for generating 32-bit binaries when compiling on 64-bit architectures, ensuring compatibility with certain systems during execution.

Stack Mechanism

Memory Management in Processes

  • Each process is allocated its own stack, alongside heap and code/data segments, organizing memory efficiently for operations.

  • The stack is managed strictly by the OS/compiler, preventing direct access by applications, to ensure stability and security.

  • Key registers associated with stack management include:

    • $esp: Points to the top of the stack, marking the most recent point in use.

    • $ebp: Represents the base pointer, delineating the current stack frame.

    • $eip: Contains the current instruction address indicating which code is being executed next.

Buffer Overflow Example

  • Function declaration:

void vul(char *str) { 
    char buffer[16]; 
    strcpy(buffer, str); 
}

  • The implementation of the vulnerable buffer demonstrated here is susceptible to overflowing; if the input str exceeds 16 characters, it compromises adjacent memory, leading to potential control over program flow or even crashing the application, showcasing the dire consequences of insufficient validation.

Crafting Buffer Overflow

Memory Manipulation

  • Techniques for changing the return address through crafted inputs emphasize the necessity of understanding memory layout and address offset calculations.

  • Knowledge regarding immediate addresses and their offsets is critical when developing effective exploits for buffer overflows, as the correct manipulation of these leads to successful control over program execution flow.


What are the attacks?

  1. Return oriented programming

  2. Heap Overflow

  3. String overflow

  4. Integer overflow