CSCI 180 - Buffer Overflow, Integer Overflow, and Software Security

Overflow

Happens when too much stuff in too small a space

Order of Stack Frame

- local variables (normal)
- saved frame ptr (prev. ebp)
- function return address
- function arguments (reverse)

Arguments

Pushed onto the stack first in reversed order, always stored in ebp+8...

Return Address

i.e., old EIPushed onto the stack after arguments. When the function finishes and returns, it needs to know where to return to. The address of the next instruction after the function call is stored here, always stored in ebp+4

Local Variables

Pushed onto the stack after SFP (old ebp), always stored in ebp-4....

Ebp Register

This register is used to reference all the function parameters and local variables in the current stack frame

Saved Frame Pointer (SFP)

There is only one ebp register which points to the stack frame of the current function. When there are a chain of function calls, and we enter a new function, a new stack frame is created. Ebp is going to point to this new stack frame, and we will lose the previous value(the base pointer for the caller function). We need a way to remember this value so that when we return from this function we can return to the stack frame of the previous function. We remember this value by storing the previous frame pointer in the stack

Extended Stack Pointer (ESP)

Shows us the top of the stack

Extended Instruction Pointer (EIP)

Holds the memory address of where in memory the next instruction is

Null Terminating Character

\0, compiler adds to the end of every string so the system knows where the end of the string is

strcpy()

Stops copying the string when \0 is found

NOP

Do nothing, go to the next instruction

How can the adversary take advantage of NOPs?

The malicious code can be placed at the end of the NOPs, so if the return address entered is one of the NOPs, we will continue executing to the next line until we reach our malicious code and can execute it. The idea is to keep hitting NOPs until we get to the first line of our malicious code

Buffer Overflow Defenses

Nonexecutable stacks, canaries, address randomization, and code analyzers

Nonexecutable Stacks

Used by some OSs to ensure that code on the stack cannot be executed, but attackers can still use heap overflow, change program variables, and use return to libc attacks

Canaries

Put a known value on the stack just before the return address. Goal is to protect the return pointer from being overwritten by a stack buffer. If a buffer overrun takes place and successfully overwrites the return address, then the canary will be overwritten with a new value as well

Bypassing Canaries

1. Find out what the canary is. Format string vulnerabilities and then overwrite it with itself

2. Write around the canary. Format string vulnerabilities

3. Overflow in the heap

Address Randomization

Rearranges the data of a process's address space at random, making it difficult to predict where to jump in order to execute code.

Code Analyzers

Since the attack comes from the program's source code, it makes sense to review the code for any security errors (code review, code scanners)

Integer Vulnerabilities

Arise because the programmer does not take into account the maximum and minimum values, including integer overflow, integer underflow, and truncation

Integer Overflow

Occurs at run-time when the result of an integer expression exceeds the maximum value for its respective type

Wraparound

When a value exceeds the maximum limit of a data type, causing it to "wrap around" to the opposite end of its range (e.g., exceeding the maximum value of an unsigned integer causes it to wrap around to 0).

Result Saturation

Wrap around does not occur, instead a MAXVALUE is always returned

Interpreting a bitstring as a _____________ integer can yield a value substantially different than interpreting it as an unsigned integer.

signed

Integer Underflow

Occurs at run-time when the result of an integer expression is smaller than its minimum value, thus "wrapping" to the maximum integer for the type
(e.g., Subtracting 0 − 1 and storing the result in an unsigned 16-bit integer will result in a value of 216 − 1, not−1)

Truncation

Assigning an integer with a larger width to a smaller width

Integer Bugs

Differ from other classes of exploits because they are usually exploited indirectly

Integer Vulnerability

Results in insufficient memory allocation, which is subsequently exploited by buffer overflows, heap overflows, overwrite attacks, etc.

Denial of Service (DoS) Attacks

Possible where the exploit causes excessive memory allocation or infinite loops

Array Index Attacks

Where a vulnerable integer is used as an array index, so that attackers can accurately overwrite arbitrary byte in memory

Integer Overflow Defenses

Input validation, compilation or build hardening, and libraries or frameworks

Input Validation

Ensure that numeric inputs are within the expected range. Enforce that the input meets both the minimum and maximum requirements for the expected range. Understand the programming language's underlying representation and how it interacts with numeric calculation

Compilation or Build Hardening

Examine compiler warnings closely and eliminate problems with potential security implications, such as signed / unsigned mismatch in memory operations, or use of uninitialized variables

Libraries or Frameworks

Use a vetted library or framework that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid

Off-by-One Error

Occurs when a program incorrectly processes one element too many or too few, often due to incorrect boundary conditions in loops or indexing

How can we exploit the off-by-one error?

The attacker can alter the least significant bit of the function's frame pointer, which will cause it to point to that new address, which will become ebp. Return address is always ebp+4, which can be altered to hold the address of our malicious code, which will be run after the function returns

Return to Libc

There is a region in the memory that contains the standard C library functions, in Linux the library is called libc. Since most programs use the functions inside the libc library, before these programs start running, the operating system will load the libc library into memory

Basic Planning of Return to Libc

Use external libraries loaded at runtime (e.g. libc) instead of injecting shellcode. Determine address of system() and jump to this address and ask it to run the"/bin/sh" program directly.

Return to Libc Steps

1. Find the address of system()

2. Find the address of "/bin/sh"

3. Pass the "/bin/sh" as an argument to system()

4. call exit() function to exit properly

For the Return to Libc Attack, why might the attacker want to call exit()?

The attacker may want to call exit so that no trace of the attack is left in the system

Return to Libc Exploit

1. fill up the vulnerable buffer up to the return address with garbage data

2. overwrite the return address with the address of system()

3. follow system() with the address of exit()

4. append the address of "/bin/sh"

Order for Return to Libc in Stack

system() address (low mem)
exit() address
/bin/sh address (high mem)

Weaknesses that Commonly Lead to Exploited Vulnerabilities

1. Anything that results in memory corruption (buffer overflows, string format errors)

2. Anything that causes an exception, and thus triggers an exception handler (attackers can replace exception handlers with their exploit code)

3. Anything that results in crashes

Mitigating Memory-Safety Vulnerabilities

Secure coding practices, memory-safe libraries/languages, add mitigations, code analysis/review, testing

Secure-Coding Practices

Add checks in code, use library functions that check bounds, perform code reviews

Add Mitigations

Stack canaries, non-executable stacks, ASLR (Address space layout randomization), and turn on compiler warnings

Code Analysis/Review

It attempts to predict whether there exist any issue that would cause failure at runtime, and then warns the programmer, and also detects bug as development time. At that time is less expensive to fix the bugs

Testing for Functionality

You want to make sure good things happen when you have normal inputs

Testing for Security

You want to make sure nothing bad happen when you have unusual circumstances

Fuzz Testing

Involves testing the program with random inputs and seeing if the program exhibits any sign of failure (crash, unexpected exception). Inputs can be random, structured, or mutated.

Other Approaches for System Security

Vulnerability scanning, penetration testing, bug bounty programs