Introduction + review of binary analysis

Why do you need to understand exactly how a system works?

Understanding exactly what rules it follows and how they are enforced allows you to understand the system better than the people that made it and find incorrect assumptions that can be exploited

Why do multi layered systems make exploitation easier?

People make assumptions about the layer below

Binaries are written in

assembly

What level of code are binaries written in?

Much lower level than Java byte code

Assembly compiled for one type of machine

won’t run on another

The x86-64 architecture

The stack

• Last in, first out

• Can read and write to the top of the stack

Sub parts of registers can be addressed

directly

x86 to x86-64 (x64) is a size difference of

32 to 64 bit (x64 is twice as long)

x86 to x86-64 (x64) registers have

a range of different names

In x86, function arguments are passed

onto the stack

Arguments in x64

• The first 6 are passed into registers

• The rest are passed onto the stack

Command: MOV

Move value between registers

Command: MOVxy

Move value of different lengths

Command: CALL

Execute a function

Command: RET, RETN, RETF

End a function and restart calling code

Command: CMP, TEST

Compare two values

Command: JE, JNE, JLE

Jump based on comparison

Command: LEA

Like MOV, but can evaluate arguments

Command: [ ]

Value at a memory location

MOV RAX RBX

Move value in RBX to RAX

MOV [RAX] RBX

Move value in RBX to address RAX points at

MOV RAX [RBX]

Move value RBX is pointing at to RAX

MOV [RAX] [RBX]

CPUs are physically unable to do this

Linux function calling conventions: Arguments

Put into registers:

• RDI, RSI, RDX, RCX, R8, R9, and then stack

• XMM0, XMM1, …, XMM7 for floating point numbers

Linux function calling conventions: Results

Return in RAX (and RDX) or XMM0 and XMM1

Linux function calling conventions: fow does the Function called makes new stack space?

Updating RSP and RPB, the instruction pointer and old RBP are stored on the stack for when the function returns

Windows function calling conventions: Registers

Arguments are in:

• RCX, RDX, R8, R9

• XMM0, XMM1, XMM2, XMM3

Patching

Changing values or commands in a binary

While patching, commands must be overwritten

in place - space cannot easily be added