Memory Vulnerabilities
Buffer Overflow
Classic Vulnerability: Writing beyond buffer bounds can overwrite adjacent memory, including return addresses, allowing attackers to hijack control flow.
; Vulnerable code pattern
section .bss
buffer resb 64 ; 64-byte buffer
section .text
vulnerable_func:
; No bounds checking!
lea rdi, [buffer]
call gets ; NEVER use gets()
ret ; May jump to attacker-controlled address
Stack Smashing Attack
A stack smashing attack overwrites the return address to hijack control flow. Let's visualize exactly what happens:
Stack Layout During Function Call:
High Memory
┌───────────────────┐
│ Return Address │ ← Attacker's target!
├───────────────────┤ Overwrite this to redirect execution
│ Saved RBP │
├───────────────────┤
│ Local Buffer[63] │
│ ... │ Buffer overflow direction ↑
│ Local Buffer[0] │ ← Input starts here, overflows upward
└───────────────────┘
Low Memory (RSP)
Overflow Attack:
1. Attacker inputs 64+ bytes via gets()
2. Input overflows buffer, overwrites saved RBP
3. Keeps writing, overwrites return address
4. When function returns, RIP = attacker-controlled!
; Exploitable function
vulnerable:
push rbp
mov rbp, rsp
sub rsp, 64 ; 64-byte buffer
lea rdi, [rbp-64] ; Buffer address
call gets ; Reads unlimited input!
leave ; mov rsp, rbp; pop rbp
ret ; pop rip - DANGER: may be overwritten!
; Attack payload (conceptual)
; [64 bytes padding][8 bytes fake RBP][8 bytes shellcode_addr]
Classic Attack Flow: Attacker sends 80 bytes: 64 bytes trash + 8 bytes fake RBP + 8 bytes pointing to shellcode. When ret executes, the CPU jumps to shellcode instead of the real caller.
Protection Mechanisms
Stack Canaries
; GCC -fstack-protector inserts canary
function_with_canary:
push rbp
mov rbp, rsp
sub rsp, 80
; Canary placed between buffer and saved RBP
mov rax, fs:[0x28] ; Load canary from TLS
mov [rbp-8], rax ; Store on stack
; ... function body ...
; Check canary before return
mov rax, [rbp-8]
xor rax, fs:[0x28]
jnz .stack_smash_detect ; Abort if modified
leave
ret
.stack_smash_detect:
call __stack_chk_fail
Address Space Layout Randomization (ASLR)
ASLR randomizes where code and data are loaded in memory each time a program runs. This breaks exploits that hardcode addresses.
# Observe ASLR in action
$ cat /proc/sys/kernel/randomize_va_space
2 # 0=off, 1=stack/mmap, 2=full (heap too)
# Run program twice, check stack address
$ ./test_aslr
Stack address: 0x7fff2a3b4c80
$ ./test_aslr
Stack address: 0x7fffde123a50 # Different each time!
# Check library base addresses
$ ldd /bin/ls | head -2
linux-vdso.so.1 => (0x00007ffd12345000) # Randomized
libc.so.6 => /lib/x86_64.../libc.so.6 (0x00007f8abc123000)
| ASLR Level | Randomized Regions | Entropy (bits) |
|---|
| Level 1 | Stack, shared libraries, mmap | ~28 bits |
| Level 2 | + Heap (brk) | ~28 bits |
| PIE Binary | + Main executable | ~28 bits |
Bypassing ASLR: Attackers use information leaks (format string bugs, buffer over-reads) to discover actual addresses at runtime, then calculate offsets to known functions.
Data Execution Prevention (DEP/NX)
DEP (Windows) / NX bit (Linux) marks memory pages as non-executable. Even if an attacker gets shellcode into memory, the CPU refuses to execute it.
Page Table Entry NX Bit:
┌─────────────────────────────────────────────┐
Bit 63: NX (No eXecute)
0 = Page is executable (code)
1 = Page is NOT executable (data/stack)
└─────────────────────────────────────────────┘
W^X (Write XOR Execute) Policy:
- A page can be Writable OR Executable, never both
- Stack: RW- (read, write, no execute)
- Code: R-X (read, execute, no write)
- Heap: RW- (read, write, no execute)
# Check memory protections on a process
$ cat /proc/$(pidof victim)/maps
00400000-00401000 r-xp /victim # Code: executable
00600000-00601000 rw-p /victim # Data: writable, not exec
7ffffffde000-7ffffffff000 rw-p [stack] # Stack: not executable
# Attempt to execute shellcode on NX stack
$ ./shellcode_test
Segmentation fault (core dumped) # CPU refused!
Legacy Note: The NX bit requires CPU support (most CPUs since ~2004) and 64-bit mode (or PAE in 32-bit). Check with grep nx /proc/cpuinfo.
Return-Oriented Programming (ROP)
ROP bypasses NX by reusing existing executable code snippets called "gadgets." Instead of injecting shellcode, attackers chain together instruction sequences that end in ret.
What is a Gadget?
A gadget is any sequence of instructions ending in RET:
; Example gadgets found in libc:
pop rdi; ret ← Set RDI (first argument)
pop rsi; ret ← Set RSI (second argument)
pop rdx; ret ← Set RDX (third argument)
mov [rdi], rax; ret ← Write memory
xchg eax, esp; ret ← Stack pivot
Building a ROP Chain
Goal: Call execve("/bin/sh", NULL, NULL)
Stack Layout (attacker-controlled):
┌───────────────────────┐
│ pop rdi; ret │ RSP → First gadget
├───────────────────────┤
│ addr of "/bin/sh" │ ← Popped into RDI
├───────────────────────┤
│ pop rsi; ret │ Next gadget
├───────────────────────┤
│ 0 (NULL) │ ← Popped into RSI (argv)
├───────────────────────┤
│ pop rdx; ret │ Next gadget
├───────────────────────┤
│ 0 (NULL) │ ← Popped into RDX (envp)
├───────────────────────┤
│ syscall; ret │ Execute syscall 59 (execve)
└───────────────────────┘
Execution Flow:
1. ret pops "pop rdi; ret" address into RIP
2. pop rdi loads "/bin/sh" address into RDI
3. ret pops next gadget address...
4. Eventually syscall executes execve("/bin/sh", NULL, NULL)
5. Attacker gets shell!
Finding Gadgets
# Use ROPgadget tool
$ ROPgadget --binary /lib/x86_64-linux-gnu/libc.so.6 | grep "pop rdi"
0x000000000002155f : pop rdi ; ret
0x00000000000215a0 : pop rdi ; pop rbp ; ret
# Or use ropper
$ ropper -f /lib/x86_64-linux-gnu/libc.so.6 --search "pop rdi"
[INFO] Searching for gadgets: pop rdi
0x000000000002155f: pop rdi; ret;
Defense: Control Flow Integrity (CFI) and Shadow Stacks detect and prevent ROP by validating return addresses match legitimate call sites.
Secure Assembly Practices
Writing secure assembly requires constant vigilance. Here are essential practices to protect your code:
1. Always Bounds Check
; SAFE: Bounds-checked copy
safe_copy:
; RDI = dest, RSI = src, RDX = max_len, RCX = dest_size
cmp rdx, rcx
jbe .size_ok
mov rdx, rcx ; Clamp to buffer size
.size_ok:
; Now safe to copy RDX bytes
rep movsb
ret
2. Zero Sensitive Data
; Clear password buffer before freeing
zero_sensitive:
mov rdi, password_buffer
mov rcx, PASSWORD_LEN
xor eax, eax
rep stosb ; Fill with zeros
; Use memory barrier to prevent optimization away
mfence
ret
3. Constant-Time Comparisons
; Timing-safe comparison (for passwords, tokens)
; Prevents timing attacks that measure comparison duration
constant_time_compare:
; RDI = buf1, RSI = buf2, RDX = length
; Returns: RAX = 0 if equal, non-zero if different
xor eax, eax ; Result accumulator
test rdx, rdx
jz .done
.loop:
mov cl, [rdi]
xor cl, [rsi] ; XOR bytes (0 if equal)
or al, cl ; Accumulate differences
inc rdi
inc rsi
dec rdx
jnz .loop
.done:
ret ; AL=0 only if all bytes matched
4. Stack Canary Manual Implementation
; Manual canary for functions with buffers
section .data
canary dq 0xDEADBEEFCAFEBABE ; Better: random at startup
section .text
protected_func:
push rbp
mov rbp, rsp
sub rsp, 80
; Place canary
mov rax, [rel canary]
mov [rbp-8], rax
; ... function body with buffer at [rbp-72] ...
; Verify canary
mov rax, [rbp-8]
cmp rax, [rel canary]
jne .abort ; Corruption detected!
leave
ret
.abort:
mov rdi, 1
mov rax, 60 ; exit(1)
syscall
Security Checklist
| Practice | Prevents | Implementation |
|---|
| Bounds checking | Buffer overflow | Compare index/length before access |
| Zero after use | Memory disclosure | rep stosb + memory barrier |
| Constant-time ops | Timing attacks | XOR + OR accumulator pattern |
| Validate pointers | NULL deref, UAF | Check against NULL, valid ranges |
| Minimize privileges | Escalation | Drop caps after initialization |
| Stack canaries | Stack smashing | Compile with -fstack-protector-strong |
Exercise
Security Audit Challenge
Find the vulnerabilities in this code:
get_user_input:
sub rsp, 64
mov rdi, rsp
call gets ; Vulnerability 1: ?
mov rax, rsp
add rsp, 64
ret ; Vulnerability 2: ?
Answers: 1) Unbounded input via gets(), 2) Returns pointer to stack memory (use-after-free when caller accesses it)