x86 Assembly Series Part 2: x86 CPU Architecture Overview

CISC Philosophy

CISC vs RISC

Design Implications

CISC architecture profoundly affects how you write and think about assembly:

Memory Operands Anywhere

Unlike RISC (load/store), x86 allows memory operands directly in arithmetic:

; CISC style - memory in arithmetic
add [count], 5          ; Add 5 directly to memory location
mul dword [factor]      ; Multiply EAX by memory value

; RISC equivalent would need:
ldr r1, [count]         ; Load
add r1, r1, #5          ; Compute
str r1, [count]         ; Store

Rich Instruction Set

; String operations
rep movsb               ; Copy RCX bytes from RSI to RDI
rep stosq               ; Fill RCX quadwords at RDI with RAX
repnz scasb             ; Search for AL in string at RDI

; Complex addressing
mov eax, [rbx + rsi*4 + 16]   ; Array[i] with base offset

; Atomic operations  
lock cmpxchg [mutex], ecx     ; Compare-and-swap for synchronization
lock xadd [counter], eax      ; Atomic fetch-and-add

CPU Execution Modes

graph TD
    RM["Real Mode
16-bit | 1 MB Address Space
No Protection | Direct HW Access"]
    PM["Protected Mode
32-bit | 4 GB Address Space
Ring Protection | Paging"]
    LM["Long Mode (x86-64)
64-bit | 256 TB Virtual
4-Level Paging | RIP-relative"]
    VM["Virtual 8086 Mode
Real Mode emulation
inside Protected Mode"]

    RM -->|"Set PE bit in CR0"| PM
    PM -->|"Set LME in EFER + PG"| LM
    PM -->|"Set VM flag"| VM
    VM -->|"Clear VM flag"| PM

    style RM fill:#fff5f5,stroke:#BF092F
    style PM fill:#f0f4f8,stroke:#16476A
    style LM fill:#e8f4f4,stroke:#3B9797
    style VM fill:#f8f9fa,stroke:#666
                        

Real Mode

Protected Mode

Introduced with the 80386, Protected Mode is where 32-bit operating systems live:

Global Descriptor Table (GDT)

The GDT defines memory segments with protection attributes:

GDT Entry (8 bytes each):
┌──────────┬─────────┬─────────┬──────────┬──────────┐
│ Base[24:31] │ Flags     │ Access    │ Base[16:23] │ Base[0:15]  │
│ Limit[16:19]│ (G,D,L,0) │ (P,DPL,S..)│             │ Limit[0:15] │
└──────────┴─────────┴─────────┴──────────┴──────────┘

Typical GDT layout:
  Index 0: Null descriptor (required)
  Index 1: Kernel Code (Ring 0, Execute)
  Index 2: Kernel Data (Ring 0, Read/Write)
  Index 3: User Code   (Ring 3, Execute)
  Index 4: User Data   (Ring 3, Read/Write)
  Index 5: TSS         (Task State Segment)

Entering Protected Mode (Bootloader Pattern)

; Minimal GDT for entering protected mode
gdt_start:
    dq 0                        ; Null descriptor (index 0)
gdt_code:                       ; Code segment descriptor (index 1)
    dw 0xFFFF                   ; Limit 0-15
    dw 0                        ; Base 0-15
    db 0                        ; Base 16-23
    db 10011010b                ; Access: Present, Ring 0, Code, Readable
    db 11001111b                ; Flags: 4K granularity, 32-bit
    db 0                        ; Base 24-31
gdt_data:                       ; Data segment descriptor (index 2)
    dw 0xFFFF
    dw 0
    db 0
    db 10010010b                ; Access: Present, Ring 0, Data, Writable
    db 11001111b
    db 0
gdt_end:

gdt_descriptor:
    dw gdt_end - gdt_start - 1  ; GDT size - 1
    dd gdt_start                ; GDT address

; Switch to protected mode
enter_protected:
    cli                         ; Disable interrupts
    lgdt [gdt_descriptor]       ; Load GDT
    mov eax, cr0
    or eax, 1                   ; Set PE (Protection Enable) bit
    mov cr0, eax
    jmp 0x08:protected_start    ; Far jump to flush pipeline, load CS

Long Mode (64-bit)

Long Mode is the native mode for 64-bit x86 processors. You must transition through Protected Mode to reach it:

Entering Long Mode (From Protected Mode)

; Prerequisites:
; 1. Already in Protected Mode with paging disabled
; 2. PAE (Physical Address Extension) enabled
; 3. 4-level page tables set up

enter_long_mode:
    ; Enable PAE in CR4
    mov eax, cr4
    or eax, (1 << 5)            ; Set PAE bit
    mov cr4, eax
    
    ; Load PML4 table address into CR3
    mov eax, pml4_table         ; Page-Map Level-4 Table
    mov cr3, eax
    
    ; Enable Long Mode in EFER MSR
    mov ecx, 0xC0000080         ; EFER MSR number
    rdmsr
    or eax, (1 << 8)            ; Set LME (Long Mode Enable)
    wrmsr
    
    ; Enable paging (this activates Long Mode)
    mov eax, cr0
    or eax, (1 << 31)           ; Set PG (Paging) bit
    mov cr0, eax
    
    ; Far jump to 64-bit code segment
    jmp 0x08:long_mode_start

[bits 64]
long_mode_start:
    ; Now in 64-bit mode!
    mov rsp, stack_top
    call kernel_main

64-bit Addressing

Privilege Rings

The Ring Model (0-3)

graph TD
    R0["Ring 0 — Kernel
Full hardware access
All instructions allowed"]
    R1["Ring 1 — Device Drivers
(Rarely used in modern OS)"]
    R2["Ring 2 — Device Drivers
(Rarely used in modern OS)"]
    R3["Ring 3 — User Applications
Restricted access
Must use syscalls for I/O"]

    R0 --- R1
    R1 --- R2
    R2 --- R3

    R3 -->|"INT 0x80 / SYSCALL"| R0
    R0 -->|"IRET / SYSRET"| R3

    style R0 fill:#BF092F,stroke:#132440,color:#fff
    style R1 fill:#16476A,stroke:#132440,color:#fff
    style R2 fill:#3B9797,stroke:#132440,color:#fff
    style R3 fill:#e8f4f4,stroke:#3B9797
                        

Ring Transitions

Code transitions between privilege levels through controlled gates:

User to Kernel (Ring 3 → Ring 0)

; Modern syscall instruction (64-bit Linux)
; Arguments: RDI, RSI, RDX, R10, R8, R9
; Syscall number: RAX
; Return value: RAX

mov rax, 1              ; sys_write
mov rdi, 1              ; fd = stdout
mov rsi, message        ; buffer
mov rdx, 13             ; length
syscall                 ; RING 3 → RING 0 → RING 3

; What happens:
; 1. CPU saves RIP to RCX, RFLAGS to R11
; 2. Loads kernel CS:RIP from STAR/LSTAR MSRs
; 3. Sets CPL to 0 (kernel mode)
; 4. Kernel handler executes
; 5. sysret instruction returns to user mode

Transition Mechanisms

# Count syscalls made by a program
strace -c ls /

# Example output:
# % time     calls     syscall
# 67.50%       101     read
# 12.00%        43     write
#  5.00%        25     openat
#  ...each is a Ring 3 → 0 → 3 transition!

CPU Internals

Instruction Pipeline

Modern CPUs process instructions through a multi-stage pipeline to increase throughput:

Classic 5-Stage Pipeline:

  ┌───────┐   ┌───────┐   ┌───────┐   ┌───────┐   ┌───────┐
  │ Fetch │ → │Decode │ → │Execute│ → │Memory │ → │Write- │
  │       │   │       │   │       │   │ Access│   │ back  │
  └───────┘   └───────┘   └───────┘   └───────┘   └───────┘
  
  Clock  1    2    3    4    5    6    7    8    9
  Inst1  IF   ID   EX   MEM  WB
  Inst2       IF   ID   EX   MEM  WB
  Inst3            IF   ID   EX   MEM  WB
  Inst4                 IF   ID   EX   MEM  WB
  
  Throughput: 1 instruction per cycle (ideally)

Pipeline Stages Explained

Instruction Decoder

x86's variable-length instructions (1-15 bytes) create a decoding challenge:

Instruction Length Examples:

nop                          ; 90                    (1 byte)
ret                          ; C3                    (1 byte)
mov eax, 1                   ; B8 01 00 00 00        (5 bytes)
mov rax, 1                   ; 48 B8 01 00 00 00 00 00 00 00  (10 bytes)
lock cmpxchg [rdi+rcx*8], rax ; F0 48 0F B1 04 CF   (6 bytes with prefix)

The decoder must:
1. Find instruction boundaries (no fixed size!)
2. Handle legacy prefixes (REX, VEX, EVEX)
3. Parse ModR/M and SIB bytes
4. Extract immediate values

Decoder Design

Modern Intel/AMD CPUs use multiple parallel decoders:

Intel Skylake Decoder:

┌──────────────────────────────────────────────┐
│          Instruction Fetch (16 bytes/cycle)  │
└──────────────────────────────────────────────┘
                      │
                      ▼
┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
│Decode│ │Decode│ │Decode│ │Decode│ │Complex│
│  1   │ │  2   │ │  3   │ │  4   │ │Decode │
│simple│ │simple│ │simple│ │simple│ │(MSROM)│
└──────┘ └──────┘ └──────┘ └──────┘ └───────┘
  ↓                                    ↓
 1 µop                              Multiple
 each                                 µops

Simple: Can be decoded to 1 micro-op (fused)
Complex: Needs microcode ROM lookup

Microcode Layer

Complex CISC instructions are internally translated to simple RISC-like operations called micro-operations (µops):

µop Translation Examples

; Simple instruction → 1 µop
add rax, rbx             ; 1 µop: add

; Memory operand → 2 µops (fused in some cases)
add rax, [rbx]           ; 2 µops: load + add (may fuse to 1)

; Memory-memory (read-modify-write) → 4 µops
add [count], 5           ; load + add + store-address + store-data

; Complex string operation → many µops
rep movsb                ; ~100+ µops for copying 100 bytes
                         ; (internally loops in microcode)

Why Microcode Matters

# Profile µops on Intel
perf stat -e uops_issued.any,uops_executed.thread ./program

# Sample output:
# 1,500,000,000   uops_issued.any
# 1,200,000,000   uops_executed.thread  
# 1.0 seconds elapsed

# More µops issued than executed = speculation wasted on mispredicts

Next Steps

With a solid understanding of CPU architecture, we'll now dive deep into the registers—the CPU's working memory for assembly programming.