ARM Assembly Part 21: ARM Microarchitecture Deep Dive

Pipeline Overview

Out-of-Order Execution

Register Renaming

The architectural register file (X0–X30) is far too small to hold the inflight work of a wide OOO core. A physical register file (PRF) of 128–256 entries is allocated dynamically. The rename stage maps each destination register write to a fresh PRF entry, eliminating WAR (Write-After-Read) and WAW (Write-After-Write) hazards while preserving true RAW (Read-After-Write) data dependences.

// Visible code — appears serial:
MUL  X1, X2, X3    // WAW: both write X1 ... but after renaming →
ADD  X1, X4, X5    //   MUL  → PRF[p47], ADD → PRF[p48]  (no conflict)

// RAW hazard (cannot rename away):
SDIV X0, X6, X7    // Long latency (≥12 cycles on A78)
ADD  X8, X0, X1    // Must wait for SDIV result in PRF[p47]

Reorder Buffer (ROB)

Instructions are inserted into the ROB in program order at dispatch. They can execute out of order — the ROB records which have completed — but they commit (become visible state) only in program order from the head of the ROB. This is what makes precise exceptions possible: any instruction that faulted can be identified because nothing past it has committed.

Reservation Stations & Execution Ports

// Cortex-A78 issue queues (approximate per ARM TRM):
// IQ0 (integer ALU, shift, CSEL): 4 entries × 2 pipes → 2 ALU/cycle
// IQ1 (multiply, divide):          2 entries × 1 pipe → 1 MUL/cycle
// IQ2 (load):                      12 entries × 2 pipes → 2 loads/cycle
// IQ3 (store):                     8 entries × 1 pipe → 1 store/cycle
// IQ4 (NEON/FP):                   8 entries × 2 pipes → 2 NEON/cycle

// Throughput-limiting example: 4 dependent multiplies, 1 cycle each
MUL  X4, X0, X1    // Uses MUL pipe (1 cycle latency approx)
MUL  X5, X2, X3    // Independent → can dual-issue if separate src regs
MUL  X6, X4, X5    // Depends on X4 and X5 → must wait → serialised
MUL  X7, X6, X3    // Depends on X6 → more serialisation

// Solution: expose more independent chains (see Part 18, multiple accumulators)

Branch Prediction

Bimodal & Two-Level Predictors

A bimodal predictor is a table of 2-bit saturating counters indexed by PC bits. Two-level (correlated) predictors index the counter table with a combination of the PC and the Global History Register (GHR), a shift register of the last N branch outcomes. The GHR captures correlation patterns like "this branch is always taken if the last three were also taken."

// Branch that aliases in a bimodal predictor (same PC bits, different ctx):
// Two calls to same function with different condition chains → predictor thrashes
// Solution: pad hot inner loops with NOPs or use PRFM for instruction cache

TAGE Predictor

Tagged GEometric history length (TAGE) uses a base bimodal table plus K tagged tables indexed by hash(PC, GHR[0:2^k−1]). The longest history that hits provides the prediction. A "use-count" meta-predictor arbitrates ties. Cortex-A78 and Neoverse N2 implement variants of TAGE achieving >98% accuracy on SPECint workloads.

Branch Target Buffer (BTB) & Return Address Stack (RAS)

// BTB Miss: indirect branch (BR Xn) where target changes frequently
// Triggering BTB miss costs ~20 cycles pipeline flush on A78
// Perf event: 0x010 = BR_MIS_PRED
mrs x0, pmcr_el0
// ... configure PMU event 0x010 as in Part 18 ...

// Indirect call via function pointer (high BTB miss rate if target varies):
BLR X9        // BTB must predict X9 content — if wrong: flush + refetch

// Static indirect branches (e.g. switch jump table): BTB usually correct after warmup
// Dynamic virtual dispatch: 1 call site → many targets → BTB capacity exhausted

// RAS (Return Address Stack, ~16–32 deep):
BL  func      // Pushes PC+4 onto RAS
...
func:
    ...
    RET       // Pops RAS → correct most of the time (mismatch on longjmp/setjmp)

Memory Subsystem

Store-to-Load Forwarding

A store writes to the store buffer (not directly to cache). A subsequent load to the same address is fulfilled from the store buffer — this is store-to-load forwarding. It avoids the cache latency when the data is still "in flight." However, partial overlaps (e.g., write 8 bytes, read 4 bytes misaligned) cause a forwarding bubble of ~4–8 extra cycles.

// Perfect forwarding: same address, same size → data from store buffer
STR  X0, [X1]      // Write X0 to [X1]
LDR  X2, [X1]      // Forwarded from store buffer: ~4 cycle latency vs ~12 cache hit

// Forwarding failure: partial overlap
STR  W0, [X1]      // Write 4 bytes to [X1]
LDR  X2, [X1]      // Read 8 bytes from [X1] — only 4 overlap → forwarding stall

// Common kernel bug: write byte, read word (struct alignment pitfall)
STRB W0, [X1]      // 1-byte store
LDR  W2,  [X1]     // 4-byte load → partial overlap → ~10 cycle penalty on A78

Memory Dependence Speculation

ARM OOO cores speculate that loads do not alias most stores. If a load is issued early and a later-dispatched store has the same address, the core detects the violation at commit time, flushes the load (and everything dispatched after it), and re-executes. The PMU event 0x074 = MEM_ACCESS_LD and the perf stat metric memory_bound reveal when this is expensive.

Cache & TLB Architecture

// Query cache parameters at runtime (ARM CTR_EL0):
// CTR_EL0[19:16] = L1 instruction cache line size (log2 words)
// CTR_EL0[3:0]   = L1 data cache line size (log2 words)
mrs  x0, ctr_el0
ubfx x1, x0, #16, #4   // Extract ICache line size field
mov  x2, #4
lsl  x3, x2, x1         // Cache line = 4 << field (bytes)
// On A78: CTR_EL0 = 0x84448004 → L1D line = 64 bytes, L1I line = 64 bytes

Prefetchers

// Hardware prefetcher types (all implicit, cannot be disabled via ISA):
// 1. Next-line: always fetch the next cache line after a miss
// 2. Stride: detect constant-stride access patterns
// 3. Stream: detect sequential streams, issue ahead-of-time prefetch

// Software prefetch (explicit, from Part 18):
PRFM PLDL1KEEP, [X0, #256]   // Prefetch 4 cache lines ahead into L1D
PRFM PLDL2KEEP, [X0, #512]   // Prefetch into L2 (for long loops)

// PRFM hint types:
// PLDL1KEEP  = load, L1, keep (non-evict hint)
// PSTL1KEEP  = store, L1, keep — prefetch for write (allocate clean line)
// PLDL3STRM  = load, L3, stream (evict-soon hint — scratchpad pattern)

TLB Structure & Shootdown

// TLB Miss: triggers full page table walk (50–200 cycles!)
// TLBI (TLB Invalidate) instructions:
TLBI  VMALLE1IS      // Invalidate all EL1 translations (inner-shareable)
TLBI  VAE1IS, X0     // Invalidate EL1 by virtual address (X0 >> 12 = VA/4KB)
TLBI  ASIDE1IS, X0   // Invalidate all entries with ASID in X0[63:48]
DSB   ISH            // Ensure TLB invalidation visible to all observers
ISB                  // Flush pipeline after TLB change

// TLB shootdown on SMP: each core running threads of the same process
// must invalidate TLBs when a mapping changes. IS (Inner-Shareable) suffix
// broadcasts the TLBI to all cores in the same inner-shareable domain.

Core Comparison

Assembly Implications

// ── Rule 1: Break dependence chains for ILP ──
// Bad: chain of 4 multiplies → 1 result/4 cycles
MUL  X0, X1, X2
MUL  X0, X0, X3
MUL  X0, X0, X4
MUL  X0, X0, X5

// Good: 4 independent multiplies → 4 results/1 cycle (wide issue)
MUL  X0, X1, X2
MUL  X6, X3, X4
MUL  X7, X5, X8
MUL  X9, X10, X11
// Then merge: MUL X12, X0, X6 / MUL X13, X7, X9 / MUL X0, X12, X13

// ── Rule 2: Avoid indirect branch thrash → prefer direct branches ──
// VTable dispatch: BLR Xn — poor BTB predictions when targets vary
// Alternative: inline or devirtualise in hot paths

// ── Rule 3: Align hot loop head to I-Cache line boundary ──
.balign 64           // 64-byte = L1I cache line on A78
.loop_head:
    LDP  X0, X1, [X2], #16
    LDP  X3, X4, [X2], #16
    // ... loop body ...
    B.NE .loop_head

// ── Rule 4: Separate store and load to same address ──
STR  X0, [X1]
// Insert ~4 independent instructions here to allow forwarding pipeline
ADD  X5, X6, X7    // Fill cycle 1
ADD  X8, X9, X10   // Fill cycle 2
ADD  X11, X12, X13 // Fill cycle 3 (A78 forwarding latency ≈ 4)
LDR  X2, [X1]      // Now forwarding hits clean without stall

Case Study: Apple M1 vs Cortex-X2 — Two Takes on Wide OOO

Hands-On Exercises

Conclusion & Next Steps

The Cortex-A78 represents the intersection of every concept in the series: ISA constraints shape what rename can do, weak memory ordering emerges from the store buffer design, and cache line boundaries determine when PRFM helps. Every assembly optimization from Part 18 maps to a physical circuit now visible in this part. The Apple M1 vs Cortex-X2 comparison shows how system-level constraints drive wildly different microarchitectural choices from the same ISA, and the exercises let you probe these mechanisms empirically by measuring ROB depth, branch predictor accuracy, and forwarding latency on real hardware.