ARM Assembly Part 28: Emerging ARMv9 & Future Directions

ARMv9 key feature changes vs ARMv8

ARMv9 processor releases

• Cortex-A510/A710/X2 (2022) — First ARMv9 CPU cores; Cortex-A510 implements SVE2 at 128-bit VL (vector length), power-efficient core.
• Cortex-A520/A720/X4 (2023) — ARMv9.2, adds MTE support; Cortex-X4 paired with Immortalis-G720 GPU in Snapdragon 8 Gen 3.
• Neoverse V2 (2022) — ARMv9 server core (AWS Graviton4, Google Axion); 256-bit SVE2, MTE, 4× DDR5 channels.
• Apple M-series — ARMv8.6+ (ARM64e); PAC everywhere, but not full ARMv9 SVE2 — Apple uses custom SIMD width strategies.
• Cortex-A725/X925 (2024) — ARMv9.2+, SME2 debut in mobile; Cortex-X925 reaches ~3.85 GHz peak.

Cortex-A510 Micro-architecture Deep Dive

The Cortex-A510 is ARM's first ARMv9-A LITTLE core, replacing the Cortex-A55. It is found in virtually every flagship SoC since 2022 (Snapdragon 8 Gen 1 / Dimensity 9000 / Exynos 2200) as the efficiency cluster that handles background tasks, Always-On-Display, and sensor hub workloads.

MTE rollout strategy (Android 13/14)

• Pixel 8/8 Pro (Tensor G3, ARMv9.2) — First production Android devices with MTE enabled in async mode for the system allocator (jemalloc variant). Catches heap overflows and UAF asynchronously at ~1% perf overhead.
• Async vs sync modes: Async — tag check happens off the critical path, fault reported asynchronously (lowest overhead, misses some races). Sync — fault on every mismatching access, higher overhead, precise attribution. Used for development/fuzzing.
• Scudo allocator integration: Android's production allocator Scudo uses MTE natively — IRG on each malloc() to assign a random pointer tag, STGM to initialise the page, zeroing tag on free() to catch use-after-free.
• Stack tagging: -fsanitize=memtag-stack tags stack variables at function entry; enables detection of stack buffer overflows and dangling stack pointer use at near-zero overhead vs software AddressSanitizer.

ARM four-world security model

ML instruction evolution — multiply-accumulate throughput per instruction

Emerging directions in ARM architecture research

• ARMv10 (speculative): ARM has not publicly announced ARMv10. Industry speculation points to deeper confidential-computing substrate, enhanced SME3/4 for LLM inference, and possible native CHERI-style capability checks integrated rather than bolted on.
• CHERI (Capability Hardware Enhanced RISC Instructions): A Cambridge/ARM research project adding hardware capability support to AArch64. Each pointer is a 128-bit capability (address + bounds + permissions + tag bit). The Morello evaluation board (Neoverse N1 with CHERI extensions) shipped in 2022. CHERI eliminates entire classes of spatial/temporal memory safety errors in hardware — stronger than MTE's probabilistic 4-bit tags, but with higher pointer storage overhead.
• SME2 (ARMv9.2): Extends SME with multi-vector outer product (FMOPA variants using ZA0–ZA7 tiles simultaneously), lookup tables (LUTI2/LUTI4 for non-linear activation functions), and structured sparse matrix operations for pruned model inference.
• MTE3 (ARMv8.7+): Adds Store-Only tag write mode — reduces overhead when writing untagged data into tagged regions, important for zero-copy network paths.
• BRBE (Branch Record Buffer Extension): Hardware branch history ring buffer — like Intel LBR. Software-controlled, captures up to 64 last taken branches per core with timestamps and EL information. Enables low-overhead production profiling without ETM streaming infrastructure.
• RME v1.1 / CCA silicon: First commercial CCA silicon expected 2025–2026 in cloud server platforms. NVIDIA GH200 descendants and AWS Graviton evolution are candidate first adopters. This enables fully attested confidential VMs on ARM-hosted public cloud.

AWS Graviton — The ARM Server Timeline

Market Impact: ARM server market share grew from <1% (2018) to ~10% (2024), with industry projections of 20%+ by 2027. Major cloud providers now all offer ARM instances: Azure Cobalt (2024, Neoverse N2, 128 cores, ARMv9), Google Axion (2024, Neoverse V2, ARMv9), and NVIDIA Grace (2023, Neoverse V2/Hopper GPU, 480 GB LPDDR5X, 900 GB/s NVLink-C2C for AI supercomputing). The "ARM can't do servers" myth died with Graviton2 — ARMv9 is now accelerating the transition.

Four decades of ARM architecture innovation

MTE Feature Detection & Tag Mismatch Experiment

Explore Memory Tagging Extension by detecting hardware support, allocating tagged memory, and deliberately triggering a tag mismatch fault to understand sync vs async modes.

Step 1 — Check MTE support:

# On a physical ARMv8.5+ device (Pixel 8, Graviton4) or QEMU with MTE:
cat /proc/cpuinfo | grep -i mte

# From EL1 (kernel module or baremetal):
# MRS X0, ID_AA64PFR1_EL1   → bits [11:8] = MTE level (0=none, 1=EL0 only, 2=full, 3=MTE3)
# On QEMU: qemu-system-aarch64 -cpu max -machine virt -append "arm64.nomte=0"

Step 2 — Write a C program that triggers a tag mismatch:

// mte_test.c — MTE tag mismatch demonstration
// Requires Linux 5.10+, kernel CONFIG_ARM64_MTE=y, ARMv8.5+ hardware or QEMU

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/prctl.h>
#include <string.h>

// MTE prctl constants (Linux uapi)
#ifndef PR_SET_TAGGED_ADDR_CTRL
#define PR_SET_TAGGED_ADDR_CTRL  55
#define PR_TAGGED_ADDR_ENABLE    (1UL << 0)
#define PR_MTE_TCF_SYNC          (1UL << 1)   // Synchronous fault
#define PR_MTE_TCF_ASYNC         (1UL << 2)   // Asynchronous fault
#define PR_MTE_TAG_SHIFT         3
#endif

#ifndef PROT_MTE
#define PROT_MTE  0x20
#endif

int main(void) {
    // Enable MTE in synchronous mode for this process
    if (prctl(PR_SET_TAGGED_ADDR_CTRL,
              PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC |
              (0xffffUL << PR_MTE_TAG_SHIFT), 0, 0, 0)) {
        perror("prctl MTE enable failed (no MTE support?)");
        return 1;
    }
    printf("[+] MTE enabled in SYNC mode\n");

    // Allocate MTE-tagged memory (must use mmap with PROT_MTE)
    size_t sz = 4096;
    char *buf = mmap(NULL, sz, PROT_READ | PROT_WRITE | PROT_MTE,
                     MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (buf == MAP_FAILED) { perror("mmap"); return 1; }

    // Tag the pointer using IRG (compiler builtin or inline asm)
    // __arm_mte_create_random_tag assigns a random tag to buf
    char *tagged = __arm_mte_create_random_tag(buf, 0);
    __arm_mte_set_tag(tagged);    // STG — set allocation tag in memory

    printf("[+] Untagged ptr: %p\n", buf);
    printf("[+] Tagged ptr:   %p\n", tagged);

    // Valid access — tags match
    tagged[0] = 'A';
    printf("[+] Valid write succeeded: '%c'\n", tagged[0]);

    // Deliberate mismatch — access with wrong tag (untagged pointer)
    printf("[!] Attempting access with mismatched tag (should fault in SYNC)...\n");
    buf[0] = 'X';  // BUG: buf has tag=0, memory has random tag → SEGV!

    printf("[-] If you see this, MTE did not catch the mismatch\n");
    munmap(buf, sz);
    return 0;
}

// Build: clang --target=aarch64-linux-gnu -march=armv8.5-a+memtag -O2 -o mte_test mte_test.c
// Run:   ./mte_test   → expect SIGSEGV (sync) or SIGBUS (async) on the buf[0] write

Step 3 — Observe sync vs async behaviour: Recompile and run with PR_MTE_TCF_ASYNC instead of PR_MTE_TCF_SYNC. In async mode, the fault is reported later (possibly after the function returns) via TFSRE0_EL1 — useful for production (low overhead) but imprecise. Sync mode faults immediately at the offending instruction — ideal for debugging and fuzzing.

SDOT vs SMMLA Throughput Benchmark

Compare the effective multiply-accumulate throughput of SDOT (ARMv8.4, 16 MACs/instr) versus SMMLA (ARMv8.6, 32 MACs/instr) by timing tight loops with the PMU cycle counter.

// sdot_bench.S — SDOT throughput measurement
.global sdot_bench
.type sdot_bench, %function

sdot_bench:
    // x0 = iteration count (e.g., 10000000)
    // Returns: x0 = cycle count
    movi    v0.16B, #1              // activations = all 1s
    movi    v1.16B, #2              // weights = all 2s
    movi    v2.4S,  #0              // accumulator = 0

    mrs     x1, PMCCNTR_EL0        // read cycle counter (start)
.Lsdot_loop:
    sdot    v2.4S, v0.16B, v1.16B  // 16 MACs
    sdot    v3.4S, v0.16B, v1.16B  // 16 MACs (pipeline fill)
    sdot    v4.4S, v0.16B, v1.16B  // 16 MACs
    sdot    v5.4S, v0.16B, v1.16B  // 16 MACs (4 × 16 = 64 MACs/iteration)
    subs    x0, x0, #1
    b.ne    .Lsdot_loop

    mrs     x2, PMCCNTR_EL0        // read cycle counter (end)
    sub     x0, x2, x1             // x0 = elapsed cycles
    ret

// smmla_bench.S — SMMLA throughput measurement
.global smmla_bench
.type smmla_bench, %function

smmla_bench:
    // x0 = iteration count
    // Returns: x0 = cycle count
    movi    v0.16B, #1
    movi    v1.16B, #2
    movi    v2.4S,  #0

    mrs     x1, PMCCNTR_EL0
.Lsmmla_loop:
    smmla   v2.4S, v0.16B, v1.16B  // 32 MACs
    smmla   v3.4S, v0.16B, v1.16B  // 32 MACs
    smmla   v4.4S, v0.16B, v1.16B  // 32 MACs
    smmla   v5.4S, v0.16B, v1.16B  // 32 MACs (4 × 32 = 128 MACs/iteration)
    subs    x0, x0, #1
    b.ne    .Lsmmla_loop

    mrs     x2, PMCCNTR_EL0
    sub     x0, x2, x1
    ret

// bench_main.c — Driver program
#include <stdio.h>
#include <stdint.h>

extern uint64_t sdot_bench(uint64_t iterations);
extern uint64_t smmla_bench(uint64_t iterations);

int main(void) {
    uint64_t iters = 10000000;

    uint64_t sdot_cycles  = sdot_bench(iters);
    uint64_t smmla_cycles = smmla_bench(iters);

    double sdot_macs  = (double)iters * 64;   // 4 SDOT × 16 MACs
    double smmla_macs = (double)iters * 128;   // 4 SMMLA × 32 MACs

    printf("SDOT:  %lu cycles, %.2f MACs/cycle\n",  sdot_cycles,  sdot_macs / sdot_cycles);
    printf("SMMLA: %lu cycles, %.2f MACs/cycle\n", smmla_cycles, smmla_macs / smmla_cycles);
    printf("SMMLA speedup: %.2fx throughput\n", (smmla_macs/smmla_cycles) / (sdot_macs/sdot_cycles));
    return 0;
}

// Makefile:
//   CC = aarch64-linux-gnu-gcc
//   CFLAGS = -march=armv8.6-a+i8mm -O2
//   bench: bench_main.c sdot_bench.S smmla_bench.S
//       $(CC) $(CFLAGS) -o $@ $^
// Run on ARMv8.6+ or QEMU: qemu-aarch64 -cpu max ./bench

Expected results: On Cortex-X4/Neoverse V2, SMMLA typically achieves ~1.8–2× the throughput of SDOT per cycle, since the 2×8×2 tile covers more output elements per instruction even though both may have similar pipeline latency. On in-order cores (Cortex-A520), the gap is narrower because the frontend is the bottleneck, not execution width.

ARMv9 Feature Register Interrogation (Kernel Module)

Write a Linux kernel module that reads ARM ID registers to produce a comprehensive ARMv9 feature summary — similar to what Linux dmesg reports at boot but with full field-level decoding.

// armv9_features.c — Linux kernel module for ARMv9 feature detection
// Build: make -C /lib/modules/$(uname -r)/build M=$PWD modules
// Load:  sudo insmod armv9_features.ko && dmesg | tail -30

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <asm/sysreg.h>

MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("ARMv9 Feature Register Decoder");

static void decode_pfr0(u64 val) {
    unsigned sve   = (val >> 32) & 0xF;
    unsigned el2   = (val >> 8)  & 0xF;
    unsigned el3   = (val >> 12) & 0xF;
    unsigned fp    = (val >> 16) & 0xF;
    unsigned asimd = (val >> 20) & 0xF;

    pr_info("  ID_AA64PFR0_EL1 = 0x%016llx\n", val);
    pr_info("    SVE:   %s (level %u)\n", sve ? "YES" : "NO", sve);
    pr_info("    EL2:   %s\n", el2 ? "implemented" : "not implemented");
    pr_info("    EL3:   %s\n", el3 ? "implemented" : "not implemented");
    pr_info("    FP:    %s\n", fp != 0xF ? "YES" : "NO");
    pr_info("    ASIMD: %s\n", asimd != 0xF ? "YES" : "NO");
}

static void decode_pfr1(u64 val) {
    unsigned mte   = (val >> 8)  & 0xF;
    unsigned sme   = (val >> 24) & 0xF;
    unsigned bt    = (val >> 0)  & 0xF;

    pr_info("  ID_AA64PFR1_EL1 = 0x%016llx\n", val);
    pr_info("    MTE:   level %u (%s)\n", mte,
            mte == 0 ? "none" : mte == 1 ? "EL0 insn only" :
            mte == 2 ? "full MTE2" : mte == 3 ? "MTE3 (asym)" : "unknown");
    pr_info("    SME:   %s (level %u)\n", sme ? "YES" : "NO", sme);
    pr_info("    BTI:   %s\n", bt ? "YES" : "NO");
}

static void decode_zfr0(u64 val) {
    unsigned svever  = (val >> 0)  & 0xF;
    unsigned bf16    = (val >> 20) & 0xF;
    unsigned i8mm    = (val >> 44) & 0xF;

    pr_info("  ID_AA64ZFR0_EL1 = 0x%016llx\n", val);
    pr_info("    SVE2:    %s (ver %u)\n", svever ? "YES" : "SVE1 only", svever);
    pr_info("    BFloat16: %s\n", bf16 ? "YES" : "NO");
    pr_info("    I8MM:    %s\n", i8mm ? "YES" : "NO");
}

static void decode_smfr0(u64 val) {
    unsigned smever  = (val >> 56) & 0xF;

    pr_info("  ID_AA64SMFR0_EL1 = 0x%016llx\n", val);
    pr_info("    SME version: %u (%s)\n", smever,
            smever == 0 ? "not implemented" : smever == 1 ? "SME" :
            smever == 2 ? "SME2" : "SME2+");
}

static int __init armv9_features_init(void) {
    u64 pfr0, pfr1, zfr0, smfr0;

    pr_info("=== ARMv9 Feature Register Dump ===\n");

    pfr0 = read_sysreg_s(SYS_ID_AA64PFR0_EL1);
    decode_pfr0(pfr0);

    pfr1 = read_sysreg_s(SYS_ID_AA64PFR1_EL1);
    decode_pfr1(pfr1);

    zfr0 = read_sysreg_s(SYS_ID_AA64ZFR0_EL1);
    decode_zfr0(zfr0);

    // SME register only valid if SME is present
    if (((pfr1 >> 24) & 0xF) > 0) {
        smfr0 = read_sysreg_s(SYS_ID_AA64SMFR0_EL1);
        decode_smfr0(smfr0);
    } else {
        pr_info("  SME not present — skipping ID_AA64SMFR0_EL1\n");
    }

    pr_info("=== End Feature Dump ===\n");
    return 0;
}

static void __exit armv9_features_exit(void) {
    pr_info("ARMv9 feature module unloaded\n");
}

module_init(armv9_features_init);
module_exit(armv9_features_exit);

# Makefile for the kernel module
# obj-m += armv9_features.o
# all:
#     make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
# clean:
#     make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

# Expected output on AWS Graviton4 (Neoverse V2, ARMv9):
# === ARMv9 Feature Register Dump ===
#   ID_AA64PFR0_EL1 = 0x1101000011112222
#     SVE:   YES (level 1)
#     EL2:   implemented
#     EL3:   implemented
#     FP:    YES
#     ASIMD: YES
#   ID_AA64PFR1_EL1 = 0x0000000000000220
#     MTE:   level 2 (full MTE2)
#     SME:   NO (level 0)
#     BTI:   YES
#   ID_AA64ZFR0_EL1 = 0x0000100000100001
#     SVE2:    YES (ver 1)
#     BFloat16: YES
#     I8MM:    YES
#   SME not present — skipping ID_AA64SMFR0_EL1
# === End Feature Dump ===

Challenge extensions: (a) Add decoding for ID_AA64ISAR0_EL1 to detect AES, SHA, CRC32, and atomic instruction support. (b) Add ID_AA64MMFR0_EL1 to detect physical address size (PA range), granule sizes, and stage-2 translation support. (c) Compare output between QEMU -cpu max (all features) and -cpu cortex-a76 (ARMv8.2 subset) to see the feature gap.

28-part learning arc