Kernel Development Series Phase 0: Orientation & Big Picture

Kernel vs Full Operating System

People often use "kernel" and "operating system" interchangeably, but they're different things. Understanding this distinction is crucial before you start building.

The Kernel: Heart of the OS

The kernel is the core component that runs in privileged mode with full hardware access. It provides the most fundamental services:

┌─────────────────────────────────────────────────────┐
│                  User Applications                   │
│    (Chrome, VS Code, Games, Your Programs)          │
├─────────────────────────────────────────────────────┤
│              System Services / Daemons               │
│    (Window Manager, Network Stack, File Services)    │
├─────────────────────────────────────────────────────┤
│                   System Libraries                   │
│         (libc, Win32 API, Cocoa Framework)          │
├─────────────────────────────────────────────────────┤
│           ═══════════════════════════════           │
│                    SYSTEM CALL BOUNDARY              │
│           ═══════════════════════════════           │
├─────────────────────────────────────────────────────┤
│                      KERNEL                          │
│  ┌─────────────┬─────────────┬─────────────────┐    │
│  │  Process    │   Memory    │   File System   │    │
│  │  Manager    │   Manager   │   Interface     │    │
│  ├─────────────┼─────────────┼─────────────────┤    │
│  │  Device     │  Inter-Proc │   Security      │    │
│  │  Drivers    │  Communic.  │   Module        │    │
│  └─────────────┴─────────────┴─────────────────┘    │
├─────────────────────────────────────────────────────┤
│                    HARDWARE                          │
│      CPU      RAM      Disk      Network            │
└─────────────────────────────────────────────────────┘

The Full Operating System

A full operating system includes the kernel plus many additional components:

Kernel Architectures

One of the first decisions in OS design is the kernel architecture. This fundamental choice affects everything from performance to reliability to how you'll structure your code.

Monolithic Kernels

In a monolithic kernel, all kernel services run in the same address space with full privileges. Think of it as a single large program that handles everything.

MONOLITHIC KERNEL
═══════════════════════════════════════════════════════
                    
  ┌────────────────────────────────────────────────┐
  │                USER SPACE (Ring 3)              │
  │     App1         App2         App3             │
  └────────────────────────────────────────────────┘
                         │
              ════════════════════════
                   SYSTEM CALL
              ════════════════════════
                         │
  ┌────────────────────────────────────────────────┐
  │              KERNEL SPACE (Ring 0)             │
  │  ┌──────────────────────────────────────────┐  │
  │  │  Process   Memory    VFS    Network      │  │
  │  │  Scheduler Manager        TCP/IP Stack   │  │
  │  ├──────────────────────────────────────────┤  │
  │  │  Device Drivers (disk, keyboard, GPU)    │  │
  │  ├──────────────────────────────────────────┤  │
  │  │  Hardware Abstraction Layer (HAL)        │  │
  │  └──────────────────────────────────────────┘  │
  └────────────────────────────────────────────────┘
                         │
                    HARDWARE

Advantages:

• Performance - No context switches between kernel components; direct function calls are fast
• Simplicity - Components can share data structures directly
• Mature ecosystem - Linux, FreeBSD, most classic UNIX systems use this model

Disadvantages:

• Stability risk - A bug in any driver can crash the entire system
• Security risk - A compromised driver has full system access
• Large codebase - Everything compiled together makes changes risky

# See loaded kernel modules
lsmod

# Load a module (e.g., USB storage driver)
sudo modprobe usb_storage

# Remove a module
sudo rmmod usb_storage

Microkernels

A microkernel moves as much functionality as possible out of kernel space into user space. The kernel handles only the absolute essentials:

• Inter-process communication (IPC)
• Basic scheduling
• Low-level address space management

MICROKERNEL
═══════════════════════════════════════════════════════

  ┌────────────────────────────────────────────────┐
  │                USER SPACE (Ring 3)              │
  │                                                 │
  │  ┌─────────┐ ┌─────────┐ ┌─────────┐          │
  │  │  File   │ │ Network │ │  Device │  ← SERVERS│
  │  │ System  │ │  Stack  │ │ Drivers │          │
  │  │ Server  │ │ Server  │ │ Servers │          │
  │  └────┬────┘ └────┬────┘ └────┬────┘          │
  │       │           │           │                │
  │       └───────────┼───────────┘                │
  │                   │                            │
  │  ┌─────────┐ ┌─────────┐ ┌─────────┐          │
  │  │  App1   │ │  App2   │ │  App3   │          │
  │  └────┬────┘ └────┬────┘ └────┬────┘          │
  └───────┼───────────┼───────────┼────────────────┘
          │           │           │
     ════════════════════════════════════
                IPC (Message Passing)
     ════════════════════════════════════
          │           │           │
  ┌───────┴───────────┴───────────┴────────────────┐
  │              KERNEL SPACE (Ring 0)             │
  │  ┌──────────────────────────────────────────┐  │
  │  │   IPC    │   Scheduler   │   Basic MM    │  │
  │  └──────────────────────────────────────────┘  │
  └────────────────────────────────────────────────┘
                         │
                    HARDWARE

Advantages:

• Reliability - Crash in a driver only affects that service, not the whole system
• Security - Smaller trusted computing base (TCB)
• Modularity - Services can be restarted independently

Disadvantages:

• Performance overhead - IPC is slower than direct function calls
• Complexity - Everything becomes a distributed systems problem
• Less mature tooling - Fewer developers, smaller ecosystems

Hybrid Kernels

A hybrid kernel attempts to combine the best of both worlds. It runs more code in kernel space than a microkernel (for performance) but adopts microkernel ideas like message passing and modularity.

HYBRID KERNEL (Windows NT)
═══════════════════════════════════════════════════════

  ┌────────────────────────────────────────────────┐
  │                USER SPACE                       │
  │     Applications    Subsystems (Win32, POSIX)  │
  └────────────────────────────────────────────────┘
                         │
              ═══════════════════════════
                      SYSTEM CALLS
              ═══════════════════════════
                         │
  ┌────────────────────────────────────────────────┐
  │              KERNEL SPACE                      │
  │  ┌──────────────────────────────────────────┐  │
  │  │            Executive Services            │  │
  │  │   I/O Manager │ Memory │ Process        │  │
  │  │   PnP Manager │ Manager│ Manager        │  │
  │  ├──────────────────────────────────────────┤  │
  │  │              Microkernel Core            │  │
  │  │   Thread Scheduling │ Exception Handling │  │
  │  │   Interrupt Dispatch│ Synchronization    │  │
  │  ├──────────────────────────────────────────┤  │
  │  │         Hardware Abstraction Layer       │  │
  │  └──────────────────────────────────────────┘  │
  └────────────────────────────────────────────────┘

Examples: Windows NT/XP/10/11, macOS/XNU, BeOS/Haiku

Real-World Kernels Compared

Let's examine how major operating systems implement their kernels. Understanding their design decisions will inform our own choices.

Linux Kernel

/* Linux kernel entry point (simplified) - arch/x86/kernel/head_64.S → init/main.c */
asmlinkage __visible void __init start_kernel(void)
{
    /* Lock scheduler to prevent races during boot */
    local_irq_disable();
    
    /* Initialize CPU, memory, and core subsystems */
    setup_arch(&command_line);      /* Architecture-specific setup */
    setup_per_cpu_areas();          /* Per-CPU data structures */
    mm_init();                      /* Memory management */
    sched_init();                   /* Scheduler */
    
    /* Initialize drivers and filesystems */
    init_IRQ();                     /* Interrupt handlers */
    time_init();                    /* Timekeeping */
    console_init();                 /* Console output */
    
    /* Start the init process (PID 1) */
    rest_init();                    /* Creates kernel threads, runs /sbin/init */
}

Windows NT Kernel

/* Conceptual Windows kernel initialization (not actual code) */
void KiSystemStartup(LOADER_PARAMETER_BLOCK* LoaderBlock)
{
    /* Phase 0: Basic initialization */
    HalInitSystem(0, LoaderBlock);     /* Initialize HAL */
    KiInitializeKernel();              /* Kernel data structures */
    KeInitializeInterrupts();          /* Set up IDT */
    
    /* Phase 1: Start executive components */
    ExInitSystem();                    /* Executive layer */
    ObInitSystem();                    /* Object Manager */
    SeInitSystem();                    /* Security Reference Monitor */
    MmInitSystem();                    /* Memory Manager */
    IoInitSystem();                    /* I/O Manager - starts drivers */
    
    /* Start Session Manager (smss.exe) - creates user sessions */
    PsCreateSystemProcess("\\SystemRoot\\System32\\smss.exe");
}

macOS XNU Kernel

The Complete Computing Stack

Before diving into kernel code, let's understand where the kernel fits in the complete picture - from silicon to applications.

Hardware Layer

At the bottom is the physical hardware:

Firmware (BIOS/UEFI)

When you press the power button, the CPU doesn't immediately run your operating system. First, it runs firmware - software built into the motherboard.

1. POWER ON
   └─→ Power Supply sends "Power Good" signal to motherboard

2. CPU RESET
   └─→ CPU starts executing at a fixed address (0xFFFFFFF0)
   └─→ This address is mapped to firmware ROM

3. FIRMWARE INITIALIZATION (BIOS or UEFI)
   └─→ POST (Power-On Self-Test)
   └─→ Initialize hardware (RAM, USB, display)
   └─→ Find bootable device (disk, USB, network)
   └─→ Load boot sector / EFI application

4. BOOTLOADER (GRUB, Windows Boot Manager, etc.)
   └─→ Present boot menu (optional)
   └─→ Load kernel into memory
   └─→ Pass control to kernel

5. KERNEL INITIALIZATION
   └─→ Set up memory management
   └─→ Initialize drivers
   └─→ Start first user process

6. USER SPACE
   └─→ Init system starts services
   └─→ Login manager appears
   └─→ You start doing work!

Kernel & User Space

Modern CPUs have multiple privilege levels (called "rings" on x86). This is fundamental to OS security:

       ╔═══════════════════════════════════════════════════════╗
       ║              CPU PRIVILEGE RINGS (x86)                ║
       ╠═══════════════════════════════════════════════════════╣
       ║                                                        ║
       ║    ┌──────────────────────────────────────────────┐    ║
       ║    │             RING 3 (User Mode)               │    ║
       ║    │                                              │    ║
       ║    │   Applications, Shells, Utilities           │    ║
       ║    │   - Cannot access hardware directly         │    ║
       ║    │   - Cannot execute privileged instructions  │    ║
       ║    │   - Limited to its own address space        │    ║
       ║    │                                              │    ║
       ║    │  ┌──────────────────────────────────────┐   │    ║
       ║    │  │         RING 0 (Kernel Mode)         │   │    ║
       ║    │  │                                      │   │    ║
       ║    │  │   Operating System Kernel            │   │    ║
       ║    │  │   - Full hardware access             │   │    ║
       ║    │  │   - Execute any instruction          │   │    ║
       ║    │  │   - Access all memory                │   │    ║
       ║    │  │                                      │   │    ║
       ║    │  └──────────────────────────────────────┘   │    ║
       ║    │                                              │    ║
       ║    └──────────────────────────────────────────────┘    ║
       ║                                                        ║
       ║  (Rings 1 & 2 exist but are rarely used in modern OS) ║
       ╚═══════════════════════════════════════════════════════╝

Why this separation matters:

• Protection - A buggy application can't crash the whole system
• Security - Malware can't directly modify kernel memory
• Stability - The kernel can terminate misbehaving programs

/* User space: "I want to write to a file" */
write(fd, buffer, size);    /* This triggers a system call */

/* Internally:
   1. Arguments placed in registers
   2. "syscall" instruction executed
   3. CPU switches to Ring 0
   4. Kernel writes data to file
   5. CPU switches back to Ring 3
   6. Control returns to user program
*/

Series Overview & Learning Path

This series takes you from zero to a working operating system. Here's the journey:

Development Milestones

Tools You'll Use

Exercises

Before moving to Phase 1, strengthen your foundation with these exercises:

# View kernel version
uname -a

# See kernel boot messages
dmesg | head -50

# List loaded kernel modules
lsmod

# View system calls made by a program
strace ls

# View system information (PowerShell)
systeminfo

# View kernel drivers
driverquery

# View system calls (requires Process Monitor from Sysinternals)

#include <stdio.h>
#include <stdlib.h>

int global_variable = 42;

void sample_function(void) {
    int local_variable = 100;
    printf("Local variable address: %p\n", (void*)&local_variable);
}

int main() {
    int stack_variable = 10;
    int *heap_variable = malloc(sizeof(int));
    
    printf("Global variable address: %p\n", (void*)&global_variable);
    printf("Stack variable address:  %p\n", (void*)&stack_variable);
    printf("Heap variable address:   %p\n", (void*)heap_variable);
    printf("Function address:        %p\n", (void*)&sample_function);
    printf("Main address:            %p\n", (void*)&main);
    
    sample_function();
    
    free(heap_variable);
    return 0;
}

Next Steps

You now understand:

• What an operating system is and why we need one
• The difference between kernel and full OS
• Major kernel architectures (monolithic, microkernel, hybrid)
• How Linux, Windows, and macOS implement their kernels
• The complete computing stack from hardware to applications
• The learning path ahead of us