Chapter 16: Virtual Memory

16.1 What is Virtual Memory?

Virtual memory is an abstraction that gives each process its own private address space. When a process accesses memory at address 0x1000, the MMU (Memory Management Unit) translates that virtual address to a physical address in RAM. Different processes can use the same virtual address but map to different physical addresses.

Virtual memory provides three benefits:

1. Isolation: one process cannot access another process's memory
2. Simplification: each process thinks it has the entire address space to itself
3. Demand paging: the kernel can load pages lazily from disk as they are accessed

16.2 Virtual Address Space on ARM64

ARM64 uses a 48-bit virtual address space by default (can be configured up to 52 bits). The 48-bit space is divided into two halves:

Address Range	Size	Usage
0x0000_0000_0000_0000 - 0x0000_FFFF_FFFF_FFFF	256 TB	User space (EL0), mapped by TTBR0_EL1
0xFFFF_0000_0000_0000 - 0xFFFF_FFFF_FFFF_FFFF	256 TB	Kernel space (EL1), mapped by TTBR1_EL1

The top 16 bits of a 48-bit virtual address must match bit 47 (sign-extended). Addresses with bit 47 = 0 are user space; bit 47 = 1 is kernel space.

Our kernel's virtual memory layout (simplified):

Virtual Address         | Mapping
------------------------+------------------------
0x0000_0000_0000_0000   | User process (per-process)
0x0000_0000_4000_0000   | User program load address
0x0000_0000_8000_0000   | User heap (grows up)
0x0000_0000_C000_0000   | User stack (grows down)

0xFFFF_0000_0000_0000   | Kernel image (physical = 0x4000_0000)
0xFFFF_0000_4000_0000   | Kernel load address (direct map)
0xFFFF_0000_8000_0000   | Kernel heap
0xFFFF_0000_9000_0000   | Device MMIO (UART, GIC, etc.)
0xFFFF_0000_C000_0000   | Page tables
0xFFFF_FFFF_FFFF_FFFF   | End of address space

16.3 Page Tables and Translation

The MMU translates virtual addresses to physical addresses using page tables. ARM64 uses a multi-level page table structure. With 4 KB pages and 48-bit addresses, there are 4 levels (L0, L1, L2, L3):

• L0: covers 512 GB per entry (512 entries)
• L1: covers 1 GB per entry (512 entries)
• L2: covers 2 MB per entry (512 entries)
• L3: covers 4 KB per entry (512 entries)

Each level's table has 512 entries of 8 bytes each (4 KB total). The translation walks from L0 to L3, using portions of the virtual address as indices at each level. At L2 or L3, an entry can be a block (large page, like 2 MB or 1 GB) or a page (4 KB).

16.4 Enabling the MMU

Enabling the MMU requires careful setup because the CPU is currently running with physical addresses. We must:

1. Set up page tables that identity-map the kernel's current location
2. Configure TCR_EL1 (translation control registers)
3. Configure MAIR_EL1 (memory attributes)
4. Set TTBR0_EL1 and TTBR1_EL1 (page table base registers)
5. Set SCTLR_EL1.M (MMU enable bit)
6. Execute a branch to a virtual address (now running with MMU on)

void mmu_enable(void) {
    /* Configure TCR: 48-bit VA, 4 KB pages, inner/shareable */
    uint64_t tcr = (64 - 48) << 0 |     /* T0SZ = 16 (48-bit user VA) */
                   (64 - 48) << 16 |    /* T1SZ = 16 (48-bit kernel VA) */
                   2 << 12 |            /* TG0 = 4 KB pages */
                   2 << 14 |            /* TG1 = 4 KB pages */
                   3 << 8 |             /* Inner shareable */
                   3 << 24;             /* Inner shareable (kernel) */
    __asm__("msr TCR_EL1, %0" : : "r" (tcr));

    /* Configure MAIR: index 0 = normal memory, index 1 = device memory */
    uint64_t mair = (0xFFUL << 0) |     /* Attr 0: Normal, WB, RW, non-transient */
                    (0x00UL << 8);      /* Attr 1: Device-nGnRnE */
    __asm__("msr MAIR_EL1, %0" : : "r" (mair));

    /* Set page table bases */
    /* TTBR0_EL1 = user space page table (per-process) */
    /* TTBR1_EL1 = kernel space page table (global, same for all processes) */
    __asm__("msr TTBR1_EL1, %0" : : "r" (kernel_page_table));
    __asm__("msr TTBR0_EL1, %0" : : "r" (kernel_page_table));  /* start with kernel */

    /* Ensure all writes complete before enabling MMU */
    __asm__("dsb sy; isb");

    /* Enable MMU (bit 0), data cache (bit 2), instruction cache (bit 12) */
    uint64_t sctlr;
    __asm__("mrs %0, SCTLR_EL1" : "=r" (sctlr));
    sctlr |= (1 << 0) | (1 << 2) | (1 << 12);
    __asm__("msr SCTLR_EL1, %0" : : "r" (sctlr));
    __asm__("isb");

    /* After this point, all addresses are virtual */
}

16.5 Identity Mapping and the Direct Map

When the MMU is first enabled, we must identity-map the kernel's current location (virtual address = physical address). Otherwise the CPU will try to fetch the next instruction from a virtual address that has no translation, causing a page fault.

After the MMU is on, we can add a direct map that maps physical memory at a high kernel virtual address (e.g., 0xFFFF_0000_0000_0000). This allows the kernel to access any physical page by adding a fixed offset. Linux calls this linear mapping or direct mapping.

16.6 Our Implementation

In our kernel, virtual memory is initialized in stages:

1. Early boot: physical addresses only, MMU off
2. Identity map: set up minimal page tables, enable MMU
3. Kernel map: add the direct map for all physical memory
4. Device map: map UART, GIC, and other MMIO regions
5. Per-process: create separate TTBR0_EL1 for each user process

After the MMU is enabled, the kernel runs at virtual addresses in the 0xFFFF_ range. User processes run at low addresses (0x0000_). The MMU provides complete isolation: a user process cannot access kernel memory, and different user processes cannot access each other's memory.

16.7 Exercises

16.8 Summary

Virtual memory gives each process its own private address space, enforced by the MMU. On ARM64, the 48-bit virtual address space is split into user (lower half) and kernel (upper half). The MMU translates virtual addresses to physical addresses using multi-level page tables. Our kernel enables the MMU after setting up identity-mapped page tables, then maintains separate address spaces for each user process via TTBR0_EL1 switching during context switches.