# Firecracker vs Volt: CPU State Setup Comparison

This document compares how Firecracker and Volt set up vCPU state for 64-bit Linux kernel boot.

## Executive Summary

| Aspect | Firecracker | Volt | Verdict |
|--------|-------------|-----------|---------|
| Boot protocols | PVH + Linux boot | Linux boot (64-bit) | Firecracker more flexible |
| CR0 flags | Minimal (PE+PG+ET) | Extended (adds WP, NE, AM, MP) | Volt more complete |
| CR4 flags | Minimal (PAE only) | Extended (adds PGE, OSFXSR, OSXMMEXCPT) | Volt more complete |
| Page tables | Single identity map (1GB) | Identity + high kernel map | Volt more thorough |
| Code quality | Battle-tested, production | New implementation | Firecracker proven |

---

## 1. Control Registers

### CR0 (Control Register 0)

| Bit | Name | Firecracker (Linux) | Volt | Notes |
|-----|------|---------------------|-----------|-------|
| 0 | PE (Protection Enable) | ✅ | ✅ | Required for protected mode |
| 1 | MP (Monitor Coprocessor) | ❌ | ✅ | FPU monitoring |
| 4 | ET (Extension Type) | ✅ | ✅ | 387 coprocessor present |
| 5 | NE (Numeric Error) | ❌ | ✅ | Native FPU error handling |
| 16 | WP (Write Protect) | ❌ | ✅ | Page-level write protection |
| 18 | AM (Alignment Mask) | ❌ | ✅ | Alignment checking |
| 31 | PG (Paging) | ✅ | ✅ | Enable paging |

**Firecracker CR0 values:**
```rust
// Linux boot:
sregs.cr0 |= X86_CR0_PE;  // After segments/sregs setup
sregs.cr0 |= X86_CR0_PG;  // After page tables setup
// Final: ~0x8000_0001

// PVH boot:
sregs.cr0 = X86_CR0_PE | X86_CR0_ET;  // 0x11
// No paging enabled!
```

**Volt CR0 value:**
```rust
sregs.cr0 = 0x8003_003B;  // PG | PE | MP | ET | NE | WP | AM
```

**⚠️ Key Difference:** Volt enables more CR0 features by default. Firecracker's minimal approach is intentional for PVH (no paging required), but for Linux boot both should work. Volt's WP and NE flags are arguably better defaults for modern kernels.

---

### CR3 (Page Table Base)

| VMM | Address | Notes |
|-----|---------|-------|
| Firecracker | `0x9000` | PML4 location |
| Volt | `0x1000` | PML4 location |

**Impact:** Different page table locations. Both are valid low memory addresses.

---

### CR4 (Control Register 4)

| Bit | Name | Firecracker | Volt | Notes |
|-----|------|-------------|-----------|-------|
| 5 | PAE (Physical Address Extension) | ✅ | ✅ | Required for 64-bit |
| 7 | PGE (Page Global Enable) | ❌ | ✅ | TLB optimization |
| 9 | OSFXSR (OS FXSAVE/FXRSTOR) | ❌ | ✅ | SSE support |
| 10 | OSXMMEXCPT (OS Unmasked SIMD FP) | ❌ | ✅ | SIMD exceptions |

**Firecracker CR4:**
```rust
sregs.cr4 |= X86_CR4_PAE;  // 0x20
// PVH boot: sregs.cr4 = 0
```

**Volt CR4:**
```rust
sregs.cr4 = 0x668;  // PAE | PGE | OSFXSR | OSXMMEXCPT
```

**⚠️ Key Difference:** Volt enables OSFXSR and OSXMMEXCPT which are required for SSE instructions. Modern Linux kernels expect these. Firecracker relies on the kernel to enable them later.

---

### EFER (Extended Feature Enable Register)

| Bit | Name | Firecracker (Linux) | Volt | Notes |
|-----|------|---------------------|-----------|-------|
| 8 | LME (Long Mode Enable) | ✅ | ✅ | Enable 64-bit |
| 10 | LMA (Long Mode Active) | ✅ | ✅ | 64-bit active |

**Both use:**
```rust
// Firecracker:
sregs.efer |= EFER_LME | EFER_LMA;  // 0x100 | 0x400 = 0x500

// Volt:
sregs.efer = 0x500;  // LME | LMA
```

**✅ Match:** Both correctly enable long mode.

---

## 2. Segment Registers

### GDT (Global Descriptor Table)

**Firecracker GDT (Linux boot):**
```rust
// Location: 0x500
[
    gdt_entry(0, 0, 0),            // 0x00: NULL
    gdt_entry(0xa09b, 0, 0xfffff), // 0x08: CODE64 - 64-bit execute/read
    gdt_entry(0xc093, 0, 0xfffff), // 0x10: DATA64 - read/write
    gdt_entry(0x808b, 0, 0xfffff), // 0x18: TSS
]
// Result: CODE64 = 0x00AF_9B00_0000_FFFF
//         DATA64 = 0x00CF_9300_0000_FFFF
```

**Firecracker GDT (PVH boot):**
```rust
[
    gdt_entry(0, 0, 0),                // 0x00: NULL
    gdt_entry(0xc09b, 0, 0xffff_ffff), // 0x08: CODE32 - 32-bit!
    gdt_entry(0xc093, 0, 0xffff_ffff), // 0x10: DATA
    gdt_entry(0x008b, 0, 0x67),        // 0x18: TSS
]
// Note: 32-bit code segment for PVH protected mode boot
```

**Volt GDT:**
```rust
// Location: 0x500
CODE64 = 0x00AF_9B00_0000_FFFF  // selector 0x10
DATA64 = 0x00CF_9300_0000_FFFF  // selector 0x18
```

### Segment Selectors

| Segment | Firecracker | Volt | Notes |
|---------|-------------|-----------|-------|
| CS | 0x08 | 0x10 | Code segment |
| DS/ES/FS/GS/SS | 0x10 | 0x18 | Data segments |

**⚠️ Key Difference:** Firecracker uses GDT entries 1/2 (selectors 0x08/0x10), Volt uses entries 2/3 (selectors 0x10/0x18). Both are valid but could cause issues if assuming specific selector values.

### Segment Configuration

**Firecracker code segment:**
```rust
kvm_segment {
    base: 0,
    limit: 0xFFFF_FFFF,  // Scaled from gdt_entry
    selector: 0x08,
    type_: 0xB,          // Execute/Read, accessed
    present: 1,
    dpl: 0,
    db: 0,               // 64-bit mode
    s: 1,
    l: 1,                // Long mode
    g: 1,
}
```

**Volt code segment:**
```rust
kvm_segment {
    base: 0,
    limit: 0xFFFF_FFFF,
    selector: 0x10,
    type_: 11,           // Execute/Read, accessed
    present: 1,
    dpl: 0,
    db: 0,
    s: 1,
    l: 1,
    g: 1,
}
```

**✅ Match:** Segment configurations are functionally identical (just different selectors).

---

## 3. Page Tables

### Memory Layout

**Firecracker page tables (Linux boot only):**
```
0x9000: PML4
0xA000: PDPTE
0xB000: PDE (512 × 2MB entries = 1GB coverage)
```

**Volt page tables:**
```
0x1000: PML4
0x2000: PDPT (low memory identity map)
0x3000: PDPT (high kernel 0xFFFFFFFF80000000+)
0x4000+: PD tables (2MB huge pages)
```

### Page Table Entries

**Firecracker:**
```rust
// PML4[0] -> PDPTE
mem.write_obj(boot_pdpte_addr.raw_value() | 0x03, boot_pml4_addr);

// PDPTE[0] -> PDE
mem.write_obj(boot_pde_addr.raw_value() | 0x03, boot_pdpte_addr);

// PDE[i] -> 2MB huge pages
for i in 0..512 {
    mem.write_obj((i << 21) + 0x83u64, boot_pde_addr.unchecked_add(i * 8));
}
// 0x83 = Present | Writable | PageSize (2MB huge page)
```

**Volt:**
```rust
// PML4[0] -> PDPT_LOW (identity mapping)
let pml4_entry_0 = PDPT_LOW_ADDR | PRESENT | WRITABLE;  // 0x2003

// PML4[511] -> PDPT_HIGH (kernel high mapping)
let pml4_entry_511 = PDPT_HIGH_ADDR | PRESENT | WRITABLE;  // 0x3003

// PD entries use 2MB huge pages
let pd_entry = phys_addr | PRESENT | WRITABLE | PAGE_SIZE;  // 0x83
```

### Coverage

| VMM | Identity Map | High Kernel Map |
|-----|--------------|-----------------|
| Firecracker | 0-1GB | None |
| Volt | 0-4GB | 0xFFFFFFFF80000000+ → 0-2GB |

**⚠️ Key Difference:** Volt sets up both identity mapping AND high kernel address mapping (0xFFFFFFFF80000000+). This is more thorough and matches what a real Linux kernel expects. Firecracker only does identity mapping and relies on the kernel to set up its own page tables.

---

## 4. General Purpose Registers

### Initial Register State

**Firecracker (Linux boot):**
```rust
kvm_regs {
    rflags: 0x2,                        // Reserved bit
    rip: entry_point,                   // Kernel entry
    rsp: 0x8ff0,                        // BOOT_STACK_POINTER
    rbp: 0x8ff0,                        // Frame pointer
    rsi: 0x7000,                        // ZERO_PAGE_START (boot_params)
    // All other registers: 0
}
```

**Firecracker (PVH boot):**
```rust
kvm_regs {
    rflags: 0x2,
    rip: entry_point,
    rbx: 0x6000,                        // PVH_INFO_START
    // All other registers: 0
}
```

**Volt:**
```rust
kvm_regs {
    rip: kernel_entry,
    rsi: boot_params_addr,              // Linux boot protocol
    rflags: 0x2,
    rsp: 0x8000,                        // Stack pointer
    // All other registers: 0
}
```

| Register | Firecracker (Linux) | Volt | Protocol |
|----------|---------------------|-----------|----------|
| RIP | entry_point | kernel_entry | ✅ |
| RSI | 0x7000 | boot_params_addr | Linux boot params |
| RSP | 0x8ff0 | 0x8000 | Stack |
| RBP | 0x8ff0 | 0 | Frame pointer |
| RFLAGS | 0x2 | 0x2 | ✅ |

**⚠️ Minor Difference:** Firecracker sets RBP to stack pointer, Volt leaves it at 0. Both are valid.

---

## 5. Memory Layout

### Key Addresses

| Structure | Firecracker | Volt | Notes |
|-----------|-------------|-----------|-------|
| GDT | 0x500 | 0x500 | ✅ Match |
| IDT | 0x520 | 0 (limit only) | Volt uses null IDT |
| Page Tables (PML4) | 0x9000 | 0x1000 | Different |
| PVH start_info | 0x6000 | 0x7000 | Different |
| boot_params/zero_page | 0x7000 | 0x20000 | Different |
| Command line | 0x20000 | 0x8000 | Different |
| E820 map | In zero_page | 0x9000 | Volt separate |
| Stack pointer | 0x8ff0 | 0x8000 | Different |
| Kernel load | 0x100000 (1MB) | 0x100000 (1MB) | ✅ Match |
| TSS address | 0xfffbd000 | N/A | KVM requirement |

### E820 Memory Map

Both implementations create similar E820 maps:

```
Entry 0: 0x0 - 0x9FFFF (640KB) - RAM
Entry 1: 0xA0000 - 0xFFFFF (384KB) - Reserved (legacy hole)
Entry 2: 0x100000 - RAM_END - RAM
```

---

## 6. FPU Configuration

**Firecracker:**
```rust
let fpu = kvm_fpu {
    fcw: 0x37f,       // FPU Control Word
    mxcsr: 0x1f80,    // MXCSR - SSE control
    ..Default::default()
};
vcpu.set_fpu(&fpu);
```

**Volt:** Currently does not explicitly configure FPU state.

**⚠️ Recommendation:** Volt should add FPU initialization similar to Firecracker.

---

## 7. Boot Protocol Support

| Protocol | Firecracker | Volt |
|----------|-------------|-----------|
| Linux 64-bit boot | ✅ | ✅ |
| PVH boot | ✅ | ✅ (structures only) |
| 32-bit protected mode entry | ✅ (PVH) | ❌ |
| EFI handover | ❌ | ❌ |

**Firecracker PVH boot** starts in 32-bit protected mode (no paging, CR4=0, CR0=PE|ET), while **Volt** always starts in 64-bit long mode.

---

## 8. Recommendations for Volt

### High Priority

1. **Add FPU initialization:**
   ```rust
   let fpu = kvm_fpu {
       fcw: 0x37f,
       mxcsr: 0x1f80,
       ..Default::default()
   };
   self.fd.set_fpu(&fpu)?;
   ```

2. **Consider CR0/CR4 simplification:**
   - Your extended flags (WP, NE, AM, PGE, etc.) are fine for modern kernels
   - But may cause issues with older kernels or custom code
   - Firecracker's minimal approach is more universally compatible

### Medium Priority

3. **Standardize memory layout:**
   - Consider aligning with Firecracker's layout for compatibility
   - Especially boot_params at 0x7000 and cmdline at 0x20000

4. **Add proper PVH 32-bit boot support:**
   - If you want true PVH compatibility, support 32-bit protected mode entry
   - Currently Volt always boots in 64-bit mode

### Low Priority

5. **Page table coverage:**
   - Your dual identity+high mapping is more thorough
   - But Firecracker's 1GB identity map is sufficient for boot
   - Linux kernel sets up its own page tables quickly

---

## 9. Code References

### Firecracker
- `src/vmm/src/arch/x86_64/regs.rs` - Register setup
- `src/vmm/src/arch/x86_64/gdt.rs` - GDT construction
- `src/vmm/src/arch/x86_64/layout.rs` - Memory layout constants
- `src/vmm/src/arch/x86_64/mod.rs` - Boot configuration

### Volt
- `vmm/src/kvm/vcpu.rs` - vCPU setup (`setup_long_mode_with_cr3`)
- `vmm/src/boot/gdt.rs` - GDT setup
- `vmm/src/boot/pagetable.rs` - Page table setup
- `vmm/src/boot/pvh.rs` - PVH boot structures
- `vmm/src/boot/linux.rs` - Linux boot params

---

## 10. Summary Table

| Feature | Firecracker | Volt | Status |
|---------|-------------|-----------|--------|
| CR0 | 0x80000011 | 0x8003003B | ⚠️ Volt has more flags |
| CR3 | 0x9000 | 0x1000 | ⚠️ Different |
| CR4 | 0x20 | 0x668 | ⚠️ Volt has more flags |
| EFER | 0x500 | 0x500 | ✅ Match |
| CS selector | 0x08 | 0x10 | ⚠️ Different |
| DS selector | 0x10 | 0x18 | ⚠️ Different |
| GDT location | 0x500 | 0x500 | ✅ Match |
| Stack pointer | 0x8ff0 | 0x8000 | ⚠️ Different |
| boot_params | 0x7000 | 0x20000 | ⚠️ Different |
| Kernel load | 0x100000 | 0x100000 | ✅ Match |
| FPU init | Yes | No | ❌ Missing |
| PVH 32-bit | Yes | No | ❌ Missing |
| High kernel map | No | Yes | ✅ Volt better |

---

*Document generated: 2026-03-08*
*Firecracker version: main branch*
*Volt version: current*