Tracking Issue for Live Migration

[smklein]

Note: this issue is based on https://rfd.shared.oxide.computer/rfd/0071, and likely incomplete.
Please feel free to edit / modify it.

This issue attempts to, at a high-level, capture the necessary work items for LM. Depending on the size of sub-tasks, we can consider splitting out related issues, and dividing ownership.

• Protocol
  • Authenticated, encrypted TLS
  • Versions embedded in structures being transferred. We can start by being extremely selective, requiring exact version match.
  • Define Guest Machine Architecture (CPU features) and Platform (Device features) during instance creation
  • Advertise Guest Machine Architecture and Platform, ensure compatibility between source and destination. Related to Add 'platform version' to Instances omicron#615
• State Management: Does Propolis' architecture expose necessary interfaces for quiescing, serializing, and transfer of devices?
• Live Migration: RAM Transfer #82
  • Query / Offer dirty pages
  • Fetch full address space / Xfer full address space
  • Re-establish RAM in destination VM
• CPU Transfer
  • Registers
    • GP registers
    • MSRs
    • debug registers
    • control registers
  • Virtual APIC (ISA-defined registers, timers associated with registers, emulated acceleration (AVIC/APICv state)
  • Para-virtualized facilities (KVM or hyper-v para-virt clock)
  • TSC (offsets and freq) Migrate guest time-related data #359
  • HVM-specific state (interrupt injection, interrupt windowing, etcs)
    • "run-state", pending SIPI, etc
    • pending exception
    • pending interrupts
    • interrupt shadow interrupt-shadow state must be included in vcpu migration payload #384
• In-Kernel Emulated Peripherals
  • PIT (i8254)
  • PIC (i8259)
  • IOAPIC
  • HPET
  • RTC
  • ACPI Power Management Timer
• Userspace Emulated Devices
  • Virtio Devices
  • NVMe
  • Generalize Outstanding I/Os waiting logic for devices #98
• Miscellaneous Peripherals
  • Bootroom
  • Consumer-visible Interfaces
    • Framebuffer
    • KB / Mouse
    • ttya / ttyb
  • PCI state (hostbus, bridges)
• Live Migration: Block Devices #84
  • "Deactivate" message sent to Crucible
  • Re-inflation of Crucible upstairs on target
• Live Migration: Network Devices #83
  • "Pause" message sent to OPTE
  • Re-establishment of network traffic through OPTE

Related to oxidecomputer/omicron#252 in Omicron