Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nvidia: GPU and display passthrough #944

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

jpruiz84
Copy link
Contributor

@jpruiz84 jpruiz84 commented Jan 7, 2025

Description of changes

  • Created a new microvm GPU-VM for Nvidia GPU/Display passthrough
  • Add nvidia-oot drivers patches to GPU-VM to support GPU/Display passthrough
  • Add GPU passthrough overlay to host dtb for support GPU/Display passthrough
  • Add GPU-VM device tree, and derivation to build it
  • Add patch to GPU-VM Linux kernel to hardcode the chipid

Checklist for things done

  • Summary of the proposed changes in the PR description
  • More detailed description in the commit message(s)
  • Commits are squashed into relevant entities - avoid a lot of minimal dev time commits in the PR
  • Contribution guidelines followed
  • Ghaf documentation updated with the commit - https://tiiuae.github.io/ghaf/
  • PR linked to architecture documentation and requirement(s) (ticket id)
  • Test procedure described (or includes tests). Select one or more:
    • Tested on Lenovo X1 x86_64
    • Tested on Jetson Orin NX or AGX aarch64
    • Tested on Polarfire riscv64
  • Author has run make-checks and it passes
  • All automatic Github Action checks pass - see actions
  • Author has added reviewers and removed PR draft status
  • Change requires full re-installation
  • Change can be updated with nixos-rebuild ... switch

Instructions for Testing

  • List all targets that this applies to: Nvidia Orin AGX, Nvidia Orin NX

  • Is this a new feature

    • List the test steps to verify:
      • Test GPU is present in GPU-VM:
        • Access to the gpu-vm
          ssh gpu-vm-debug
        • Run tegrastats to check if GPU is present
          [ghaf@gpu-vm:~]$ tegrastats 
          01-27-2025 11:04:46 RAM 144/3578MB (lfb 760x4MB) CPU [0%,0%,6%,0%] GR3D_FREQ 0%  
          
          If you see GR3D_FREQ 0% that means that the GPU driver is working correctly with
          the GPU passthrough
  • If it is an improvement how does it impact existing functionality?

Tanel Dettenborn and others added 8 commits December 9, 2024 12:35
- Using TII jetpack-nixos fork (fork has Jetson 36.3 with upstream Linux 6.6)
- Virtualization and bpmp is false
- Only NX target

Signed-off-by: Tanel Dettenborn <[email protected]>
- Disabled PCI passthrough until nvidia-oot device tree get integrated.

Signed-off-by: Juan Pablo Ruiz <[email protected]>
Signed-off-by: Tanel Dettenborn <[email protected]>
Signed-off-by: Tanel Dettenborn <[email protected]>
@jpruiz84 jpruiz84 temporarily deployed to internal-build-workflow January 7, 2025 13:28 — with GitHub Actions Inactive
Also, added dtb overlays for gpu_passthrough

Signed-off-by: Juan Pablo Ruiz <[email protected]>
Signed-off-by: Juan Pablo Ruiz <[email protected]>
Set the hardware.deviceTree.dtbSource to the NVIDIA jetpack bsp.
In this way the fdtPath in sdimage.nix point to the
config.hardware.deviceTree.package that is configured
through  hardware.deviceTree.dtbSource.

Signed-off-by: Juan Pablo Ruiz <[email protected]>
- Create a new microvm GPU-VM
- Add nvidia-oot drivers patches to support GPU/Display passthrough
- Add GPU passthrough overlay to host dtb for support passthrough
- Add GPU-VM device tree, and derivation to build it
- Add patch to GPU-VM Linux kernel to hardcode the chipid

Signed-off-by: Juan Pablo Ruiz <[email protected]>
@jpruiz84 jpruiz84 force-pushed the gpu_pt_kernel6_6_wip branch from 9107531 to 4f114d5 Compare January 23, 2025 14:09
@jpruiz84 jpruiz84 temporarily deployed to internal-build-workflow January 23, 2025 14:09 — with GitHub Actions Inactive
Add udev rule to assign kvm group to bpmp-host chardev

Signed-off-by: Juan Pablo Ruiz <[email protected]>
@jpruiz84 jpruiz84 temporarily deployed to internal-build-workflow January 24, 2025 11:04 — with GitHub Actions Inactive
Signed-off-by: Juan Pablo Ruiz <[email protected]>
@jpruiz84 jpruiz84 temporarily deployed to internal-build-workflow January 27, 2025 08:36 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants