Rogue Carrier Support for AGX Xavier Jetson

[frankvp11]

In this issue and this fork of the repo https://github.com/frankvp11/balena-jetson I managed to get balena-os working on the CTI rogue carrier board for L4T r32.7.1.
However, I am looking to push this support to Jetpack 5 (L4T r35.4.1).
I tested a few things, which I will outline here:

1. I tried building + flashing for a regular Jetson Xavier using ./balena-yocto-scripts/build/balena-build.sh -d jetson-xavier -s, and the flashing script with this PR . This managed to build + flash, however I wasn't able to get the GPU working. On the host OS, cat /etc/nv_tegra_release didn't exist.
2. Then, I tried changing the .dtb file from here to use this one tegra194-agx-cti-AGX101.dtb as that followed what appeared to be the main changes in L4T r32.7.1. However, that was unable to build the image.
   This was the main error:

ERROR: jetson-dtbs-1.0-r0 do_package: QA Issue: jetson-dtbs: Files/directories were installed but not shipped in any package:
  /boot
  /boot/tegra194-p2888-0001-p2822-0000.dtb
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
jetson-dtbs: 2 installed and not shipped files. [installed-vs-shipped]
ERROR: jetson-dtbs-1.0-r0 do_package: Fatal QA errors were found, failing task.
ERROR: Logfile of failure stored in: /work/build/tmp/work/armv8a-poky-linux/jetson-dtbs/1.0-r0/temp/log.do_package.74646

Does anyone have any ideas on what I am missing to get this working? I also tried modifying the patch file to use my .dtb but that failed aswell.

[patrickwasp]

I'm working with Frank on this. Below is some additional context

Past efforts to get this working on older versions of Jetpack:

• https://forums.balena.io/t/support-connect-tech-rogue-agx-carrier-board-for-jetson-xavier/192168/13
• Support for Rogue AGX 101 Carrier board balena-jetson#104

Additional details from the manufacturer of the carrier board, CTI.

BSP for JetPack 5.1.2 - L4T r35.4.1: https://connecttech.com/ftp/Drivers/CTI-L4T-AGX-35.4.1-V004.tgz

The release notes (https://connecttech.com/ftp/Drivers/L4T-Release-Notes/Jetson-AGX-Xavier/AGX-35.4.1%20.pdf) include the installation instructions below, when not using balena. And this is the install.sh script that is used during the process.

#!/bin/bash

COMPAT_MAJOR=("35")
COMPAT_MINOR=("4.1")

COMPAT_PROFILES=("Rogue, RogueX, Rogue-10G, RogueX2, Rudi-AGX" )

COMPAT_JETSON=("Xavier" "Xavier-I")

CTI_L4T_VER="AGX-35.4.1-V004"

#utility functions
chkroot(){

    if [ $EUID -ne 0 ]
    then
        echo "Please run as sudo or root"
        exit 1
    fi
}

#check if bsp installed in the correct folder on host machine
chkfolder(){
    
    if [ $(basename $(dirname $(pwd))) != "Linux_for_Tegra" ] ||
       [ $(basename $(pwd)) != "CTI-L4T" ]; then
        echo "This BSP should be extracted in the Linux_for_Tegra directory"
        echo "of your Jetpack install. This install script  must be run from the"
        echo "Linux_for_Tegra/CTI-L4T directory"
        exit 2
    fi

}

install_extra(){
    rm ../nv_tegra/l4t_deb_packages/nvidia-l4t-apt-source_*
	rm ../nv_tegra/l4t_deb_packages/nvidia-l4t-oem-config_*
    rm ../nv_tegra/l4t_deb_packages/nvidia-l4t-camera_*
	cp ./extra/*.deb ../nv_tegra/l4t_deb_packages/
    return 0
}

install_flash_files(){
    # install .conf
    cp -r ./conf/* ../

    # install cti-flash.sh
    cp cti-flash.sh ..
    chmod +x ../cti-flash.sh
}


install_kernel(){
    # remove old debs 
    rm ../kernel/*.deb
    
    # copy kernel debs
    cp ./kernel/*.deb ../kernel

    # install Image
    cp ./kernel/Image ../kernel

    # install dtbs
    cp ./kernel/dtb/* ../kernel/dtb/

}


install_bl(){
	cp -r bl/* ../bootloader
}

install_ver_file(){

    if [ ! -d "../rootfs/etc/cti" ]; then
        mkdir ../rootfs/etc/cti
    fi  

    echo $CTI_L4T_VER > ../rootfs/etc/cti/CTI-L4T.version 

}

print_versions(){

    echo "Supported Version Information"
    echo "  CTI-L4T Board Support Package Version:" $CTI_L4T_VER

    echo "  Supported CTI Products (profiles): "
    for i in ${COMPAT_PROFILES[@]}; do
        echo "    " $i
    done

    echo "  Supported Linux for Tegra Release Versions: "
    for j in ${COMPAT_MINOR[@]}; do
        echo "    " $COMPAT_MAJOR"."$j
    done

    echo "  Supported Nvidia Jetson Module Hardware: "
    for k in ${COMPAT_JETSON[@]}; do
        echo "    " $k
    done

}

#help functions 

usage(){
    echo -e "  Usage: ./install.sh "
    echo -e "  Before installing this BSP, please read the included readme.txt"
    echo -e "  Below are the compatible L4T versions and board profiles:\n"

    print_versions
}

#used to make rootfs changes before flashing
#primarily implemented because moving AX210 PNVM file from deb package
#during applied binaries leads to Qemu unsupported syscall warning
apply_rootfs_patch(){
    AX210_PNVM="../rootfs/lib/firmware/iwlwifi-ty-a0-gf-a0.pnvm"
    if [ -f "${AX210_PNVM}"  ]; then
        mv ${AX210_PNVM} ${AX210_PNVM}.bak
    fi
}


#start
usage

chkroot
chkfolder

install_flash_files

install_bl

install_kernel

install_ver_file

install_extra

apply_rootfs_patch

pushd ..

./apply_binaries.sh 

popd

# ensure the files finish copying 
sync

echo "CTI-L4T-$CTI_L4T_VER Installed!"

Installing Jetpack for Jetson AGX Xavier™

Steps to Install Jetpack

Step 1: Create Directory for Installation

Create a new directory for installing Jetpack, referred to as <BSP_ROOT> in these instructions.

Step 2: Access Jetpack Release Page

Visit the Jetpack Release Page: Jetpack Release Page

Step 3: Download Required Files

Download the following files for AGX Xavier™:

• L4T Driver Package (BSP)
• Sample Root Filesystem

Step 4: Place Files in <BSP_ROOT>

Move the downloaded files to <BSP_ROOT>. Afterward, your <BSP_ROOT> directory should contain:

• jetson_linux_r35.4.1_aarch64.tbz2
• tegra_linux_sample-root-filesystem_r35.4.1_aarch64.tbz2

Step 5: Extract "L4T Driver Package"

Run the following commands:

cd <BSP_ROOT>
sudo tar -jxf jetson_linux_r35.4.1_aarch64.tbz2

Step 6: Extract "Sample Root Filesystem"

Extract the Sample Root Filesystem into Linux_for_Tegra/rootfs:

sudo tar -C Linux_for_Tegra/rootfs/ -xjf tegra_linux_sample-root-filesystem_r35.4.1_aarch64.tbz2

Installing CTI BSP

Step 1: Copy the CTI BSP Package

Copy the CTI-L4T-AGX-35.4.1-V###.tgz package to <BSP_ROOT>/Linux_for_Tegra. For example:

cp CTI-L4T-AGX-35.4.1-V###.tgz <BSP_ROOT>/Linux_for_Tegra

If using NVIDIA’s SDK Manager, <BSP_ROOT> will be:

~/nvidia/nvidia_sdk/<JetPack_Version>_Linux_JETSON_XAVIER_NX_TARGETS/

Otherwise, it will be the directory you created earlier.

Step 2: Extract the BSP

cd <BSP_ROOT>/Linux_for_Tegra
sudo tar -xzf CTI-L4T-AGX-35.4.1-V###.tgz

Step 3: Navigate to CTI-L4T Directory

cd <BSP_ROOT>/Linux_for_Tegra/CTI-L4T

Step 4: Run Install Script

Run the install script to automatically install the BSP files to the correct locations:

sudo ./install.sh

Step 5: Return to Linux_for_Tegra

cd ..

The CTI-L4T BSP is now installed on the host system and ready to flash the AGX module.

Flashing Jetson AGX Xavier™ Modules

Step 1: Connect Jetson to Computer

Connect the Jetson AGX Xavier™ and Carrier to the computer via USB, following the instructions in the appropriate manual.

Step 2: Enter Recovery Mode

Put the system into recovery mode as per the instructions in the appropriate manual.

Step 3: Flash the Device

Use the following command to flash the Jetson AGX Xavier™. Replace <module>, <boardname>, and <config> with appropriate values (do not add .conf to <config>):

./flash.sh cti/<module>/<boardname>/<config> mmcblk0p1

For Xavier modules, <module> is xavier.

Example Command

./flash.sh cti/xavier/rogue/base mmcblk0p1

Step 4: Reboot

Once flashing is complete, the Xavier will reboot automatically.

[patrickwasp]

@acostach does Claude have the right idea about what it takes to integrate a new carrier board?

Based on the provided information, I'll analyze the issue and propose a solution for adding support for the CTI Rogue carrier board in balenaOS for L4T r35.4.1 (Jetpack 5).
Key Points from Analysis:

1. The board has previously worked with balenaOS on L4T r32.7.1
2. The current blocker is integrating the custom DTB (tegra194-agx-cti-AGX101.dtb)
3. The error occurs in the jetson-dtbs recipe due to incorrect FILES packaging

Proposed Solution:

1. Create a new machine configuration variant for the CTI Rogue:

# layers/meta-balena-jetson/conf/machine/jetson-xavier-cti-rogue.conf
#@TYPE: Machine
#@NAME: CTI Rogue carrier for Jetson AGX Xavier
#@DESCRIPTION: Machine configuration for the CTI Rogue carrier with Jetson AGX Xavier

MACHINEOVERRIDES = "jetson-agx-xavier-devkit:${MACHINE}"
include conf/machine/jetson-agx-xavier-devkit.conf

# Override the default DTB
KERNEL_DEVICETREE = "nvidia/tegra194-agx-cti-AGX101.dtb"

2. Modify the jetson-dtbs recipe to handle the new DTB:

# layers/meta-balena-jetson/recipes-kernel/linux/jetson-dtbs.bb

# Add device-specific FILES packaging
FILES:${PN}:jetson-xavier-cti-rogue += " \
    /boot/tegra194-agx-cti-AGX101.dtb \
"

do_install:jetson-xavier-cti-rogue() {
    install -d ${D}/boot/
    install -m 0644 "${DEPLOY_DIR_IMAGE}/tegra194-agx-cti-AGX101.dtb" "${D}/boot/"
}

3. Add a new coffee file for the board:

# jetson-xavier-cti-rogue.coffee
deviceTypesCommon = require '@resin.io/device-types/common'
{ networkOptions, commonImg, instructions } = deviceTypesCommon

module.exports =
    version: 1
    slug: 'jetson-xavier-cti-rogue'
    name: 'Jetson AGX Xavier CTI Rogue'
    arch: 'aarch64'
    state: 'experimental'

    # Rest of the configuration similar to jetson-xavier.coffee
    instructions: [
        'Put the carrier board in recovery mode',
        'Unzip BalenaOS image and use Jetson Flash to provision the device.',
        'After flashing is completed, please wait until the board is rebooted'
    ]

    gettingStartedLink:
        windows: 'https://docs.balena.io/jetson-xavier/nodejs/getting-started/'
        osx: 'https://docs.balena.io/jetson-xavier/nodejs/getting-started/'
        linux: 'https://docs.balena.io/jetson-xavier/nodejs/getting-started/'

    supportsBlink: false

    yocto:
        machine: 'jetson-xavier-cti-rogue'
        image: 'balena-image'
        fstype: 'balenaos-img'
        version: 'yocto-kirkstone'
        deployArtifact: 'balena-image-jetson-xavier-cti-rogue.balenaos-img'
        compressed: true

    options: [ networkOptions.group ]

    configuration:
        config:
            partition:
                primary: 37
            path: '/config.json'

    initialization: commonImg.initialization

4. Add the CTI BSP files:

# layers/meta-balena-jetson/recipes-bsp/cti-tegra-binaries/cti-tegra-binaries_35.4.1.bb
SUMMARY = "CTI BSP binaries for Jetson platforms"
DESCRIPTION = "CTI BSP binaries required for flashing"
LICENSE = "Apache-2.0"
LIC_FILES_CHKSUM = "file://${BALENA_COREBASE}/COPYING.Apache-2.0;md5=89aea4e17d99a7cacdbeed46a0096b10"

SRC_URI = "https://connecttech.com/ftp/Drivers/CTI-L4T-AGX-35.4.1-V004.tgz"
SRC_URI[sha256sum] = "INSERT_ACTUAL_CHECKSUM"

COMPATIBLE_MACHINE = "jetson-xavier-cti-rogue"

inherit deploy

do_configure[noexec] = "1"
do_compile[noexec] = "1"

do_install() {
    install -d ${D}/opt/cti/
    cp -r ${WORKDIR}/CTI-L4T/* ${D}/opt/cti/

    # Install kernel DTB
    install -d ${D}/boot
    install -m 0644 ${WORKDIR}/CTI-L4T/kernel/dtb/tegra194-agx-cti-AGX101.dtb ${D}/boot/
}

do_deploy() {
    install -d ${DEPLOYDIR}/cti-binaries
    
    # Flash configuration files
    cp -r ${WORKDIR}/CTI-L4T/conf/* ${DEPLOYDIR}/cti-binaries/
    
    # Boot loader binaries
    cp -r ${WORKDIR}/CTI-L4T/bl/* ${DEPLOYDIR}/cti-binaries/bootloader/
    
    # Extra device-specific files
    cp -r ${WORKDIR}/CTI-L4T/extra/*.deb ${DEPLOYDIR}/cti-binaries/
}

FILES:${PN} = " \
    /opt/cti/* \
    /boot/tegra194-agx-cti-AGX101.dtb \
"

addtask deploy before do_build after do_install

Then create a bbappend for tegra-binaries to integrate CTI files:

# layers/meta-balena-jetson/recipes-bsp/tegra-binaries/tegra-binaries_%.bbappend
FILESEXTRAPATHS:prepend:jetson-xavier-cti-rogue := "${THISDIR}/files:"

# Add CTI-specific flash configuration
PARTITION_LAYOUT_TEMPLATE:jetson-xavier-cti-rogue = "flash_${NVIDIA_BOARD}_custom.xml"

do_install:append:jetson-xavier-cti-rogue() {
    # Install CTI-specific flash configurations
    install -m 0644 ${DEPLOY_DIR_IMAGE}/cti-binaries/flash_${NVIDIA_BOARD}_custom.xml ${D}${datadir}/tegraflash/
    
    # Install additional bootloader files
    cp -r ${DEPLOY_DIR_IMAGE}/cti-binaries/bootloader/* ${D}${datadir}/tegraflash/
}

5. Updating Flash Scripts:

# layers/meta-balena-jetson/recipes-support/tegra-flash-dry/files/flash-cti-rogue.xml
<?xml version="1.0"?>
<partition_layout version="01.00.0000">
    <device type="sd" instance="0" sector_size="512" num_sectors="61071360">
        <partition name="master_boot_record" type="protective_master_boot_record">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 512 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 8 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
        </partition>
        
        <!-- CTI-specific partition layout -->
        <partition name="primary_gpt" type="primary_gpt">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 19968 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 8 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
        </partition>

        <!-- Add other required partitions -->
    </device>
</partition_layout>

Update the flash script handling:

# layers/meta-balena-jetson/recipes-support/tegra-flash-dry/tegra-flash-dry_35.4.1.bbappend
FILESEXTRAPATHS:prepend:jetson-xavier-cti-rogue := "${THISDIR}/files:"

SRC_URI:append:jetson-xavier-cti-rogue = " \
    file://flash-cti-rogue.xml \
    file://partition_specification_cti_rogue.txt \
"

do_install:append:jetson-xavier-cti-rogue() {
    install -m 0644 ${WORKDIR}/flash-cti-rogue.xml ${D}${bindir}/
    install -m 0644 ${WORKDIR}/partition_specification_cti_rogue.txt ${D}${bindir}/
}

Create a new hostapp update hook for the CTI Rogue:

# layers/meta-balena-jetson/recipes-support/hostapp-update-hooks/files/98-flash-bootfiles-cti-rogue
#!/bin/sh
set -o errexit

# Source balena-config-vars to get config values
. /usr/sbin/balena-config-vars

# Flash CTI-specific bootloader files
flash_cti_bootloader() {
    # Backup current boot config
    cp /boot/extlinux/extlinux.conf /tmp/

    # Flash bootloader using CTI-specific method
    if [ -x /opt/cti/flash-bootloader.sh ]; then
        /opt/cti/flash-bootloader.sh
    fi

    # Restore boot config
    mv /tmp/extlinux.conf /boot/extlinux/
}

# Main update process
update_cti_bootloader() {
    # Get current root partition index
    current_root_index=`findmnt --noheadings --output PKNAME --target / | sed -e 's/.*[^0-9]\([0-9]\+\)$/\1/'`

    # Update bootloader files
    flash_cti_bootloader

    # Update device tree
    if [ -f /opt/cti/tegra194-agx-cti-AGX101.dtb ]; then
        cp /opt/cti/tegra194-agx-cti-AGX101.dtb /boot/
    fi

    sync
}

update_cti_bootloader

Add the new hook to hostapp-update-hooks recipe:

# layers/meta-balena-jetson/recipes-support/hostapp-update-hooks/hostapp-update-hooks.bbappend
FILESEXTRAPATHS:append := ":${THISDIR}/files"

HOSTAPP_HOOKS:append:jetson-xavier-cti-rogue = " \
    98-flash-bootfiles-cti-rogue \
"

[acostach]

Hi @patrickwasp and @frankvp11 , apart from the integration in the device repository, to properly support this carrier board in balena-cloud it will need to also be placed in an automated testing setup.

Would you mind dropping us a line at https://www.balena.io/contact to tell us more about your use-case, so we can better assist you? Thank you!

[patrickwasp]

This is just a one off so we're not looking for official support or integration into the UI for flashing. But we're very interested in learning about how we could get custom hardware working for r&d projects. For just a one off, are the steps above reasonable?

[frankvp11]

Is there any updates on this @acostach?

[acostach]

HI @frankvp11 , steps 1 -> 3 look ok. From that point on, you'd probably want to patch the bootloader to use the Rogue kernel DTB by default: https://github.com/balena-os/balena-jetson-jp5/blob/master/layers/meta-balena-jetson/recipes-bsp/uefi/edk2-firmware-tegra/0005-Add-hup-and-rollback-support-agx-xavier-35-4-1.patch#L572

Also, you can use this branch https://github.com/balena-os/jetson-flash/commits/xavier_agx_nx_35_4_1_support for flashing the device, with slight changes to replace any custom files the CTI BSP may use for the Rogue board. You'll first want to isolate those files that are custom in the CTI BSP, and copy them in the unpacked BSP the similar to the way the custom UEFI fimware is replaced: https://github.com/balena-os/jetson-flash/blob/0382bde6726580a5af2364c047f8de8078569059/lib/resin-jetson-flash.js#L216

You'll also want to build an UEFI capsule if you want to do hostOS updates, to replace the Devkit one: https://github.com/balena-os/balena-jetson-jp5/blob/master/layers/meta-balena-jetson/recipes-bsp/tegra-binaries/tegra-flash-dry_35.4.1.bb#L18

This can be done using the steps here: https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/Bootloader/UpdateAndRedundancy.html#generating-a-multi-spec-bup , with the default Nvidia setup in which you replace uefi_jetson.bin with the one you built - or re-use the one that the jetson-flash branch already provides, as well as any other files BSP files that are particular to the CTI Rogue board and which are provided by the CTI BSP.

You should ignore the boot blob here: https://github.com/balena-os/balena-jetson-jp5/blob/master/layers/meta-balena-jetson/recipes-bsp/tegra-binaries/tegra-flash-dry_35.4.1.bb#L28 because you won't be using that for migrating from an older Jetpack version.

Also, you could create a separate hook for this particular board, which would be identical to the AGX Xavier, except that it would use the new UEFI capsule which you built: https://github.com/balena-os/balena-jetson-jp5/blob/master/layers/meta-balena-jetson/recipes-support/hostapp-update-hooks/files/98-resin-bootfiles-jetson-xavier

You can then use that specifically for the CTI machine in https://github.com/balena-os/balena-jetson-jp5/blob/master/layers/meta-balena-jetson/recipes-support/hostapp-update-hooks/hostapp-update-hooks.bbappend#L10