LeapfrogAI Model Skeleton

This Zarf package contains the foundational configuration required to deploy a model backend to a Kubernetes cluster hosting LeapfrogAI.

This repository is a dedicated skeleton Zarf Package. The published skeleton artifact for this repo can be found here.

Examples

Some example repositories that use this skeleton are:

• defenseunicorns/llama-cpp-python
• defenseunicorns/whisper

Publishing a Package

A workflow already automatically deploys a new package at every tag push, but the manual way to do it is as follows:

zarf package publish . ghcr.io/defenseunicorns/packages/leapfrogai/leapfrogai-model:<IMAGE_TAG>

Testing a Package

To test a package locally, you first need a running cluster and an example model backend to be tested. After those are in place, execute the following:

# create a local registry
docker run -d -p 5000:5000 --restart=always --name registry registry:2

# publish the skeleton to the local registry
zarf package publish . oci://localhost:5000/defenseunicorns/packages/leapfrogai --insecure

# create model backend package using local skeleton
# this example will use leapfrogai-backend-whisper
# edit the import-model componenet to use image: localhost:5000/defenseunicorns/packages/leapfrogai/leapfrogai-model:<IMAGE_TAG>
zarf package create --insecure