v0.0.8.4

What's New ✨

• Added support for Qwen2.5, Qwen2.5-code, and Qwen2.5-Math

Install Nexa SDK 🛠️

CPU Installation

To install the CPU version of Nexa SDK, run:

pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra-index-url https://pypi.org/simple --no-cache-dir

GPU Installation (Metal - macOS)

For the GPU version supporting Metal (macOS), run:

CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

For detailed installation instructions of Nexa SDK for CUDA and AMD GPU support, please refer to the Installation section in the main README.

To update your current SDK version to v0.0.8.4, use the same command as the installation but add a -U flag to the pip install command.

Run Qwen2.5 with Nexa SDK

Option 1: Run official GGUF files from Qwen HuggingFace Page 🤗

You could use the following command to pull and run language models in GGUF format from 🤗 HuggingFace: nexa run -hf <hf model id>. Choose one of these commands based on your preferred model size:

Qwen2.5 0.5B:

nexa run -hf Qwen/Qwen2.5-0.5B-Instruct-GGUF

Qwen2.5 1.5B:

nexa run -hf Qwen/Qwen2.5-1.5B-Instruct-GGUF

Qwen2.5 3B:

nexa run -hf Qwen/Qwen2.5-3B-Instruct-GGUF

Qwen2.5 7B:

nexa run -hf Qwen/Qwen2.5-7B-Instruct-GGUF

Qwen2.5 14B:

nexa run -hf Qwen/Qwen2.5-14B-Instruct-GGUF

The command line will prompt you to select one file from different quantization options. Use the number to indicate your choice. If you're unsure which one to choose, try "q4_0.gguf".

You will then have Qwen2.5 running locally on your computer.

Note: For Qwen2.5-code and Qwen2.5-Math, there are no official GGUF files available. Please use Option 2 for these models.

Option 2: Pull and Run Qwen2.5, Qwen2.5-code, Qwen2.5-Math from Nexa Model Hub 🐙

We have converted and uploaded the following models to the Nexa Model Hub:

Model	Nexa Run Command
Qwen2.5 0.5B	nexa run Qwen2.5-0.5B-Instruct:q4_0
Qwen2.5 1.5B	nexa run Qwen2.5-1.5B-Instruct:q4_0
Qwen2.5 3B	nexa run Qwen2.5-3B-Instruct:q4_0
Qwen2.5-code	nexa run Qwen2.5-Coder-1.5B-Instruct:q4_0
Qwen2.5-Math	nexa run Qwen2.5-Math-1.5B-Instruct:q4_0

Visit the model pages to choose your parameters and quantization preference. We will constantly upload and support more models in the Qwen2.5 family.

Please feel free to share your feedback and feature/model requests on the issue page.

v0.0.8.3-metal

What's New ✨

• Added image generation model support: SD3 and Flux (#75)
• Added NLP model support: OpenELM and Phi3.5
• Implemented logits through API (#67)

Improvements 🔧

• Added more SDK development examples (examples)
• Added RoCM support for AMD GPUs (#90)

Fixes 🐞

• Fixed server issue: execute curl command on Windows (#79)
• Fixed nanoLlava file mapping issue (#68)

Upgrade Guide 📝

To upgrade the NexaAI SDK for GPU use with Metal on macOS, follow these steps:

1. Open a terminal.

2. Run the following command:

CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install -U nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

This command will upgrade your existing NexaAI SDK installation to the latest Metal-compatible version.

Note

If you encounter any issues or want to ensure a clean installation:

1. Uninstall the current version:

   pip uninstall nexaai

2. Reinstall the package using this command:

   CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

For more detailed installation instructions, please refer to the Installation section in the README.

Full Changelog - v0.0.8.2...v0.0.8.3

v0.0.8.3-cu124

What's New ✨

• Added image generation model support: SD3 and Flux (#75)
• Added NLP model support: OpenELM and Phi3.5
• Implemented logits through API (#67)

Improvements 🔧

• Added more SDK development examples (examples)
• Added RoCM support for AMD GPUs (#90)

Fixes 🐞

• Fixed server issue: execute curl command on Windows (#79)
• Fixed nanoLlava file mapping issue (#68)

Upgrade Guide 📝

To upgrade the NexaAI SDK for GPU use with CUDA, follow these steps based on your operating system:

For Linux:

CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install -U nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows:

• PowerShell:

$env:CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON"; pip install -U nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

• Command Prompt:

set CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" & pip install -U nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

• Git Bash:

CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install -U nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

These commands will upgrade your existing NexaAI SDK installation to the latest CUDA-compatible version.

Note

If you encounter any issues or want to ensure a clean installation:

1. Uninstall the current version:

   pip uninstall nexaai

2. Reinstall the package using the appropriate command for your system as listed above, but without the -U flag.

For more detailed installation instructions, please refer to the Installation section in the README.

Full Changelog - v0.0.8.2...v0.0.8.3

v0.0.8.3

What's New ✨

• Added image generation model support: SD3 and Flux (#75)
• Added NLP model support: OpenELM and Phi3.5
• Implemented logits through API (#67)

Improvements 🔧

• Added more SDK development examples (examples)
• Added RoCM support for AMD GPUs (#90)

Fixes 🐞

• Fixed server issue: execute curl command on Windows (#79)
• Fixed nanoLlava file mapping issue (#68)

Upgrade Guide 📝

To upgrade the NexaAI SDK CPU version , follow these steps:

1. Open a terminal or command prompt.

2. Run the following command:

pip install -U nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra-index-url https://pypi.org/simple --no-cache-dir

This command will upgrade your existing NexaAI SDK installation to the latest CPU-compatible version.

Note

If you encounter any issues or want to ensure a clean installation:

1. Uninstall the current version:

   pip uninstall nexaai

2. Reinstall the package using this command:

   pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra-index-url https://pypi.org/simple --no-cache-dir

For more detailed installation instructions, please refer to the Installation section in the README.

Full Changelog - v0.0.8.2...v0.0.8.3

v0.0.8.2-metal

Merge pull request #72 from NexaAI/zack-dev

Update installation instructions

v0.0.8.2-cu124

Merge pull request #72 from NexaAI/zack-dev

Update installation instructions

v0.0.8.2

Merge pull request #72 from NexaAI/zack-dev

Update installation instructions

v0.0.8.1-metal

Merge pull request #65 from NexaAI/zack-dev

update tag extraction logic

v0.0.8.1-cu124

Merge pull request #65 from NexaAI/zack-dev

update tag extraction logic

v0.0.8.1

Merge pull request #65 from NexaAI/zack-dev

update tag extraction logic