Add `@huggingface/ollama-utils` #1111

ngxson · 2025-01-17T14:47:38Z

With the ollama compatibility layer already been up and running on HF hub for a while now, we want to open-source part of the integration for (1) provide more transparency, and (2) for encouraging the community to contribute.

Therefore, we decided to publish @huggingface/ollama-utils, a package containing tools that power this integration.

For now, the only tool that we provide is chat-template.ts, a tool that takes a parsed GGUF config from @huggingface/gguf as input, then returns the ollama Go template.

chat-template.ts

This module expose one single convertGGUFTemplateToOllama function. It works by trying (by order) these mechanisms:

Try to find a matched template already exist on ollama hub. To make this possible, we regularly dump a list of public templates from ollama.com using scripts/generate-automap.ts
If no exact match is find, we still check against the list of official templates, but now only match a list of special tokens. These tokens are extracted using regex RE_SPECIAL_TOKEN
If we still can't find a match, we search inside CUSTOM_TEMPLATE_MAPPING, a list of hand-picked template mapping
If all above still failed, we try parsing jinja template and try converting it to Go template. This works almost all of the time, except if the template is not supported by @huggingface/jinja. See convertJinjaToGoTemplate

packages/ollama-utils/README.md

coyotte508 · 2025-01-17T14:54:04Z

packages/ollama-utils/tsup.config.ts

If you feel up to it you can use tshy instead, like in @hugginface/tasks.

It has more accurate exports / better compatibility with all that includes the package (and as long as you don't have different code in browser/node it should be easy to configure)

I can't get around the error with this line:

src/chat-template.ts:1:43 - error TS1479: The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("@huggingface/jinja")' call instead. To convert this file to an ECMAScript module, change its file extension to '.mts' or create a local package.json file with `{ "type": "module" }`. 1 import { Template as JinjaTemplate } from "@huggingface/jinja"; ~~~~~~~~~~~~~~~~~~~~

Do you have any ideas @coyotte508 ?

Seems like @huggingface/jinja maybe needs to switch to tshy first to all corner cases (early/late node version, CJS importing MJS, MJS importing CJS, ...) are exported correctly cc @xenova

Strange, the exports should operate correctly

Maybe I need to restructure it like transformers.js does it here.

Do any other hf.js libs use tshy?

coyotte508 · 2025-01-17T16:29:34Z

btw the github action to publish should be added (with optionnally a check to check the @huggingface/jinja is up to date, like @huggingface/hub does with @huggingface/tasks)

julien-c

neat!

Vaibhavs10

don't know enough typescript to review, but two questions/comments from myside:

maybe in a follow-up PR we should add a scraper and something to update the mapping?
via comments/ README/ docs we should clarify how users can add their own exception as well

HuggingFaceDocBuilderDev · 2025-01-28T14:49:47Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Vaibhavs10

yay! conceptually lgtm!

Vaibhavs10 · 2025-01-29T15:02:45Z

packages/ollama-utils/README.md

+## How can I add a custom template?
+
+Most templates will be converted automatically. You can debug the output template using:
+- This space to retrieve the converted template: https://huggingface.co/spaces/ngxson/debug_ollama_manifest


maybe we can do both in the same space?

I couldn't figure out how to embed my script into gradio space, but will have a look later

(The template debugging space is static btw)

packages/ollama-utils/README.md

Co-authored-by: vb <[email protected]>

Vaibhavs10

merge + comms - today?

pcuenca

Looks good, suggesting a few nits

pcuenca · 2025-01-31T09:53:13Z

packages/ollama-utils/README.md

@@ -0,0 +1,56 @@
+# `@huggingface/ollama-utils`
+
+Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.


Suggested change

Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.

Various utilities for maintaining [Ollama compatibility with GGUF models on the Hugging Face Hub](https://huggingface.co/docs/hub/en/ollama).

For now, we are exposing chat template conversion to the Go format used by Ollama.

pcuenca · 2025-01-31T09:55:00Z

packages/ollama-utils/README.md

+- This space to retrieve the converted template: https://huggingface.co/spaces/ngxson/debug_ollama_manifest
+- And this space to apply the Go template into a list of messages: https://huggingface.co/spaces/ngxson/ollama_template_test
+
+Please only add a new template only when the conversion above is not successful. Cases that are acceptable to add a custom handler:


Suggested change

Please only add a new template only when the conversion above is not successful. Cases that are acceptable to add a custom handler:

Please only add a new template when the conversion process above is not successful. Cases that are acceptable include:

pcuenca · 2025-01-31T09:55:38Z

packages/ollama-utils/README.md

+To add a new custom handler:
+1. Edit the list of `CUSTOM_TEMPLATE_MAPPING` inside `chat-template.ts`
+2. Add a new test case in `chat-template.spec.ts`
+3. Push your change into new PR.


Suggested change

3. Push your change into new PR.

3. Push your change to a new PR.

pcuenca · 2025-01-31T09:56:56Z

README.md

@@ -62,6 +62,7 @@ This is a collection of JS libraries to interact with the Hugging Face API, with
 - [@huggingface/tasks](packages/tasks/README.md): The definition files and source-of-truth for the Hub's main primitives like pipeline tasks, model libraries, etc.
 - [@huggingface/jinja](packages/jinja/README.md): A minimalistic JS implementation of the Jinja templating engine, to be used for ML chat templates.
 - [@huggingface/space-header](packages/space-header/README.md): Use the Space `mini_header` outside Hugging Face
+- [@huggingface/ollama-utils](packages/ollama-utils/README.md): Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.


Suggested change

- [@huggingface/ollama-utils](packages/ollama-utils/README.md): Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.

- [@huggingface/ollama-utils](packages/ollama-utils/README.md): Various utilities for maintaining Ollama compatibility with models on the Hugging Face Hub.

pcuenca · 2025-01-31T10:00:47Z

packages/ollama-utils/README.md

+Documentation: https://huggingface.co/docs/hub/en/ollama
+


Suggested change

Documentation: https://huggingface.co/docs/hub/en/ollama

ngxson added 5 commits December 20, 2024 17:24

init ollama-utils

676e2f3

fix types

096231b

add test

98584ea

add more tests

8cee153

test ok

631eb5a

ngxson requested review from julien-c, coyotte508, XciD and Vaibhavs10 January 17, 2025 14:47

ngxson commented Jan 17, 2025

View reviewed changes

packages/ollama-utils/README.md Outdated Show resolved Hide resolved

coyotte508 approved these changes Jan 17, 2025

View reviewed changes

julien-c approved these changes Jan 17, 2025

View reviewed changes

Vaibhavs10 reviewed Jan 17, 2025

View reviewed changes

ngxson added 3 commits January 28, 2025 15:46

automap: preserve old list

777c442

automap: sync

897190d

fix lint

f259715

ngxson added 2 commits January 28, 2025 15:51

Merge branch 'main' into xsn/ollama_utils

8db9fd9

readme

31ad142

Vaibhavs10 approved these changes Jan 29, 2025

View reviewed changes

Vaibhavs10 reviewed Jan 29, 2025

View reviewed changes

packages/ollama-utils/README.md Outdated Show resolved Hide resolved

Vaibhavs10 reviewed Jan 29, 2025

View reviewed changes

packages/ollama-utils/README.md Outdated Show resolved Hide resolved

Vaibhavs10 reviewed Jan 29, 2025

View reviewed changes

packages/ollama-utils/README.md Outdated Show resolved Hide resolved

ngxson requested a review from Vaibhavs10 January 30, 2025 13:04

Apply suggestions from code review

43d97b4

Co-authored-by: vb <[email protected]>

Vaibhavs10 reviewed Jan 31, 2025

View reviewed changes

pcuenca reviewed Jan 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `@huggingface/ollama-utils` #1111

Add `@huggingface/ollama-utils` #1111

ngxson commented Jan 17, 2025

coyotte508 Jan 17, 2025

ngxson Jan 28, 2025

coyotte508 Jan 28, 2025 •

edited

Loading

xenova Jan 30, 2025

xenova Jan 30, 2025

coyotte508 commented Jan 17, 2025

julien-c left a comment

Vaibhavs10 left a comment

HuggingFaceDocBuilderDev commented Jan 28, 2025

Vaibhavs10 left a comment

Vaibhavs10 Jan 29, 2025

ngxson Jan 29, 2025

Vaibhavs10 left a comment

pcuenca left a comment

pcuenca Jan 31, 2025

pcuenca Jan 31, 2025

pcuenca Jan 31, 2025

pcuenca Jan 31, 2025

pcuenca Jan 31, 2025

		@@ -0,0 +1,56 @@
		# `@huggingface/ollama-utils`

		Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.

-Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.
+Various utilities for maintaining [Ollama compatibility with GGUF models on the Hugging Face Hub](https://huggingface.co/docs/hub/en/ollama).
+For now, we are exposing chat template conversion to the Go format used by Ollama.

	Please only add a new template only when the conversion above is not successful. Cases that are acceptable to add a custom handler:
	Please only add a new template when the conversion process above is not successful. Cases that are acceptable include:

	3. Push your change into new PR.
	3. Push your change to a new PR.

	- [@huggingface/ollama-utils](packages/ollama-utils/README.md): Various utilities for maintaining Ollama compatibility with models on Hugging Face hub.
	- [@huggingface/ollama-utils](packages/ollama-utils/README.md): Various utilities for maintaining Ollama compatibility with models on the Hugging Face Hub.

Add @huggingface/ollama-utils #1111

Are you sure you want to change the base?

Add @huggingface/ollama-utils #1111

Conversation

ngxson commented Jan 17, 2025

chat-template.ts

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coyotte508 Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coyotte508 commented Jan 17, 2025

julien-c left a comment

Choose a reason for hiding this comment

Vaibhavs10 left a comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Jan 28, 2025

Vaibhavs10 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vaibhavs10 left a comment

Choose a reason for hiding this comment

pcuenca left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add `@huggingface/ollama-utils` #1111

Add `@huggingface/ollama-utils` #1111

coyotte508 Jan 28, 2025 •

edited

Loading