Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Dockerfile build for the converter only #423

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jondot
Copy link
Contributor

@jondot jondot commented Sep 26, 2023

Currently, converting an existing HF model requires having (1) a Rust environment ready, (2) rust-bert repo available and, (3) setting up a Python environment, just for the conversion.

For the use case where

(a) a Rust developer wants to utilize an HF model, they would need a Python environment
(b) a data scientist wants to experiment with different models, and a given Rust project that was created for them by Rust devs: they would need a Rust environment, and to set up a rust-bert repo

As it seems, the groups are mostly mutually exclusive.

I've created a Dockerfile, which I think is minimal, that only does the conversion. It:

  1. Builds the Rust project
  2. Sets up a python environment with the prebuilt Rust converter
  3. Takes a conversion command

And so, developers and data scientists need only to depend on Docker, and assuming the image is called rustbert-converter after it was built to only run:

docker run -v "$(pwd)"/<path to model on host>:/model rustbert-converter pytorch_mode.bin

The image expects a /model folder which is shared between the container and the host, where the raw pytorch model files are.

@guillaume-be
Copy link
Owner

Thank you @jondot - this is great!
The model conversion for Python is currently tested in the CI here, would it be possible to add a test using Docker as well? This would ensure everything still works as expected and serves as a nice documentation illustrating how to run conversion in the tests.

@jondot
Copy link
Contributor Author

jondot commented Sep 27, 2023

Sure, I can try. Do you mean we want to build the docker in the CI, and then run the docker to convert a sample model?

@guillaume-be
Copy link
Owner

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants