Skip to content
This repository has been archived by the owner on Oct 9, 2024. It is now read-only.

root_dir in TemporaryCheckpointsJSON is redundant #82

Open
dc3671 opened this issue Apr 24, 2023 · 0 comments
Open

root_dir in TemporaryCheckpointsJSON is redundant #82

dc3671 opened this issue Apr 24, 2023 · 0 comments

Comments

@dc3671
Copy link

dc3671 commented Apr 24, 2023

In TemporaryCheckpointsJSON(https://github.com/huggingface/transformers-bloom-inference/blob/main/inference_server/models/ds_inference.py#L80) ,
image

When use glob.glob(f"{self.model_path}/*.bin"), files path in the list will all contain model_path prefix (eg: modelname is bigscience/bloom ).

{"type": "BLOOM", "checkpoints": ["bigscience/bloom/pytorch_model.bin"], "version": 1.0}

While set it as root_dir (glob.glob("*.bin", root_dir=self.model_path)) will not:

{"type": "BLOOM", "checkpoints": ["pytorch_model.bin"], "version": 1.0}

And it will align to DeepSpeed's loading way (replace_module.py). Because when loading, it will add root_dir again:

sd = [
  torch.load(os.path.join(base_dir1, checkpoint[i]), map_location='cpu')
]

So, current dump way will duplicate model_path.

And I raised a PR: #71 .

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant