FileNotFoundError: [Errno 2] No such file or directory: '/root/dolphin-2_6-phi-2/config.json' when running self build docker image

[OliverWalter]

When running a newly self build docker image, I'm getting the error message:
FileNotFoundError: [Errno 2] No such file or directory: '/root/dolphin-2_6-phi-2/config.json'

Some more context:

s6-rc: info: service legacy-services successfully started
/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/root/WhisperFusion/llm_service.py", line 205, in run
self.initialize_model(
File "/root/WhisperFusion/llm_service.py", line 98, in initialize_model
model_name = read_model_name(engine_dir)
File "/root/WhisperFusion/llm_service.py", line 23, in read_model_name
engine_version = tensorrt_llm.runtime.engine.get_engine_version(engine_dir)
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/runtime/engine.py", line 81, in get_engine_version
with open(config_path, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/root/dolphin-2_6-phi-2/config.json'
/usr/local/lib/python3.10/dist-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
[2024-02-09 13:49:45,605] [0/0] torch._dynamo.variables.torch: [WARNING] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored

I had to rebuild for compute capability 8.9 with:
bash build.sh 89-real

I can't immediately see any obvious error messages in the build (also not guaranteed that I missed something)
I used the master branch at c90a694

Could you provide an image with compute capability 8.9? Or any ideas for the fix? Should I use another checkout to rebuild the docker image?

[makaveli10]

Hello @OliverWalter, the docker image mentioned in the readme is actually for 4090 i.e. 89-real. So, you should be able to use:

 docker run --gpus all --shm-size 64G -p 6006:6006 -p 8888:8888 -it ghcr.io/collabora/whisperfusion:latest

[OliverWalter]

Hi @makaveli10, thanks for the reply. The Current image does not seem to run for me. It shows a different error (undefined symbol) See Issue #40 for details.