error when PytorchStreamReader failed reading file

[yujunlhz]

python3 ./mmi_bigram_train.py

World size: 1 Rank: 0
Traceback (most recent call last):
File "./mmi_bigram_train.py", line 502, in
main()
File "./mmi_bigram_train.py", line 280, in main
lexicon = Lexicon(lang_dir)
File "/tmp/lingvo/snowfall/build/lib/snowfall/lexicon.py", line 21, in init
L_inv = k2.Fsa.from_dict(torch.load(self.lang_dir / 'Linv.pt'))
File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 592, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 851, in _load
result = unpickler.load()
File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 843, in persistent_load
load_tensor(data_type, size, key, _maybe_decode_ascii(location))
File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 831, in load_tensor
storage = zip_file.get_storage_from_record(name, size, dtype).storage()
RuntimeError: [enforce fail at inline_container.cc:145] . PytorchStreamReader failed reading file data/57885792: invalid header or archive is corrupted

k2 0.3.3+cu102.dev20210507
kaldialign 0.1.7
keyring 23.0.1
lhotse 0.6.0
lilcom 1.1.1
Markdown 3.3.4
numpy 1.20.2
torch 1.8.1+cu102
torchaudio 0.8.1

what might be the culprit?
Thanks

[danpovey]

I'm not sure what would cause this, but removing Linv.pt should work around it.

…

On Fri, May 7, 2021 at 11:18 PM yujunlhz ***@***.***> wrote: python3 ./mmi_bigram_train.py World size: 1 Rank: 0 Traceback (most recent call last): File "./mmi_bigram_train.py", line 502, in main() File "./mmi_bigram_train.py", line 280, in main lexicon = Lexicon(lang_dir) File "/tmp/lingvo/snowfall/build/lib/snowfall/lexicon.py", line 21, in *init* L_inv = k2.Fsa.from_dict(torch.load(self.lang_dir / 'Linv.pt')) File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 592, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 851, in _load result = unpickler.load() File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 843, in persistent_load load_tensor(data_type, size, key, _maybe_decode_ascii(location)) File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 831, in load_tensor storage = zip_file.get_storage_from_record(name, size, dtype).storage() RuntimeError: [enforce fail at inline_container.cc:145] . PytorchStreamReader failed reading file data/57885792: invalid header or archive is corrupted k2 0.3.3+cu102.dev20210507 kaldialign 0.1.7 keyring 23.0.1 lhotse 0.6.0 lilcom 1.1.1 Markdown 3.3.4 numpy 1.20.2 torch 1.8.1+cu102 torchaudio 0.8.1 what might be the culprit? Thanks — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#186>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO5A5IKZKQLR3VZC2BLTMQADZANCNFSM44KLCHLQ> .

[yujunlhz]

it turn out that Linv.pt was generated with a another version. thanks