DefaultCPUAllocator: can't allocate memory: you tried to allocate 1589575680 bytes.

[wangsu-google-language]

Hi Jacob,

Following the setup instructions in https://github.com/jacobkrantz/VLN-CE to run the model, with

python run.py --exp-config=vlnce_baselines/config/rxr_baselines/rxr_cma_en.yml --run-type=train

I got the following error:

  File "/home/wangsu/anaconda3/envs/vlnce_py3.6_h0.1.7/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wangsu/KrantzVLNCE/habitat-lab/VLN-CE/vlnce_baselines/models/encoders/resnet_encoders.py", line 199, in forward
    resnet_output = self.cnn(normalize(rgb_observations))
  File "/home/wangsu/anaconda3/envs/vlnce_py3.6_h0.1.7/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wangsu/anaconda3/envs/vlnce_py3.6_h0.1.7/lib/python3.6/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/home/wangsu/anaconda3/envs/vlnce_py3.6_h0.1.7/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wangsu/anaconda3/envs/vlnce_py3.6_h0.1.7/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 446, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/wangsu/anaconda3/envs/vlnce_py3.6_h0.1.7/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 443, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 1589575680 bytes. Error code 12 (Cannot allocate memory)

My machine however does have enough memory:

              total        used        free      shared  buff/cache   available
Mem:    27390640128   574816256 25532850176     8773632  1282973696 26407305216

Could you help look into this please? Thanks!