BUG: Overwriting with DataParallel

[EWitting]

We automatically enable Cuda and Dataparallel when 1 or more gpus are available. This is reasonable, since you can just set CUDA_VISIBLE_DEVICES as environment variable, although it makes testing less convenient.

The main problem is that we overwrite self.model with the dataparallel model. This means that self.model can be either the architecture directly, or a wrapped architecture depending on whether you're using gpu. This sometimes makes it more complicated when trying to access part of the architecture.

Most importantly, when saving self.model, it also saves the parallel wrapper, which references cuda:0 and cuda:1. If the same model is loaded on a PC with a different number of gpus, this crashes, and model.cpu() doesn't work either. Solution would probably be to separately track self.module, with the architecture and weights for saving and loading, and self.model for training and inference, which is set in the post_init depending on hardware.