OOM problem in RGATConv

[songsong0425]

Hi, always thank you for your effort in the library maintenance.
I have a question about the OOM problem in RGATConv.
When I converted the original model using GATConv, it didn't evoke any problem and it was able to control using the hyperparameters, but RGATConv returned the OOM message as below:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 9.40 GiB. GPU 0 has a total capacty of 39.39 GiB of which 1.37 GiB is free. Process 3949644 has 524.00 MiB memory in use. Process 3323507 has 37.50 GiB memory in use. Of the allocated memory 36.92 GiB is allocated by PyTorch, and 97.95 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I guess that RGATConv saved the larger size of weights or parameters, and it evoked the problem. However, it's hard to solve the problem because the memory was allocated automatically during training.
Can anyone suggest any tips for this situation?

[akihironitta]

I'd suggest checking the number of parameters in your model and calculating the model size, or you could also try profiling your GPU memory with PyTorch profiler. https://pytorch.org/blog/understanding-gpu-memory-1/, https://pytorch.org/blog/understanding-gpu-memory-2/ might be helpful :)