Multi-GPU training for DGL+ALIGNN

[knc6]

❓ Questions and Help

Hi,

Here is an issue regarding multi-GPU usage case for DGL for atomistic prediction in ALIGNN: usnistgov/alignn#90

We implemented the DDP feature, but still the RAM memory cost is a challenge. Any thoughts/suggestions how to tackle this issue efficiently?

[rudongyu]

Hi @knc6 , original dataset in older dgl put all features and graph structures in memory, and I guess your dataset is pretty large, which makes the RAM memory cost huge. You may try the graphbolt in the newest version of dgl: https://docs.dgl.ai/stochastic_training/ondisk-dataset.html. It supports on-disk storage of features.

[knc6]

Thanks for the recommendation. I found that LMDB worked well dealing with large dataset.