You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The reading code in extra.components tries to work around HDF problems with respect so slicing and thus reads entire datasets into memory for subsequent slicing. This relies on virtual memory tricks allowing to allocate potentially very large amounts of memory, even overcomitting physical memory.
This has shown to fail on shared nodes under high memory pressure, potentially due to different settings. Selecting down a detector object to a single train and pulse still caused MultimodKeyData.ndarray() to allocate an entire sequence worth of memory, which failed.
The text was updated successfully, but these errors were encountered:
The reading code in
extra.components
tries to work around HDF problems with respect so slicing and thus reads entire datasets into memory for subsequent slicing. This relies on virtual memory tricks allowing to allocate potentially very large amounts of memory, even overcomitting physical memory.This has shown to fail on shared nodes under high memory pressure, potentially due to different settings. Selecting down a detector object to a single train and pulse still caused
MultimodKeyData.ndarray()
to allocate an entire sequence worth of memory, which failed.The text was updated successfully, but these errors were encountered: