Feature Request: a local cache for downloading files

[wjaskowski]

I often statistically analyze files sent to the neptune. In the notebook, I first download the files, then do some analyses. Since the files change on the server (e.g., new runs are added), I redownload everything every time I need to redo the analysis. This takes a lot of time since the files are large and there are many of them. So, recently, I have written a small function that manages a cache and now it is much faster and I am putting less pressure on your and mine bandwidth. However, this is not an ideal solution since I am not able to check whether the file on the server actually changed (no md5, for instance) so my solution is not robust.

So, I was thinking, maybe you could consider implementing a cache on your side?

I imagine that from the user point of view, it could be just an additional parameter to Handler.download(..., use_cache=True) or Handler.download(..., use_cache, cache_dir=Path('/tmp').