Initialization of API on MOSMIX slow?

[guidocioni]

So I just updated to wetterdienst 0.16.1 and I haven't updated in a while so I may miss something but...
To get mosmix data I have to know define the API every time, e.g.

stations = API(parameter="large",
                mosmix_type=DwdMosmixType.LARGE,
                humanize=True,
                tidy=True).filter(station_id=["01001"])

This step takes 4 seconds on my PC, 14 seconds on my raspberry pi 4.

As I have to run this into a webapp every time that a new station is selected is kinda killing the whole execution timeout :)
Is there a reason why this is so slow? As the data is downloaded only later (when calling df = stations.values.all().df) why is this step taking so much time? I guess this is disk write speed bounded, which would explain the different performance on the raspberry.

[gutzbenj]

Dear Guido,

apparently I didn't apply caching to the stations method, which causes it to reload stations every time the request is setup. We are looking forward to replace the current caching method which is at some point causing trouble (#243) however as we are not there yet I will just apply a patch that caches the stations and that should pretty much fix the problem. We'll do another release later so that you can again run the app smoothly.

[guidocioni]

Yeah I suspected it had something to do with the query of the station list.
In the app I'm using a pickle to load station data (as it is not supposed to change much in the future) to speed up this process.

Would be interesting to know why is taking so long though. The list is just a TAB separated tex file on DWD, no? As far as I remember...

[gutzbenj]

You are right except for some lines like these

TABLE St 99999 99991 gmos 99792 ecmgl 99791 gmegl
clu CofX id ICAO name nb. el. elev Hmod-H type
===== ----- ===== ---- -------------------- ------ ------- ----- ------ ----

However I simplified the method so that should speed it up even a bit more.