You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A large part of what resolvers, drivers and data_adapters do is querying the data (assumed to be large) data source, so that only a small part of that data source needs to be present. That happens at discovery-time (e.g. do not read all files, but only read the relevant files), during parsing of the file (e.g. doing an http-range request to only read certain bytes of the COG), or during post-processing (data adapter).
What can be queried depends on the data type. A geodataset has no zoom level. A dataset has no geometry component. Currenly, there is a lot of boilerplate code, as different DataSources, have different keyword only parameters that differ slightly from other datasets.
If all classes would accept a different object, instead of all these different parameters, a lot of HydroMT logic could be better generalized.
These query parameters could still exist in higher-level methods (DataCatalog.get_rasterdataset), but would contribute to deduplication efforts in the DataSource, DataAdapter and Driver classes.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
A large part of what
resolvers
,drivers
anddata_adapters
do is querying the data (assumed to be large) data source, so that only a small part of that data source needs to be present. That happens at discovery-time (e.g. do not read all files, but only read the relevant files), during parsing of the file (e.g. doing an http-range request to only read certain bytes of the COG), or during post-processing (data adapter).What can be queried depends on the data type. A geodataset has no zoom level. A dataset has no geometry component. Currenly, there is a lot of boilerplate code, as different
DataSource
s, have different keyword only parameters that differ slightly from other datasets.If all classes would accept a different object, instead of all these different parameters, a lot of HydroMT logic could be better generalized.
These query parameters could still exist in higher-level methods (
DataCatalog.get_rasterdataset
), but would contribute to deduplication efforts in theDataSource
,DataAdapter
andDriver
classes.Beta Was this translation helpful? Give feedback.
All reactions