Override attributes for GeoDataset in init?

[cjrd]

GeoDatasets (RasterDataset and VectorDataset) both have attributes for finding/parsing files, e.g.

class RasterDataset(GeoDataset):
    filename_glob = "*"
...

Where the index is then built in __init__ using these attributes. It looks like the preferred way to override these attributes is to create a subclass, e.g.

class XView3Polarizations(RasterDataset):
    filename_glob = "*_dB.tif"

ds = XView3Polarizations(data_root="...")

I think it would be useful to allow overriding these attributes on initialization, e.g.

ds = RasterDataset(data_root="...", filename_glob = "*_dB.tif")

I'm using a number of different datasets that are small variations on a Raster/VectorDataset, so instead of writing classes for each, I've added a dynamic attribute override in GeoDataset (see diff):

for key, value in kwargs.items():
    if hasattr(self, key):
        setattr(self, key, value)
    else:
        raise ValueError(f"Unknown initialization argument: '{key}'" + 
            f" is not an attribute for {self.__class__.__name__}")

Of course, this could be done in a less dynamic, and probably safer, way such as listing out all attributes in RasterDataset/VectorDataset. But in any case, I think this would allow torchgeo to work with a greater variety of raw datasources without requiring the users to write their own classes for each source.

[adamjstewart]

Thanks for the suggestion! This will probably be solved once we refactor to TorchData (composition instead of inheritance). No timeline for this at the moment, but it's something I want to explore this fall.

@calebrob6 actually wanted to get rid of filename_glob and filename_regex entirely and instead have users pass in a list of filenames or a function to extract the relevant parts of the filename. I think this will also be solved by TorchData.

[cjrd]

Thanks - a compositional GeoDataset sounds great. I've certainly enjoyed using DataPipes elsewhere.

Edit: for anyone else that comes across this, you can shim this behavior without editing torchgeo internal via:

class CustomRasterDataset(RasterDataset):
    def __init__(self, root: str, 
        crs: Optional[CRS] = None,
        res: Optional[float] = None,
        transforms: Optional[Callable[[Dict[str, Any]], Dict[str, Any]]] = None,
        cache: bool = True,
        **kwargs) -> None:
        for key, value in kwargs.items():
            if hasattr(self, key):
                setattr(self, key, value)
            else:
                raise ValueError(f"Unknown initialization argument: '{key}'" + 
                    f" is not an attribute for {self.__class__.__name__}")

        super().__init__(root, crs, res, transforms, cache)