Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of the full tile case in batch processing #75

Open
GreatEmerald opened this issue Jul 17, 2017 · 2 comments
Open

Handling of the full tile case in batch processing #75

GreatEmerald opened this issue Jul 17, 2017 · 2 comments

Comments

@GreatEmerald
Copy link

The tutorial says that when processing entire tiles, the extent parameter should still be passed to the batch processing functions, or else stacking will fail. It would be nice if this could be handled by the batching functions themselves, instead of requiring the user to know the extent of the whole tile.

A straight-forward approach would be to use the first tile of input as a reference, and then crop the rest to its extent. However, this may result in loss of data around the edges (the differences in extent can be a few kilometres). While for the most part this should not be a big deal, at times this may be important, for instance if reference data is somewhere near the edge of the tile.

A lossless approach would to to use the raster::extend function to pad the images to the largest extent from the batch. However, this requires knowing the extent of each tile that is being processed, so that the largest extent could be determined, which would require unpacking all tiles first (which would have potential storage space complications).

A yet another approach would be to store a table of all tiles' maximum known extents, and use information from it when extending. But that would require determining the maximum extent of each tile in the world for each supported sensor, which may not be feasible.

@GreatEmerald
Copy link
Author

A variant of the extend approach that does not require unpacking everything is to use addLayer instead of stack, determine which extent is bigger for each layer and extend the smaller one as needed. The downside of this is that several extensions of the large stack may be required if one is unlucky enough to have a bunch of larger-extent tiles at the end. The probability of that happening decreases with more time steps, however.

@GreatEmerald
Copy link
Author

GreatEmerald commented Aug 10, 2017

Actually, this seems to be a bigger problem than I initially thought! Options 1 and 3 don't work after all, because if the first image does not have the smallest extent, then the other images cropped by it will have a smaller extent and thus it's once again impossible to stack them. So either the largest extent needs to be determined, then all images extended to it; or the smallest extent needs to be determined and all images cropped to it; or use the addLayermethod to go through the images one by one (extend or crop as needed).

One option for obtaining the minimum or maximum extent would be to parse the XML metadata files that are provided with ESPA data (it has the element corner_point that indicates, in UTM, the extent minus half the size of a pixel), but I'm not sure if that's any faster than extracting all the files (though certainly more space-efficient).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant