Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add conflict resolution methods to DataCollection.union() #176

Open
philsmt opened this issue May 4, 2021 · 0 comments
Open

Add conflict resolution methods to DataCollection.union() #176

philsmt opened this issue May 4, 2021 · 0 comments

Comments

@philsmt
Copy link
Contributor

philsmt commented May 4, 2021

As discussed in #174, the current workaround assumes that processed files only ever modify all sources also present in the corresponding raw file. While true for the current facilty implementations , this assumption may be no longer be true when thinking towards custom processing supplied by users.

In the interest of sane storage policies, copying over all raw files should be discontinued too for facilty processing, and never be considered with users. Hence, it would be useful for DataCollection.union to support conflict resolution methods (such as "keep oldest", "keep last") to allow loading all these different locations into a cohesive DataCollection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant