Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent relationships between proto objects #4573

Open
tokoko opened this issue Sep 25, 2024 · 0 comments
Open

Inconsistent relationships between proto objects #4573

tokoko opened this issue Sep 25, 2024 · 0 comments

Comments

@tokoko
Copy link
Collaborator

tokoko commented Sep 25, 2024

We have a strange track record of defining relationships between various feast proto objects, namely we sometimes include "child" object protos inside parent objects, while in other cases we only refer to them by name.

For example, FeatureView includes DataSource objects as part of the message, which means that we are effectively keeping identical DataSource objects in the registry backends more than once (incl. additional information like tags). This can also lead to strange data consistency issues, because a user can call registry.apply_data_source on an existing data source which will update only the data source object in the registry and leave the one embedded in FeatureView with stale value. Actual FeatureStore methods will most likely use a stale value for both operations and permission checking.

In other cases, some relationships are defined by name only, for example FeatureView keeps a list of Entity names only, FeatureViewProjection (and FeatureService by association) only stores feature view names, rather than FeatureView proto messages themselves.

While having relationships based on names forces you to do some additional lookups to reconstruct necessary information after retrieval, I think it's still worth it as it will avoid data duplication and consistency/security issues.

P.S. This is almost certainly a very breaking change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant