-
Notifications
You must be signed in to change notification settings - Fork 446
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(rust, python): treat FSCK files_removed as strings #3219
fix(rust, python): treat FSCK files_removed as strings #3219
Conversation
python/deltalake/table.py
Outdated
deserialized = json.loads(metrics) | ||
deserialized[FSCK_METRICS_FILES_REMOVED_LABEL] = json.loads(deserialized["files_removed"]) | ||
return deserialized |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we don't care about maintaining the current response then we can omit this step, but for sake of breaking the interface, added a deserializing step here.
c2be848
to
ff8cf2d
Compare
@liamphmurphy can you squash the commits please |
5b11979
to
4468c85
Compare
Done! Let me know if something isn't looking right. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3219 +/- ##
==========================================
- Coverage 72.31% 72.30% -0.02%
==========================================
Files 138 138
Lines 45398 45412 +14
Branches 45398 45412 +14
==========================================
+ Hits 32831 32833 +2
- Misses 10489 10502 +13
+ Partials 2078 2077 -1 ☔ View full report in Codecov by Sentry. |
…h Spark Signed-off-by: Liam Murphy <[email protected]>
6cbba07
to
3f11d97
Compare
Description
This changes how the files_removed array is serialized, making it a stringified representation of the list of files. This is done to ensure compatibility with Spark.
To maintain the current behavior with the python bindings, the repair method wrapper in python will deserialize back to a list.
Related Issue(s)
closes #3140