OnDemandFeatureView: Allow python
transformations for offline
processing.
#4490
Labels
kind/feature
New feature or request
Is your feature request related to a problem? Please describe.
When running a
PythonTransformation
for offline feature retrieval, an exception is raised here. I understand thatpython
transformations on very large datasets do not scale well. However, in some cases, a user might still want to apply apython
transformation to a smaller dataset in the offline store. Due to this error (and the missing implementation ofPythonTransformation().transform_arrow()
), this is currently not possible.Describe the solution you'd like
Instead of raising an exception, we could still run the transformation, and also log a warning indicating that it may not scale well. On top of that we can implement the functionality in
transform_arrow
to make the transformation happen.Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: