Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: set VirtualTableScan schema explicitly #272

Merged

Commits on Jun 19, 2024

  1. fix: maintain defined schema in VirtualTableScan

    VirtualTableScan in Substrait is expected to contain both a NamedSchema
    (field names in dfs form and types) like any other read rel, plus the
    actual data rows. However substrait-java was ignoring the types of the
    NamedSchema, instead grabbing type info from the first row of the data.
    This however is not always sufficient since the data can contain varying
    nullability row-by-row.
    
    This also just simplifies VirtualTableScan and makes it behave more like
    one would expect by looking at the Substrait spec as well as other ReadRels
    Blizzara committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    d997326 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Configuration menu
    Copy the full SHA
    14b7a1b View commit details
    Browse the repository at this point in the history