Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Delta Kernel FFI support for read path and expose Apache.Arrow.Table and Microsoft.Data.Analysis.DataFrame read interface #89

Merged
merged 107 commits into from
Oct 12, 2024

Commits on Oct 1, 2024

  1. Added git submodule for delta-kernel-rs with version pinning

    Using cargo to recursively generate FFI headers from the repo was near impossible for my Rust skill level 😄
    mdrakiburrahman committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    6541108 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    eea3526 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1ee5d1e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b224e0a View commit details
    Browse the repository at this point in the history
  5. Updated README

    mdrakiburrahman committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    c8e6d15 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    ddc5a1e View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    cfec531 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    3b7a7c1 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Configuration menu
    Copy the full SHA
    2210400 View commit details
    Browse the repository at this point in the history
  2. Updated signature

    mdrakiburrahman committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    c63255a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e7daf85 View commit details
    Browse the repository at this point in the history
  4. Header

    mdrakiburrahman committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    c96575f View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    7a01f02 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    789c9d3 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. Quick version test

    mdrakiburrahman committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    4251edf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c567926 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    56708d8 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. Configuration menu
    Copy the full SHA
    94c3eba View commit details
    Browse the repository at this point in the history
  2. Docs

    mdrakiburrahman committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    ebf8edb View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ea98bf1 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2024

  1. Update FFI gen

    mdrakiburrahman committed Oct 5, 2024
    Configuration menu
    Copy the full SHA
    4ddc671 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    408d15d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5ee5ef7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b38df01 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    262fc6b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    dee0cdc View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    7dd9a9e View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. Configuration menu
    Copy the full SHA
    d566d3c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    dbc945f View commit details
    Browse the repository at this point in the history
  3. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    96e6e80 View commit details
    Browse the repository at this point in the history
  4. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    456e5bf View commit details
    Browse the repository at this point in the history
  5. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    160a9b0 View commit details
    Browse the repository at this point in the history
  6. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    02f563a View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    36415dd View commit details
    Browse the repository at this point in the history
  8. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    bc5fdfa View commit details
    Browse the repository at this point in the history
  9. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    03ba177 View commit details
    Browse the repository at this point in the history
  10. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    de0fd31 View commit details
    Browse the repository at this point in the history
  11. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    95541f3 View commit details
    Browse the repository at this point in the history
  12. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    43114ff View commit details
    Browse the repository at this point in the history
  13. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    cd31c37 View commit details
    Browse the repository at this point in the history
  14. Add docstring

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    d0ca4b4 View commit details
    Browse the repository at this point in the history
  15. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    3f0afb2 View commit details
    Browse the repository at this point in the history
  16. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    6cb9fef View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    db53471 View commit details
    Browse the repository at this point in the history
  18. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    f10d0d3 View commit details
    Browse the repository at this point in the history
  19. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    332ef4b View commit details
    Browse the repository at this point in the history
  20. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    8f448dc View commit details
    Browse the repository at this point in the history
  21. Get rid of ArrowFfiSchemaConverter and simply cast FFI_ArrowSchema po…

    …inter to CArrowSchema pointer
    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    2ebf2c3 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    0e3134a View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    721762a View commit details
    Browse the repository at this point in the history
  24. Proofread

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    0f3c545 View commit details
    Browse the repository at this point in the history
  25. Fix build warnings

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    e2efdc4 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    a739fa6 View commit details
    Browse the repository at this point in the history
  27. YAML

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    3aeb94d View commit details
    Browse the repository at this point in the history
  28. YAML

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    a552f3e View commit details
    Browse the repository at this point in the history
  29. YAML

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    db85b6c View commit details
    Browse the repository at this point in the history
  30. YAML

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    83ead4f View commit details
    Browse the repository at this point in the history
  31. YAML

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    1830ffa View commit details
    Browse the repository at this point in the history
  32. dotnet format

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    791cb16 View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    1b68e79 View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    7fe50e7 View commit details
    Browse the repository at this point in the history
  35. Configuration menu
    Copy the full SHA
    ef22eac View commit details
    Browse the repository at this point in the history
  36. Configuration menu
    Copy the full SHA
    78e33f9 View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    48169a1 View commit details
    Browse the repository at this point in the history
  38. Whitespace format

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    0236a92 View commit details
    Browse the repository at this point in the history
  39. Run nuget pipeline

    mdrakiburrahman committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    ca998b5 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    04b6510 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a580bd0 View commit details
    Browse the repository at this point in the history
  3. chmod

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    bf6db8e View commit details
    Browse the repository at this point in the history
  4. Try recursive chmod

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    89f28a1 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ab38782 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    0b05e92 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    48fbc15 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    3621208 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    d11f6e5 View commit details
    Browse the repository at this point in the history
  10. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    8444c03 View commit details
    Browse the repository at this point in the history
  11. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    45671ce View commit details
    Browse the repository at this point in the history
  12. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    b170ba4 View commit details
    Browse the repository at this point in the history
  13. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    001569d View commit details
    Browse the repository at this point in the history
  14. Attempt

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    ae617e8 View commit details
    Browse the repository at this point in the history
  15. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e800b67 View commit details
    Browse the repository at this point in the history
  16. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    389a3c1 View commit details
    Browse the repository at this point in the history
  17. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e03e549 View commit details
    Browse the repository at this point in the history
  18. YAML

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    59b7fb5 View commit details
    Browse the repository at this point in the history
  19. Proofread

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    1f78dd6 View commit details
    Browse the repository at this point in the history
  20. Proofread

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    edbbb7a View commit details
    Browse the repository at this point in the history
  21. Build warning

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    7cb1d38 View commit details
    Browse the repository at this point in the history
  22. CRLF -> LF changes

    mdrakiburrahman committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    cf969a6 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    17d4641 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    5907432 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    6a50637 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    b55a973 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Moved ArrowContext into ManagedTableState and solved a bunch of point…

    …er problems with Arrow disposables - stress tested on Windows
    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d2aecf5 View commit details
    Browse the repository at this point in the history
  2. Proofread

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e24fa48 View commit details
    Browse the repository at this point in the history
  3. Proofread

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    968d53d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cbe3a44 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    aa9e205 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    f25536c View commit details
    Browse the repository at this point in the history
  7. Dev env setup script

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    cb38beb View commit details
    Browse the repository at this point in the history
  8. Add libssl-dev

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    51e344d View commit details
    Browse the repository at this point in the history
  9. Docs

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    65d856c View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    0376c69 View commit details
    Browse the repository at this point in the history
  11. Add sample output

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4811e6a View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    efc8e95 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    b4d065d View commit details
    Browse the repository at this point in the history
  14. Proofread

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c48108a View commit details
    Browse the repository at this point in the history
  15. Proofread

    mdrakiburrahman committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d3561c9 View commit details
    Browse the repository at this point in the history