Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add rust extension lib alluxiocommon to the repo #39

Merged
merged 34 commits into from
Apr 29, 2024
Merged

Add rust extension lib alluxiocommon to the repo #39

merged 34 commits into from
Apr 29, 2024

Commits on Apr 12, 2024

  1. Port changes from https://github.com/lucyge2022/alluxiofs/tree/multit… 

    …hread

5f80059 small fixes for REST and metrics
1c426c2 add access debug logs for all fsspec ops & add quick sanity unit tests for read
7dcb833 checkin multi process http benchmark for worker http server
4b4ebce use alluxiocommon pyo3 package to try to do multithread http req
523f748 add rust pyo3 extension
8979d2b remove first
0367659 add related files
4d0eebe small fixes
a0519b3 if any error on any thread running result, raise exception back to python directly
    @lucyge2022
    lucyge2022 committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    22c7f42 View commit details
    Browse the repository at this point in the history

  2. missing changes from port

    @lucyge2022
    lucyge2022 committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    814629d View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. fix the bug of inpage_read_len calculation

    @lucyge2022
    lucyge2022 committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    8913fa8 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. make ondemand pool default and enable it by flag during instantiation

    @lucyge2022
    lucyge2022 committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    e321c84 View commit details
    Browse the repository at this point in the history

  2. cleanup fixes

    @lucyge2022
    lucyge2022 committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    1d5d5f8 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. WIP add fixture to test with alluxio client with alluxiocommon lib en… 

    …abled
    @lucyge2022
    lucyge2022 committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    5b20fac View commit details
    Browse the repository at this point in the history

  2. for read pages if we got any data just return without throwing any fo… 

    …llowing exception
    @lucyge2022
    lucyge2022 committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    e9bc8c0 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. add _all_page_generator with alluxiocommon

    @lucyge2022
    lucyge2022 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    9efb902 View commit details
    Browse the repository at this point in the history

  2. 1. add pytest fixture for alluxiocommon enabled to use in test_docker… 

    …_alluxio_fsspec.py

2. make tests/client/test_read_range_docker.py (now named as test_read_docker.py
to handle full read as well
add alluxiocommon fixture in it too
    @lucyge2022
    lucyge2022 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    a226893 View commit details
    Browse the repository at this point in the history

  3. remove unnecessary files

    @lucyge2022
    lucyge2022 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    9cda7cf View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. try adding a ci yaml file with alluxiocommon enabled as dependency

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    9d3c31a View commit details
    Browse the repository at this point in the history

  2. cleanup fixes

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    c585c8a View commit details
    Browse the repository at this point in the history

  3. add to ci.yml of alluxiocommon dependency

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    b517162 View commit details
    Browse the repository at this point in the history

  4. add rust/alluxiocommon/tests/

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    b5b5b8b View commit details
    Browse the repository at this point in the history

  5. fixes

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    a24652f View commit details
    Browse the repository at this point in the history

  6. remove changes in ci.yml as it wont run

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    647e3ea View commit details
    Browse the repository at this point in the history

  7. try add alluxiocommon pyo3 build

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    083a2a5 View commit details
    Browse the repository at this point in the history

  8. fix pyo3 related

    @lucyge2022
    lucyge2022 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    5fe1c1d View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. pre-commit style changes + manually install openssl

    @lucyge2022
    lucyge2022 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    3eccac5 View commit details
    Browse the repository at this point in the history

  2. chanegs

    @lucyge2022
    lucyge2022 committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    c9c1dbc View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2024

  1. try to use rustls instead of rust-native-tls as reqwest dependency as… 

    … rust-native-tls requires dynamic link of tls in runtime env
    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    f44d550 View commit details
    Browse the repository at this point in the history

  2. enable run alluxiocommon tests job in ci.yml

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    b93e580 View commit details
    Browse the repository at this point in the history

  3. reorder to have alluxiocommon install after alluxiocommon wheel building

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    0b23d38 View commit details
    Browse the repository at this point in the history

  4. try fix pip install alluxiocommon from dist/

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    e509554 View commit details
    Browse the repository at this point in the history

  5. add --no-index

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    006ffd8 View commit details
    Browse the repository at this point in the history

  6. add ls dist/ to debug

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    e0a62b4 View commit details
    Browse the repository at this point in the history

  7. add pyo3 with python version when buildoing

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    b50cd32 View commit details
    Browse the repository at this point in the history

  8. merge pytest for alluxiocommon and alluxiofs together

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    7571b50 View commit details
    Browse the repository at this point in the history

  9. syntax fix

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    06fd446 View commit details
    Browse the repository at this point in the history

  10. syntax fix

    @lucyge2022
    lucyge2022 committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    c8d9743 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. try reorder

    @lucyge2022
    lucyge2022 committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    b2deaff View commit details
    Browse the repository at this point in the history

  2. name change

    @lucyge2022
    lucyge2022 committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    03a16da View commit details
    Browse the repository at this point in the history

  3. modify rust/alluxiocommon/README.md and cleanup ci.yml

    @lucyge2022
    lucyge2022 committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    1de7808 View commit details
    Browse the repository at this point in the history

  4. remove comments for debugging purpose before

    @lucyge2022
    lucyge2022 committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    8f73b59 View commit details
    Browse the repository at this point in the history