Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PECO-238] Initial CloudFetch implementation #158

Merged
merged 8 commits into from
Aug 16, 2023
Merged

[PECO-238] Initial CloudFetch implementation #158

merged 8 commits into from
Aug 16, 2023

Conversation

kravets-levko
Copy link
Contributor

@kravets-levko kravets-levko commented Aug 3, 2023 •
edited
Loading

PECO-238

  • Basic cloud fetch results support
  • Handle expired links
  • Concurrent downloads
  • Memory usage optimizations (cloud fetch chunks are quite big, and if many chunks are loaded into memory and being processed, rics of getting OOM is very high)
  • Tests

@kravets-levko
Fix tests
cff71f8 
Signed-off-by: Levko Kravets <[email protected]>
@databricks databricks deleted a comment from codecov-commenter Aug 3, 2023
@kravets-levko kravets-levko marked this pull request as ready for review August 10, 2023 14:12
nithinkdb
nithinkdb reviewed Aug 10, 2023
View reviewed changes
Copy link
Contributor

@nithinkdb nithinkdb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just have a few questions, but looks good overall.

tests/unit/result/CloudFetchResult.test.js Show resolved Hide resolved
lib/globalConfig.ts Show resolved Hide resolved
lib/DBSQLOperation/index.ts Show resolved Hide resolved
@nithinkdb nithinkdb self-requested a review August 16, 2023 01:33
nithinkdb
nithinkdb approved these changes Aug 16, 2023
View reviewed changes
Copy link
Contributor

@nithinkdb nithinkdb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@kravets-levko kravets-levko merged commit f265368 into main Aug 16, 2023
3 checks passed
@kravets-levko kravets-levko deleted the PECO-238-cloudfetch branch August 16, 2023 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nithinkdb nithinkdb nithinkdb approved these changes

@arikfr arikfr Awaiting requested review from arikfr

@superdupershant superdupershant Awaiting requested review from superdupershant superdupershant is a code owner

@yunbodeng-db yunbodeng-db Awaiting requested review from yunbodeng-db yunbodeng-db is a code owner

@susodapop susodapop Awaiting requested review from susodapop

@andrefurlan-db andrefurlan-db Awaiting requested review from andrefurlan-db andrefurlan-db is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kravets-levko @nithinkdb