make datafusion-catalog-listing and move some implementation of listing out of datafusion/core/datasource/listing #14464
Conversation
|
cc @alamb |
😍 |
…atafusion into datafusion_catalog_listing
There was a problem hiding this comment.
Thanks @logan-keede -- I think this is a great step forward. I think it would be better if we avoided moving the unit tests if possible, but I don't think that is a blocker strictly speaking
Love to see this project starting!
| use std::sync::Arc; | ||
|
|
||
| pub use self::url::ListingTableUrl; | ||
| pub use datafusion_catalog_listing::*; |
| @@ -46,6 +46,8 @@ mod physical_optimizer; | |||
|
|
|||
| mod catalog; | |||
|
|
|||
| mod catalog_listing; | |||
|
|
||
| # DataFusion catalog-listing | ||
|
|
||
| [DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format. |
There was a problem hiding this comment.
I pushed a commit to improve this README content
| @@ -569,218 +580,6 @@ mod tests { | |||
| assert_eq!(0, chunks.len()); | |||
| } | |||
|
|
|||
| #[tokio::test] | |||
| async fn test_pruned_partition_list_empty() { | |||
There was a problem hiding this comment.
Theses tests look like unit tests to me (they are are for pruned_partition_list which is defined in this module
I looked at make_test_store_and_state and it seems to me like it is fairly simple and could be copy/pasted into this test module to avoid a new dependency.
Would you be ok with doing that? I can also push a commit to this PR if it is good with you
There was a problem hiding this comment.
I took the liberty of doing this in 53dbc2f
Which issue does this PR close?
datafusion-catalog-listingout ofcore/src/datasource#14462datafusioncrate (datafusion/core) #14444.Rationale for this change
Decrease build time, Increase Modularity.
What changes are included in this PR?
Moving some part of
datasource/listingtodatafusion-catalog-listingAre these changes tested?
As much as I could on my PC by CI.
Are there any user-facing changes?
There should not be any user-facing changes, it is only a refactor.