[opt](hive) support orc generated from hive 1.x for all file scan node #28806
Conversation
|
run buildall |
morningman
changed the title
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| @@ -483,6 +484,19 @@ class OrcReader : public GenericReader { | |||
| int64_t get_remaining_rows() { return _remaining_rows; } | |||
| void set_remaining_rows(int64_t rows) { _remaining_rows = rows; } | |||
|
|
|||
| // check if the given name is like _col0, _col1, ... | |||
| bool inline _is_hive1_col_name(const std::string& name) { | |||
There was a problem hiding this comment.
warning: method '_is_hive1_col_name' can be made static [readability-convert-member-functions-to-static]
| bool inline _is_hive1_col_name(const std::string& name) { | |
| static bool inline _is_hive1_col_name(const std::string& name) { |
|
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools |
|
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools |
|
TeamCity be ut coverage result: |
|
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools |
|
TeamCity be ut coverage result: |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools |
|
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools |
|
PR approved by anyone and no changes requested. |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
Proposed changes
Previous, we only handle orc generated from hive 1.x for
HiveScanNodeif user set
"hive.version" = "1.1.0"explicitly.It should be for all kinds for FileScanNode.
And no need to set
hive.version = 1.1.0explicitly for this feature.In this PR,
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...