Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement](paimon)support predict for null and notnull #29134

Merged
merged 2 commits into from
Jan 3, 2024
Merged

[enhancement](paimon)support predict for null and notnull #29134

merged 2 commits into from
Jan 3, 2024

Conversation

wuwenchi
Copy link
Contributor

Proposed changes

support predict for null and notnull

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@xiaokang xiaokang added usercase Important user case type label dev/2.0.4 labels Dec 27, 2023
@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.5 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17183939260 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit a00410cdf84be81b6fe64f1c998760c844186768, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5021	4681	4660	4660
q2	368	147	142	142
q3	1460	1230	1198	1198
q4	1136	939	926	926
q5	3189	3193	3169	3169
q6	250	129	127	127
q7	1020	497	503	497
q8	2294	2258	2279	2258
q9	6741	6752	6671	6671
q10	3197	3277	3283	3277
q11	319	213	211	211
q12	353	210	207	207
q13	4163	3417	3401	3401
q14	244	213	217	213
q15	573	514	520	514
q16	434	389	386	386
q17	1049	832	673	673
q18	7097	6882	6727	6727
q19	1634	1636	1653	1636
q20	563	334	306	306
q21	3217	2657	2760	2657
q22	363	297	302	297
Total cold run time: 44685 ms
Total hot run time: 40153 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4571	4565	4575	4565
q2	269	169	168	168
q3	3369	3368	3356	3356
q4	2218	2214	2213	2213
q5	5748	5731	5704	5704
q6	244	117	119	117
q7	2379	1896	1845	1845
q8	3613	3623	3616	3616
q9	8973	9017	8970	8970
q10	3794	3881	3862	3862
q11	506	370	362	362
q12	764	593	597	593
q13	3891	3166	3148	3148
q14	296	253	260	253
q15	578	524	511	511
q16	492	450	466	450
q17	1991	1970	1974	1970
q18	8788	8081	8349	8081
q19	1755	1756	1763	1756
q20	2245	1930	1905	1905
q21	6115	5748	5773	5748
q22	530	451	453	451
Total cold run time: 63129 ms
Total hot run time: 59644 ms

@morningman
Copy link
Contributor

Please add regression test

@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.64 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17184071171 Bytes

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 2, 2024
Copy link
Contributor

github-actions bot commented Jan 2, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Jan 2, 2024

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 2a9b4a0 into apache:master Jan 3, 2024
25 of 28 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Jan 3, 2024
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
@wuwenchi wuwenchi deleted the fix-parimon-predicte branch February 19, 2024 09:11
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.5-merged dev/3.0.0-merged reviewed usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants