Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](nereids)disable join reorder if any table row count is not available #43000 (branch-2.0) #43245

Merged
merged 2 commits into from
Nov 5, 2024

Conversation

englefly
Copy link
Contributor

@englefly englefly commented Nov 5, 2024

What problem does this PR solve?

pick #43000

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Check List (For Committer)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.
  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

englefly commented Nov 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49446 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 95c3a1ffc76d25364c0e173418d4ddee22b582d2, data reload: false

------ Round 1 ----------------------------------
q1	17970	4417	4326	4326
q2	2086	155	149	149
q3	10420	1892	1944	1892
q4	10321	1231	1331	1231
q5	8536	3936	3939	3936
q6	235	127	127	127
q7	2018	1631	1576	1576
q8	9243	2744	2787	2744
q9	10252	9917	9865	9865
q10	8666	3561	3553	3553
q11	421	253	252	252
q12	479	305	308	305
q13	21379	3964	4049	3964
q14	359	339	332	332
q15	501	451	451	451
q16	552	481	459	459
q17	1131	964	921	921
q18	7319	6858	6863	6858
q19	1700	1616	1558	1558
q20	549	343	314	314
q21	4584	4268	4217	4217
q22	519	416	424	416
Total cold run time: 119240 ms
Total hot run time: 49446 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4580	4551	4459	4459
q2	356	262	239	239
q3	4252	4298	4286	4286
q4	2835	2795	2817	2795
q5	7210	7119	7142	7119
q6	236	121	123	121
q7	3298	2797	2820	2797
q8	4394	4478	4503	4478
q9	13684	13544	13531	13531
q10	4235	4231	4276	4231
q11	745	668	685	668
q12	1028	861	851	851
q13	7201	3782	3772	3772
q14	449	435	415	415
q15	501	460	466	460
q16	645	614	609	609
q17	3784	3886	3816	3816
q18	8766	8662	8809	8662
q19	1716	1659	1668	1659
q20	2354	2130	2171	2130
q21	8570	8392	8568	8392
q22	1035	915	940	915
Total cold run time: 81874 ms
Total hot run time: 76405 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212005 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 95c3a1ffc76d25364c0e173418d4ddee22b582d2, data reload: false

query1	939	394	419	394
query2	6556	2133	2294	2133
query3	6924	196	200	196
query4	23078	22344	21776	21776
query5	19728	6506	6520	6506
query6	285	216	234	216
query7	4323	313	313	313
query8	260	268	311	268
query9	3069	2648	2601	2601
query10	461	323	305	305
query11	15748	15352	14827	14827
query12	131	76	73	73
query13	1035	441	431	431
query14	17331	13585	13505	13505
query15	395	220	233	220
query16	6526	288	262	262
query17	1745	936	889	889
query18	891	323	320	320
query19	217	140	148	140
query20	98	90	100	90
query21	201	96	99	96
query22	5162	4843	4979	4843
query23	34172	33516	33627	33516
query24	7951	6343	6287	6287
query25	529	423	430	423
query26	1273	166	165	165
query27	2543	295	292	292
query28	6109	2239	2210	2210
query29	2868	2680	2805	2680
query30	246	173	164	164
query31	931	720	781	720
query32	70	63	62	62
query33	466	250	252	250
query34	858	473	465	465
query35	1156	896	922	896
query36	1269	1158	1206	1158
query37	174	64	62	62
query38	3075	2926	2942	2926
query39	1372	1327	1342	1327
query40	307	96	96	96
query41	41	37	37	37
query42	93	90	84	84
query43	673	624	642	624
query44	1182	731	729	729
query45	245	229	232	229
query46	1227	962	997	962
query47	1794	1717	1859	1717
query48	503	407	415	407
query49	659	364	388	364
query50	853	607	572	572
query51	4754	4666	4695	4666
query52	88	85	87	85
query53	236	188	200	188
query54	2674	2487	2491	2487
query55	90	80	85	80
query56	219	207	201	201
query57	1321	1136	1154	1136
query58	229	208	219	208
query59	3586	3194	3357	3194
query60	216	207	204	204
query61	95	95	93	93
query62	820	488	440	440
query63	205	178	184	178
query64	3596	1595	1444	1444
query65	3648	3598	3516	3516
query66	770	393	384	384
query67	17280	15102	15515	15102
query68	7018	652	629	629
query69	505	291	266	266
query70	1456	1326	1295	1295
query71	382	301	318	301
query72	6853	4775	4772	4772
query73	798	321	324	321
query74	6315	5788	5802	5788
query75	4534	3795	3682	3682
query76	4148	1116	1173	1116
query77	558	256	267	256
query78	12646	11556	11745	11556
query79	6689	624	644	624
query80	3021	390	392	390
query81	527	246	232	232
query82	1597	97	102	97
query83	198	140	135	135
query84	256	70	69	69
query85	1413	307	318	307
query86	357	322	311	311
query87	3293	3035	2998	2998
query88	4891	2297	2300	2297
query89	413	277	294	277
query90	1717	219	218	218
query91	162	126	132	126
query92	61	52	49	49
query93	4816	587	543	543
query94	883	211	205	205
query95	1962	1964	1926	1926
query96	635	326	317	317
query97	6515	6444	6433	6433
query98	227	210	219	210
query99	2866	907	825	825
Total cold run time: 315281 ms
Total hot run time: 212005 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 95c3a1ffc76d25364c0e173418d4ddee22b582d2, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.02	0.02
query3	0.26	0.05	0.04
query4	1.77	0.07	0.08
query5	0.55	0.54	0.53
query6	1.23	0.62	0.61
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.51	0.50	0.47
query10	0.53	0.53	0.52
query11	0.12	0.08	0.08
query12	0.12	0.08	0.09
query13	0.62	0.62	0.61
query14	0.78	0.80	0.80
query15	0.78	0.76	0.76
query16	0.38	0.36	0.38
query17	1.01	1.00	1.00
query18	0.22	0.25	0.22
query19	1.93	1.79	1.84
query20	0.02	0.00	0.01
query21	15.46	0.56	0.54
query22	2.29	2.20	1.72
query23	17.26	1.09	0.83
query24	6.28	1.94	1.45
query25	0.39	0.09	0.06
query26	0.83	0.15	0.16
query27	0.05	0.05	0.04
query28	5.52	0.76	0.76
query29	12.62	2.30	2.26
query30	0.60	0.53	0.52
query31	2.81	0.39	0.37
query32	3.37	0.50	0.50
query33	3.11	3.07	3.05
query34	15.25	4.78	4.78
query35	4.86	4.84	4.82
query36	1.06	1.03	1.02
query37	0.06	0.05	0.04
query38	0.04	0.01	0.02
query39	0.02	0.01	0.02
query40	0.15	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 103.12 s
Total hot run time: 31.13 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 95c3a1ffc76d25364c0e173418d4ddee22b582d2 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.5 seconds inserted 10000000 Rows, about 444K ops/s

@englefly englefly merged commit a4a3e7e into apache:branch-2.0 Nov 5, 2024
22 of 24 checks passed
@englefly englefly deleted the pick43000-br2.0 branch November 5, 2024 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants