Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat](Nereids) Support use mv hint #45855

Merged
merged 3 commits into from
Dec 30, 2024

Conversation

LiBinfeng-01
Copy link
Collaborator

@LiBinfeng-01 LiBinfeng-01 commented Dec 24, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

support use_mv and no_use_mv hint to control mv chose strategy in cbo
explain select /*+ use_mv(mv1)*/ * from t1;

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40176 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ba21a85474e2e66a8bac88edea74d999cacbda42, data reload: false

------ Round 1 ----------------------------------
q1	17596	7397	7312	7312
q2	2041	201	177	177
q3	10518	1129	1196	1129
q4	10218	772	741	741
q5	7588	2695	2622	2622
q6	239	148	146	146
q7	997	625	625	625
q8	9248	1844	1898	1844
q9	6597	6474	6392	6392
q10	6994	2340	2339	2339
q11	467	261	266	261
q12	437	221	221	221
q13	17768	2945	2965	2945
q14	249	213	213	213
q15	551	501	496	496
q16	663	593	593	593
q17	983	585	511	511
q18	7341	6786	6830	6786
q19	1391	1024	1068	1024
q20	487	183	185	183
q21	4003	3319	3304	3304
q22	367	312	316	312
Total cold run time: 106743 ms
Total hot run time: 40176 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7287	7255	7235	7235
q2	335	231	234	231
q3	2888	2755	3093	2755
q4	2120	1908	1794	1794
q5	5554	5587	5624	5587
q6	228	138	139	138
q7	2217	1781	1810	1781
q8	3390	3528	3490	3490
q9	8896	8914	8949	8914
q10	3634	3552	3559	3552
q11	603	508	506	506
q12	803	623	614	614
q13	10913	3068	3068	3068
q14	309	276	281	276
q15	567	504	510	504
q16	714	634	671	634
q17	1834	1604	1595	1595
q18	7825	7402	7286	7286
q19	1711	1610	1539	1539
q20	2058	1850	1837	1837
q21	5436	5259	5259	5259
q22	630	609	544	544
Total cold run time: 69952 ms
Total hot run time: 59139 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191442 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ba21a85474e2e66a8bac88edea74d999cacbda42, data reload: false

query1	987	404	379	379
query2	6542	2311	2331	2311
query3	6704	212	205	205
query4	34273	23575	24036	23575
query5	4426	468	452	452
query6	290	186	192	186
query7	4629	306	307	306
query8	312	243	242	242
query9	9734	2763	2758	2758
query10	484	259	247	247
query11	18288	15271	15235	15235
query12	162	104	104	104
query13	1708	449	436	436
query14	11594	7123	7024	7024
query15	303	192	178	178
query16	8147	431	424	424
query17	1645	599	577	577
query18	2159	303	303	303
query19	383	160	195	160
query20	116	107	107	107
query21	212	102	105	102
query22	4723	4389	4413	4389
query23	34032	33724	33595	33595
query24	10169	2562	2466	2466
query25	632	374	386	374
query26	1569	148	148	148
query27	2828	327	334	327
query28	7869	2443	2470	2443
query29	948	408	398	398
query30	289	173	144	144
query31	1049	798	814	798
query32	96	57	59	57
query33	797	289	284	284
query34	977	509	509	509
query35	883	730	733	730
query36	1108	915	941	915
query37	150	77	71	71
query38	4441	4063	4166	4063
query39	1479	1438	1425	1425
query40	282	101	100	100
query41	48	45	45	45
query42	118	102	99	99
query43	546	496	482	482
query44	1224	799	816	799
query45	183	167	166	166
query46	1161	699	700	699
query47	1979	1865	1855	1855
query48	413	327	313	313
query49	1211	383	394	383
query50	813	375	400	375
query51	7239	6963	7102	6963
query52	98	94	91	91
query53	270	181	181	181
query54	1127	403	411	403
query55	79	75	76	75
query56	259	227	250	227
query57	1262	1115	1126	1115
query58	243	213	224	213
query59	3251	3087	2989	2989
query60	269	244	244	244
query61	105	108	105	105
query62	933	722	732	722
query63	213	177	184	177
query64	4707	655	635	635
query65	3237	3238	3254	3238
query66	1068	339	311	311
query67	15758	15796	15485	15485
query68	5587	567	569	567
query69	444	251	250	250
query70	1200	1118	1106	1106
query71	463	263	243	243
query72	6404	4134	4110	4110
query73	788	361	367	361
query74	10795	9213	9399	9213
query75	3478	2653	2658	2653
query76	3385	1125	1168	1125
query77	589	267	285	267
query78	10372	9708	9426	9426
query79	2396	602	617	602
query80	1013	414	420	414
query81	549	227	231	227
query82	626	120	123	120
query83	242	153	152	152
query84	232	67	73	67
query85	1739	302	301	301
query86	497	303	263	263
query87	4612	4602	4501	4501
query88	3864	2251	2195	2195
query89	413	295	296	295
query90	2143	189	187	187
query91	154	100	106	100
query92	65	51	52	51
query93	1555	557	560	557
query94	997	287	288	287
query95	359	254	252	252
query96	614	292	279	279
query97	2858	2701	2710	2701
query98	222	194	192	192
query99	1695	1441	1456	1441
Total cold run time: 305583 ms
Total hot run time: 191442 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.65 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ba21a85474e2e66a8bac88edea74d999cacbda42, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.04	0.03
query3	0.23	0.08	0.07
query4	1.59	0.10	0.10
query5	0.41	0.41	0.40
query6	1.15	0.65	0.64
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.50
query10	0.54	0.57	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.59	0.59
query14	2.72	2.78	2.83
query15	0.91	0.83	0.82
query16	0.39	0.36	0.38
query17	0.97	0.98	0.98
query18	0.23	0.22	0.20
query19	1.84	1.79	2.01
query20	0.02	0.01	0.01
query21	15.36	0.58	0.60
query22	2.42	2.14	2.21
query23	16.97	0.93	0.80
query24	3.06	1.89	1.71
query25	0.21	0.14	0.09
query26	0.69	0.14	0.14
query27	0.04	0.04	0.05
query28	9.83	1.11	1.07
query29	12.62	3.26	3.24
query30	0.26	0.07	0.06
query31	2.85	0.37	0.38
query32	3.31	0.48	0.45
query33	3.07	3.06	3.21
query34	17.33	4.50	4.57
query35	4.52	4.53	4.54
query36	0.67	0.47	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.32 s
Total hot run time: 33.65 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32428 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1d16c55f41c90b0c8d806e8c3e617dfa8b4eddcf, data reload: false

------ Round 1 ----------------------------------
q1	17602	6149	6022	6022
q2	2050	292	167	167
q3	10479	1223	737	737
q4	10213	866	428	428
q5	7699	2178	1955	1955
q6	210	184	152	152
q7	911	762	607	607
q8	9248	1368	1179	1179
q9	5174	4896	4948	4896
q10	6790	2300	1877	1877
q11	498	280	258	258
q12	355	361	224	224
q13	17796	3550	2908	2908
q14	237	253	232	232
q15	569	510	489	489
q16	620	606	599	599
q17	555	859	337	337
q18	6948	6549	6332	6332
q19	2315	966	573	573
q20	306	322	194	194
q21	2828	2185	1964	1964
q22	358	337	298	298
Total cold run time: 103761 ms
Total hot run time: 32428 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6319	6208	6205	6205
q2	237	331	234	234
q3	2230	2636	2344	2344
q4	1389	1798	1336	1336
q5	4340	4783	4837	4783
q6	190	181	145	145
q7	2038	1948	1800	1800
q8	2618	2841	2717	2717
q9	7284	7268	7243	7243
q10	3093	3360	2824	2824
q11	584	502	508	502
q12	625	788	603	603
q13	3421	3826	3181	3181
q14	300	307	284	284
q15	568	524	507	507
q16	655	677	620	620
q17	1240	1759	1273	1273
q18	7692	7516	7342	7342
q19	831	1200	1102	1102
q20	2038	2058	1950	1950
q21	5791	5452	4883	4883
q22	613	608	589	589
Total cold run time: 54096 ms
Total hot run time: 52467 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196772 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1d16c55f41c90b0c8d806e8c3e617dfa8b4eddcf, data reload: false

query1	1340	1004	935	935
query2	6506	2440	2359	2359
query3	10965	4602	4819	4602
query4	33014	23522	23432	23432
query5	3963	630	463	463
query6	300	200	219	200
query7	3989	487	308	308
query8	317	263	260	260
query9	9521	2756	2743	2743
query10	446	313	247	247
query11	18056	15213	15178	15178
query12	163	110	105	105
query13	1589	546	415	415
query14	11455	7298	7289	7289
query15	248	215	187	187
query16	7835	616	448	448
query17	1536	769	563	563
query18	2064	411	320	320
query19	214	197	170	170
query20	132	110	107	107
query21	204	121	102	102
query22	4615	4623	4599	4599
query23	34676	33764	33474	33474
query24	6549	2316	2291	2291
query25	474	470	401	401
query26	791	275	148	148
query27	2136	460	328	328
query28	5675	2461	2459	2459
query29	590	558	415	415
query30	204	185	152	152
query31	1016	947	845	845
query32	69	60	57	57
query33	512	364	296	296
query34	788	860	537	537
query35	807	845	786	786
query36	1026	1078	977	977
query37	125	104	73	73
query38	4388	4258	4278	4258
query39	1687	1480	1463	1463
query40	204	113	101	101
query41	49	42	43	42
query42	119	104	103	103
query43	527	517	506	506
query44	1352	822	823	822
query45	187	174	169	169
query46	891	1085	659	659
query47	2004	1985	1931	1931
query48	372	440	332	332
query49	741	488	392	392
query50	652	669	392	392
query51	7302	7296	7170	7170
query52	102	99	101	99
query53	227	259	193	193
query54	491	498	415	415
query55	88	80	82	80
query56	257	268	248	248
query57	1247	1253	1162	1162
query58	246	226	251	226
query59	3234	3216	3188	3188
query60	288	270	258	258
query61	111	103	108	103
query62	874	886	745	745
query63	236	195	194	194
query64	3130	1050	658	658
query65	3329	3278	3244	3244
query66	771	413	331	331
query67	16545	15822	15614	15614
query68	8987	747	498	498
query69	473	285	250	250
query70	1225	1158	1146	1146
query71	425	268	256	256
query72	5911	3816	4034	3816
query73	657	769	363	363
query74	10413	9018	8827	8827
query75	4603	3185	2628	2628
query76	4637	1187	768	768
query77	840	368	271	271
query78	10356	10135	9473	9473
query79	3454	900	578	578
query80	703	509	435	435
query81	490	263	227	227
query82	628	148	123	123
query83	206	167	143	143
query84	284	90	82	82
query85	802	411	305	305
query86	346	287	306	287
query87	4682	4675	4376	4376
query88	4185	2230	2215	2215
query89	421	324	307	307
query90	1909	185	181	181
query91	136	137	103	103
query92	67	54	51	51
query93	983	847	530	530
query94	649	382	285	285
query95	357	265	254	254
query96	485	603	294	294
query97	2769	2788	2691	2691
query98	225	204	194	194
query99	1662	1556	1446	1446
Total cold run time: 299531 ms
Total hot run time: 196772 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.39 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1d16c55f41c90b0c8d806e8c3e617dfa8b4eddcf, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.03	0.04
query3	0.23	0.07	0.06
query4	1.60	0.10	0.11
query5	0.42	0.42	0.42
query6	1.16	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.52
query10	0.56	0.54	0.54
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.59	0.60
query14	2.72	2.83	2.82
query15	0.89	0.83	0.83
query16	0.39	0.38	0.38
query17	0.99	1.05	1.05
query18	0.23	0.22	0.21
query19	1.98	1.85	1.98
query20	0.01	0.01	0.02
query21	15.36	0.95	0.57
query22	0.75	0.95	0.70
query23	15.05	1.38	0.56
query24	2.71	0.94	1.01
query25	0.27	0.13	0.05
query26	0.27	0.14	0.13
query27	0.05	0.06	0.06
query28	14.07	1.52	1.05
query29	12.54	3.93	3.28
query30	0.25	0.09	0.06
query31	2.83	0.58	0.38
query32	3.23	0.56	0.47
query33	3.04	3.08	3.08
query34	16.53	5.10	4.47
query35	4.51	4.44	4.46
query36	0.64	0.49	0.50
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.44 s
Total hot run time: 31.39 s

}
explain {
sql """select /*+ use_mv(mv3, mv1) */ c1 from t1"""
contains("Used: use_mv([mv3].[mv1] )")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should add mv_rewrite_success in Suite.groovy to check if use mv or not

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace by explain memo plan

explain {
sql """select /*+ no_use_mv(mv1, mv3) */ c1 from t1"""
contains("Used: no_use_mv([mv1].[mv3] )")
notContains("internal.test_cbo_use_mv.mv3")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notContains should use mv_not_part_in in Suite.groovy

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace by explain memo plan


sql """use ${db};"""
explain {
sql """select /*+ use_mv(mv1)*/ * from t1;"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add three part names for async materialized view and for part names for sync materialized view test

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

List<MaterializationContext> hintMTMVs = new ArrayList<>();
if (useMvHint.isAllMv()) {
useMvHint.setStatus(Hint.HintStatus.SUCCESS);
return hintMTMVs;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this means no mv can be written?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32769 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a449e197b70d05d70dd9b298276ea68ea0cb1ac3, data reload: false

------ Round 1 ----------------------------------
q1	17608	6198	6078	6078
q2	2045	297	165	165
q3	10503	1286	742	742
q4	10231	877	440	440
q5	7554	2197	1984	1984
q6	205	179	150	150
q7	912	758	607	607
q8	9316	1378	1185	1185
q9	5252	4971	5025	4971
q10	6776	2292	1858	1858
q11	481	291	264	264
q12	351	372	221	221
q13	17770	3543	2947	2947
q14	237	233	208	208
q15	548	505	492	492
q16	644	631	595	595
q17	581	857	321	321
q18	7161	6588	6451	6451
q19	1722	981	598	598
q20	314	314	185	185
q21	2905	2199	2003	2003
q22	364	340	304	304
Total cold run time: 103480 ms
Total hot run time: 32769 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6273	6216	6219	6216
q2	236	322	232	232
q3	2244	2656	2310	2310
q4	1410	1831	1372	1372
q5	4335	4725	4878	4725
q6	189	189	148	148
q7	2110	1966	1769	1769
q8	2643	2810	2643	2643
q9	7318	7262	7263	7262
q10	3058	3330	2789	2789
q11	581	533	495	495
q12	672	758	621	621
q13	3381	3768	3124	3124
q14	298	315	278	278
q15	587	498	495	495
q16	633	685	628	628
q17	1204	1709	1244	1244
q18	7671	7249	7137	7137
q19	813	1097	1034	1034
q20	1885	1976	1836	1836
q21	5604	5151	4982	4982
q22	602	645	590	590
Total cold run time: 53747 ms
Total hot run time: 51930 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190591 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a449e197b70d05d70dd9b298276ea68ea0cb1ac3, data reload: false

query1	981	371	378	371
query2	6539	2370	2353	2353
query3	6704	213	211	211
query4	33904	23567	23447	23447
query5	4332	632	486	486
query6	286	207	177	177
query7	4620	500	311	311
query8	318	252	239	239
query9	9589	2769	2743	2743
query10	464	313	263	263
query11	18119	15639	15288	15288
query12	167	114	107	107
query13	1692	551	429	429
query14	10858	7428	6733	6733
query15	230	206	194	194
query16	7930	587	432	432
query17	1552	725	541	541
query18	2073	388	299	299
query19	218	181	146	146
query20	118	115	112	112
query21	209	119	100	100
query22	4653	4740	4566	4566
query23	35711	33671	33597	33597
query24	6307	2165	2287	2165
query25	457	438	379	379
query26	922	253	150	150
query27	2025	449	321	321
query28	5241	2460	2434	2434
query29	526	551	424	424
query30	227	180	148	148
query31	1017	863	818	818
query32	69	59	65	59
query33	492	350	324	324
query34	741	830	501	501
query35	786	823	730	730
query36	1020	1062	967	967
query37	124	95	75	75
query38	4198	4088	4147	4088
query39	1495	1404	1445	1404
query40	210	112	99	99
query41	45	50	45	45
query42	116	102	99	99
query43	521	520	477	477
query44	1296	807	793	793
query45	187	173	164	164
query46	879	1050	657	657
query47	1902	1895	1880	1880
query48	378	420	322	322
query49	762	484	386	386
query50	625	650	387	387
query51	7159	7094	7160	7094
query52	114	100	88	88
query53	224	247	189	189
query54	464	485	394	394
query55	80	76	78	76
query56	257	261	256	256
query57	1196	1158	1127	1127
query58	241	232	242	232
query59	2961	3219	2933	2933
query60	274	259	247	247
query61	118	113	119	113
query62	853	798	734	734
query63	222	179	190	179
query64	3890	1012	681	681
query65	3316	3231	3296	3231
query66	1057	435	309	309
query67	16168	15911	15671	15671
query68	8066	770	526	526
query69	469	296	251	251
query70	1214	1098	1163	1098
query71	436	293	248	248
query72	6039	3919	3781	3781
query73	657	753	361	361
query74	10163	9005	9090	9005
query75	4411	3060	2659	2659
query76	3809	1214	793	793
query77	784	374	269	269
query78	10009	10090	9376	9376
query79	3682	909	606	606
query80	732	509	444	444
query81	464	263	234	234
query82	626	152	172	152
query83	205	163	144	144
query84	281	92	71	71
query85	782	366	302	302
query86	349	320	305	305
query87	4534	4703	4388	4388
query88	4106	2250	2214	2214
query89	400	342	294	294
query90	1928	188	185	185
query91	133	134	109	109
query92	64	57	60	57
query93	963	880	544	544
query94	653	397	286	286
query95	334	262	248	248
query96	485	604	276	276
query97	2756	2819	2713	2713
query98	238	205	195	195
query99	1679	1557	1462	1462
Total cold run time: 293880 ms
Total hot run time: 190591 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a449e197b70d05d70dd9b298276ea68ea0cb1ac3, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.39	0.41
query6	1.16	0.66	0.65
query7	0.03	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.53	0.51
query10	0.54	0.57	0.55
query11	0.15	0.10	0.11
query12	0.14	0.11	0.10
query13	0.60	0.60	0.59
query14	2.73	2.76	2.73
query15	0.89	0.81	0.82
query16	0.39	0.40	0.37
query17	1.03	1.05	1.07
query18	0.22	0.22	0.22
query19	1.91	1.87	1.99
query20	0.01	0.02	0.00
query21	15.36	0.94	0.58
query22	0.74	0.73	0.70
query23	15.39	1.38	0.54
query24	3.27	0.95	1.34
query25	0.28	0.20	0.14
query26	0.19	0.16	0.13
query27	0.05	0.05	0.05
query28	14.39	1.48	1.05
query29	12.61	3.94	3.30
query30	0.24	0.08	0.07
query31	2.84	0.59	0.39
query32	3.23	0.54	0.46
query33	3.18	3.10	3.07
query34	16.67	5.22	4.50
query35	4.57	4.53	4.51
query36	0.66	0.48	0.47
query37	0.09	0.07	0.06
query38	0.05	0.03	0.04
query39	0.04	0.02	0.03
query40	0.17	0.13	0.14
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.94 s
Total hot run time: 31.43 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32375 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

------ Round 1 ----------------------------------
q1	17595	6139	6034	6034
q2	2044	291	165	165
q3	10430	1203	756	756
q4	10205	865	426	426
q5	7559	2194	1962	1962
q6	205	179	152	152
q7	875	731	619	619
q8	9234	1324	1106	1106
q9	5284	4886	4879	4879
q10	6730	2273	1865	1865
q11	469	285	258	258
q12	348	353	224	224
q13	17797	3700	2976	2976
q14	237	222	208	208
q15	572	503	497	497
q16	641	631	594	594
q17	562	845	327	327
q18	6876	6381	6274	6274
q19	2213	951	549	549
q20	307	309	187	187
q21	2880	2291	2005	2005
q22	358	346	312	312
Total cold run time: 103421 ms
Total hot run time: 32375 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6297	6267	6165	6165
q2	236	322	228	228
q3	2271	2653	2304	2304
q4	1394	1859	1351	1351
q5	4302	4709	4750	4709
q6	183	177	150	150
q7	2069	1952	1816	1816
q8	2624	2795	2692	2692
q9	7313	7319	7399	7319
q10	3094	3365	2847	2847
q11	569	513	503	503
q12	645	713	570	570
q13	3475	3778	3154	3154
q14	280	313	285	285
q15	549	525	519	519
q16	652	689	673	673
q17	1238	1737	1254	1254
q18	7668	7449	7267	7267
q19	858	1177	1057	1057
q20	2017	1980	1872	1872
q21	5867	5378	4993	4993
q22	611	608	590	590
Total cold run time: 54212 ms
Total hot run time: 52318 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196586 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

query1	1286	918	922	918
query2	6481	2247	2254	2247
query3	11097	4937	4969	4937
query4	33168	23740	23288	23288
query5	4024	614	463	463
query6	281	207	186	186
query7	3997	493	303	303
query8	289	241	234	234
query9	9518	2744	2759	2744
query10	459	305	240	240
query11	17901	15464	15249	15249
query12	149	109	98	98
query13	1560	550	420	420
query14	10187	6786	7781	6786
query15	249	210	211	210
query16	8605	631	451	451
query17	1650	797	590	590
query18	2013	407	342	342
query19	203	192	167	167
query20	124	121	122	121
query21	205	127	109	109
query22	4721	4701	4542	4542
query23	34382	33246	33721	33246
query24	6587	2257	2380	2257
query25	472	495	406	406
query26	755	273	146	146
query27	2066	473	335	335
query28	5758	2515	2515	2515
query29	591	552	414	414
query30	204	186	158	158
query31	984	922	859	859
query32	68	80	56	56
query33	492	353	305	305
query34	772	866	544	544
query35	825	828	792	792
query36	1036	1060	987	987
query37	113	100	77	77
query38	4321	4280	4306	4280
query39	1689	1490	1488	1488
query40	204	118	97	97
query41	43	44	45	44
query42	125	99	102	99
query43	521	532	482	482
query44	1381	812	853	812
query45	194	184	172	172
query46	876	1069	656	656
query47	2038	2026	2007	2007
query48	388	414	331	331
query49	704	470	386	386
query50	649	680	416	416
query51	7432	7314	7212	7212
query52	110	101	96	96
query53	226	249	180	180
query54	484	497	426	426
query55	81	83	79	79
query56	257	279	246	246
query57	1267	1245	1144	1144
query58	244	221	237	221
query59	2984	3391	3236	3236
query60	279	269	248	248
query61	115	108	106	106
query62	870	829	771	771
query63	236	206	201	201
query64	2957	1019	658	658
query65	3300	3319	3340	3319
query66	815	410	304	304
query67	16662	15961	15537	15537
query68	10001	829	532	532
query69	495	306	261	261
query70	1220	1122	1138	1122
query71	439	297	266	266
query72	6266	3844	3784	3784
query73	718	763	365	365
query74	9982	9161	8871	8871
query75	4640	3153	2651	2651
query76	5490	1192	797	797
query77	1021	363	275	275
query78	9942	10262	9459	9459
query79	3357	922	598	598
query80	705	523	458	458
query81	476	279	230	230
query82	277	154	123	123
query83	243	160	149	149
query84	281	91	73	73
query85	746	345	304	304
query86	347	306	302	302
query87	4514	4481	4435	4435
query88	3278	2242	2216	2216
query89	434	334	292	292
query90	2025	189	192	189
query91	135	148	106	106
query92	67	54	52	52
query93	2033	916	534	534
query94	662	425	295	295
query95	343	286	259	259
query96	484	626	275	275
query97	2769	2816	2665	2665
query98	214	203	194	194
query99	1660	1564	1458	1458
Total cold run time: 299988 ms
Total hot run time: 196586 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

query1	0.03	0.03	0.02
query2	0.08	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.11	0.10
query5	0.43	0.39	0.42
query6	1.14	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.49	0.53
query10	0.55	0.57	0.57
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.73	2.85	2.74
query15	0.90	0.82	0.85
query16	0.38	0.40	0.38
query17	1.02	1.05	1.07
query18	0.24	0.21	0.20
query19	1.97	1.87	1.97
query20	0.01	0.00	0.01
query21	15.36	0.96	0.60
query22	0.74	0.76	0.56
query23	15.42	1.50	0.65
query24	3.16	1.72	1.59
query25	0.18	0.21	0.10
query26	0.20	0.15	0.13
query27	0.05	0.04	0.05
query28	14.34	1.47	1.04
query29	12.56	3.99	3.27
query30	0.25	0.09	0.06
query31	2.83	0.62	0.40
query32	3.22	0.55	0.46
query33	3.10	3.17	3.15
query34	16.73	5.13	4.55
query35	4.52	4.50	4.53
query36	0.66	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.67 s
Total hot run time: 32.13 s

@LiBinfeng-01
Copy link
Collaborator Author

run p0

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32778 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

------ Round 1 ----------------------------------
q1	17580	6190	6067	6067
q2	2039	316	173	173
q3	10414	1255	730	730
q4	10208	888	444	444
q5	7514	2186	1987	1987
q6	205	186	150	150
q7	903	760	593	593
q8	9241	1351	1159	1159
q9	5200	4919	4964	4919
q10	6751	2295	1857	1857
q11	513	274	255	255
q12	359	361	230	230
q13	17772	3589	2908	2908
q14	247	229	205	205
q15	573	489	495	489
q16	624	622	606	606
q17	559	865	330	330
q18	7078	6888	6556	6556
q19	2053	992	583	583
q20	312	320	191	191
q21	2973	2332	2019	2019
q22	382	361	327	327
Total cold run time: 103500 ms
Total hot run time: 32778 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6220	6279	6252	6252
q2	234	329	242	242
q3	2245	2665	2340	2340
q4	1414	1822	1397	1397
q5	4338	4755	4866	4755
q6	200	191	144	144
q7	2072	1975	1792	1792
q8	2695	2800	2704	2704
q9	7312	7263	7277	7263
q10	3102	3344	2840	2840
q11	562	525	487	487
q12	661	757	630	630
q13	3379	3789	3169	3169
q14	279	296	281	281
q15	592	543	513	513
q16	662	684	661	661
q17	1246	1754	1264	1264
q18	7653	7459	7353	7353
q19	826	1182	1124	1124
q20	2019	2035	1919	1919
q21	5886	5344	4884	4884
q22	629	584	583	583
Total cold run time: 54226 ms
Total hot run time: 52597 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195973 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

query1	1305	916	937	916
query2	6508	2427	2310	2310
query3	10910	4654	4853	4654
query4	33343	23795	23381	23381
query5	4734	624	466	466
query6	281	201	179	179
query7	3980	490	297	297
query8	296	240	228	228
query9	9555	2757	2767	2757
query10	491	312	244	244
query11	17970	15467	15115	15115
query12	154	109	105	105
query13	1605	548	416	416
query14	9123	6762	6893	6762
query15	240	228	195	195
query16	8001	596	445	445
query17	1529	790	606	606
query18	2077	435	346	346
query19	215	181	157	157
query20	120	124	116	116
query21	203	127	107	107
query22	4830	4728	4407	4407
query23	34544	33466	33789	33466
query24	7713	2294	2420	2294
query25	451	465	389	389
query26	1201	290	159	159
query27	2761	471	324	324
query28	6581	2488	2467	2467
query29	651	556	453	453
query30	209	185	151	151
query31	995	902	831	831
query32	75	60	58	58
query33	484	356	287	287
query34	774	868	531	531
query35	826	847	734	734
query36	1026	1066	975	975
query37	116	104	75	75
query38	4142	4295	4353	4295
query39	1550	1573	1433	1433
query40	213	118	100	100
query41	44	45	44	44
query42	120	104	103	103
query43	526	537	497	497
query44	1272	819	823	819
query45	188	174	163	163
query46	896	1087	668	668
query47	1993	2032	1944	1944
query48	387	412	328	328
query49	749	476	384	384
query50	655	699	390	390
query51	7346	7308	7034	7034
query52	106	100	91	91
query53	240	260	186	186
query54	478	512	400	400
query55	79	85	86	85
query56	255	254	258	254
query57	1257	1237	1186	1186
query58	248	223	232	223
query59	3252	3419	3169	3169
query60	274	253	244	244
query61	112	110	107	107
query62	864	830	769	769
query63	231	196	199	196
query64	3874	1038	660	660
query65	3399	3205	3213	3205
query66	931	416	351	351
query67	16560	15919	15529	15529
query68	8618	747	519	519
query69	478	298	243	243
query70	1204	1158	1089	1089
query71	439	319	260	260
query72	5926	3818	3836	3818
query73	650	744	359	359
query74	10242	9238	9162	9162
query75	4600	3161	2665	2665
query76	4515	1189	760	760
query77	815	356	277	277
query78	10077	10335	9418	9418
query79	2041	894	597	597
query80	673	511	480	480
query81	479	268	230	230
query82	398	152	126	126
query83	191	162	150	150
query84	286	100	67	67
query85	779	359	308	308
query86	343	291	282	282
query87	4442	4639	4356	4356
query88	3378	2241	2208	2208
query89	400	335	300	300
query90	1883	187	187	187
query91	131	133	112	112
query92	67	57	52	52
query93	1535	887	510	510
query94	655	391	278	278
query95	343	265	253	253
query96	476	609	284	284
query97	2756	2797	2716	2716
query98	218	209	195	195
query99	1683	1533	1452	1452
Total cold run time: 298839 ms
Total hot run time: 195973 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.23 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.40	0.42
query6	1.14	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.51	0.50
query10	0.56	0.57	0.55
query11	0.15	0.11	0.10
query12	0.14	0.12	0.11
query13	0.60	0.60	0.59
query14	2.83	2.82	2.75
query15	0.89	0.83	0.82
query16	0.39	0.37	0.38
query17	1.01	1.07	1.04
query18	0.23	0.22	0.20
query19	1.86	1.82	2.03
query20	0.02	0.01	0.01
query21	15.38	0.95	0.56
query22	0.76	0.87	0.68
query23	15.21	1.47	0.59
query24	3.27	0.98	0.93
query25	0.19	0.15	0.11
query26	0.25	0.14	0.13
query27	0.04	0.08	0.04
query28	13.97	1.53	1.04
query29	12.61	3.98	3.28
query30	0.24	0.10	0.06
query31	2.83	0.59	0.38
query32	3.23	0.56	0.45
query33	3.15	3.16	3.09
query34	16.92	5.14	4.47
query35	4.46	4.47	4.42
query36	0.65	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.03	0.02
query40	0.17	0.13	0.13
query41	0.07	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.5 s
Total hot run time: 31.23 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32797 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

------ Round 1 ----------------------------------
q1	17568	6232	6103	6103
q2	2047	312	173	173
q3	10450	1238	767	767
q4	10556	895	444	444
q5	9353	2197	1963	1963
q6	212	178	150	150
q7	907	735	602	602
q8	9231	1389	1219	1219
q9	5289	5028	4948	4948
q10	6802	2321	1862	1862
q11	488	289	258	258
q12	346	373	215	215
q13	17793	3573	2994	2994
q14	238	221	209	209
q15	572	507	518	507
q16	647	637	612	612
q17	554	855	323	323
q18	6852	6441	6360	6360
q19	4579	983	592	592
q20	298	319	184	184
q21	2795	2280	2002	2002
q22	361	336	310	310
Total cold run time: 107938 ms
Total hot run time: 32797 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6485	6287	6279	6279
q2	238	328	237	237
q3	2248	2615	2320	2320
q4	1385	1821	1348	1348
q5	4390	4910	4941	4910
q6	182	177	148	148
q7	2149	2019	1830	1830
q8	2630	2843	2674	2674
q9	7278	7340	7311	7311
q10	3070	3288	2820	2820
q11	592	538	508	508
q12	679	745	627	627
q13	3451	3736	3157	3157
q14	291	308	288	288
q15	557	508	506	506
q16	665	708	644	644
q17	1208	1772	1247	1247
q18	7691	7388	7479	7388
q19	852	1167	1119	1119
q20	1967	2013	1918	1918
q21	5716	5091	5028	5028
q22	634	605	585	585
Total cold run time: 54358 ms
Total hot run time: 52892 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196907 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

query1	1321	1020	930	930
query2	6420	2347	2312	2312
query3	11055	4534	4849	4534
query4	33394	23676	23451	23451
query5	4316	621	449	449
query6	285	211	197	197
query7	3994	496	301	301
query8	311	264	245	245
query9	9273	2738	2738	2738
query10	458	312	235	235
query11	18088	15422	15163	15163
query12	155	113	103	103
query13	1567	551	407	407
query14	10570	7233	6924	6924
query15	242	214	186	186
query16	7882	627	446	446
query17	1540	773	612	612
query18	2068	404	304	304
query19	204	182	151	151
query20	122	117	119	117
query21	202	126	107	107
query22	4629	4587	4361	4361
query23	34658	33782	33620	33620
query24	6427	2354	2349	2349
query25	508	487	440	440
query26	816	280	163	163
query27	2087	477	365	365
query28	5683	2540	2471	2471
query29	611	539	432	432
query30	206	194	159	159
query31	972	950	850	850
query32	74	58	55	55
query33	472	343	308	308
query34	768	855	549	549
query35	805	830	767	767
query36	1020	1063	987	987
query37	113	99	78	78
query38	4326	4399	4189	4189
query39	1559	1514	1471	1471
query40	213	118	102	102
query41	44	43	44	43
query42	119	104	108	104
query43	517	535	502	502
query44	1317	821	813	813
query45	189	180	179	179
query46	896	1062	666	666
query47	1954	2016	1901	1901
query48	404	429	349	349
query49	744	502	414	414
query50	665	691	401	401
query51	7367	7337	7315	7315
query52	108	118	93	93
query53	248	275	202	202
query54	512	508	439	439
query55	80	76	80	76
query56	256	256	254	254
query57	1206	1251	1166	1166
query58	236	231	219	219
query59	3107	3307	3208	3208
query60	266	271	264	264
query61	112	111	115	111
query62	890	827	770	770
query63	245	192	190	190
query64	3396	1061	666	666
query65	3302	3198	3292	3198
query66	826	410	305	305
query67	16365	15801	15413	15413
query68	9455	788	535	535
query69	495	300	245	245
query70	1194	1174	1163	1163
query71	432	282	249	249
query72	5992	3918	3910	3910
query73	656	753	370	370
query74	10088	9037	8893	8893
query75	4585	3171	2665	2665
query76	4238	1196	776	776
query77	815	360	275	275
query78	9890	10143	9518	9518
query79	2902	856	601	601
query80	703	515	432	432
query81	481	266	240	240
query82	598	155	126	126
query83	198	166	146	146
query84	281	91	79	79
query85	792	371	307	307
query86	357	331	290	290
query87	4498	4548	4484	4484
query88	3458	2284	2219	2219
query89	412	334	294	294
query90	1849	189	189	189
query91	135	128	111	111
query92	68	55	53	53
query93	1095	876	546	546
query94	662	408	294	294
query95	327	277	250	250
query96	494	605	277	277
query97	2745	2857	2814	2814
query98	211	200	199	199
query99	1695	1551	1468	1468
Total cold run time: 296586 ms
Total hot run time: 196907 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.55 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e077958cbfd03460f2a771423ff9722e89ae16ba, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.43	0.40	0.40
query6	1.17	0.66	0.64
query7	0.03	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.53	0.51
query10	0.55	0.57	0.56
query11	0.15	0.11	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.61
query14	2.84	2.82	2.76
query15	0.91	0.82	0.82
query16	0.39	0.39	0.40
query17	1.06	1.05	1.06
query18	0.22	0.20	0.21
query19	1.99	1.79	2.02
query20	0.01	0.01	0.02
query21	15.37	0.91	0.57
query22	0.75	0.73	0.64
query23	15.39	1.43	0.58
query24	2.92	1.07	1.76
query25	0.25	0.15	0.21
query26	0.24	0.14	0.13
query27	0.05	0.04	0.04
query28	14.30	1.49	1.05
query29	12.60	4.00	3.25
query30	0.26	0.09	0.06
query31	2.82	0.61	0.38
query32	3.24	0.56	0.47
query33	3.12	3.17	3.10
query34	16.53	5.13	4.54
query35	4.52	4.51	4.55
query36	0.68	0.48	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.02
Total cold run time: 106.6 s
Total hot run time: 31.55 s

Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 30, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@englefly englefly merged commit 52ac59c into apache:master Dec 30, 2024
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants