Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](pipeline) FragmentMgr may return OK when some instance prepare failed #29353

Closed
wants to merge 3 commits into from
Closed

[fix](pipeline) FragmentMgr may return OK when some instance prepare failed #29353

wants to merge 3 commits into from

Conversation

zhiqiang-hhhh
Copy link
Contributor

If fragmentmgr gets lock after instances prepare finished, it will retrun Status::OK instead of check prepare status of instance.

@zhiqiang-hhhh
Fix
129e28a 
N

Ne
@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.63% (8614/23519)
Line Coverage: 28.68% (70010/244088)
Region Coverage: 27.67% (36240/130964)
Branch Coverage: 24.37% (18516/75978)
Coverage Report: http://coverage.selectdb-in.cc/coverage/129e28ab4a3f69194cab256f74060ca9cb154a2d_129e28ab4a3f69194cab256f74060ca9cb154a2d/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 129e28ab4a3f69194cab256f74060ca9cb154a2d, data reload: false

------ Round 1 ----------------------------------
q1	18694	5592	5073	5073
q2	2026	162	149	149
q3	10606	1081	1070	1070
q4	10222	783	851	783
q5	7802	2926	2915	2915
q6	212	129	130	129
q7	937	505	565	505
q8	9275	2023	2039	2023
q9	6884	6418	6437	6418
q10	8255	3058	3063	3058
q11	436	235	221	221
q12	395	239	242	239
q13	18024	3677	3592	3592
q14	251	210	205	205
q15	588	539	534	534
q16	460	417	386	386
q17	986	466	430	430
q18	7500	6783	6670	6670
q19	1598	1350	1363	1350
q20	737	332	329	329
q21	2796	2365	2415	2365
q22	393	330	337	330
Total cold run time: 109077 ms
Total hot run time: 38774 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5102	5092	5128	5092
q2	342	259	255	255
q3	3337	3287	3239	3239
q4	2100	2008	2002	2002
q5	5813	5913	6077	5913
q6	222	129	127	127
q7	2431	2092	2132	2092
q8	3393	3448	3500	3448
q9	8917	8853	8804	8804
q10	3838	3826	3863	3826
q11	605	486	516	486
q12	811	648	660	648
q13	7916	3234	3183	3183
q14	285	278	261	261
q15	600	540	533	533
q16	562	503	506	503
q17	1932	1784	1775	1775
q18	8721	8465	8331	8331
q19	1621	1577	1572	1572
q20	2231	1967	1962	1962
q21	5567	5343	5343	5343
q22	547	496	468	468
Total cold run time: 66893 ms
Total hot run time: 59863 ms

@doris-robot
Copy link

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit 129e28ab4a3f69194cab256f74060ca9cb154a2d, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	876	367	351	351
query2	4961	2085	2051	2051
query3	5169	206	200	200
query4	26674	22562	22487	22487
query5	4885	532	541	532
query6	262	186	187	186
query7	4144	269	272	269
query8	223	208	204	204
query9	7335	2655	2516	2516
query10	410	284	265	265
query11	16287	15691	15489	15489
query12	136	81	79	79
query13	1541	317	328	317
query14	10792	7182	7184	7182
query15	242	191	197	191
query16	6203	286	268	268
query17	1651	502	502	502
query18	1847	276	259	259
query19	255	146	140	140
query20	86	81	81	81
query21	171	98	98	98
query22	4980	4805	4628	4628
query23	32227	31278	31261	31261
query24	11261	2868	2839	2839
query25	587	348	345	345
query26	1593	149	149	149
query27	2818	294	295	294
query28	6711	1957	1956	1956
query29	1838	392	388	388
query30	292	148	146	146
query31	1004	787	799	787
query32	92	65	63	63
query33	716	279	287	279
query34	869	466	450	450
query35	874	777	755	755
query36	1306	1338	1218	1218
query37	166	76	79	76
query38	3397	3249	3281	3249
query39	1312	1291	1274	1274
query40	295	107	94	94
query41	43	36	35	35
query42	103	94	97	94
query43	574	493	510	493
query44	1065	711	716	711
query45	197	198	187	187
query46	1069	648	656	648
query47	1690	1607	1539	1539
query48	337	265	259	259
query49	1058	338	331	331
query50	793	366	364	364
query51	5392	5292	5252	5252
query52	101	87	91	87
query53	223	157	147	147
query54	1498	565	619	565
query55	101	89	84	84
query56	212	204	205	204
query57	1029	958	942	942
query58	238	213	207	207
query59	2975	2699	2682	2682
query60	259	227	238	227
query61	88	87	88	87
query62	625	481	460	460
query63	163	156	154	154
query64	5557	1689	1696	1689
query65	3355	3292	3285	3285
query66	1294	349	336	336
query67	16074	15232	15338	15232
query68	12176	536	513	513
query69	502	278	254	254
query70	1791	1525	1474	1474
query71	472	239	236	236
query72	5646	3598	3547	3547
query73	2810	307	311	307
query74	6917	6486	6508	6486
query75	5308	2335	2253	2253
query76	6238	1143	1160	1143
query77	696	308	304	304
query78	9098	8768	8552	8552
query79	1061	513	505	505
query80	555	370	364	364
query81	452	204	207	204
query82	190	102	107	102
query83	181	137	145	137
query84	243	53	58	53
query85	916	298	281	281
query86	371	377	387	377
query87	3592	3380	3354	3354
query88	2854	2267	2255	2255
query89	338	269	257	257
query90	1883	218	219	218
query91	124	93	89	89
query92	62	56	56	56
query93	1398	512	446	446
query94	852	193	190	190
query95	468	415	416	415
query96	620	317	313	313
query97	4320	4168	4152	4152
query98	240	212	195	195
query99	1076	848	858	848
Total cold run time: 286051 ms
Total hot run time: 180099 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.18 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17183842820 Bytes

@zhiqiang-hhhh
Copy link
Contributor Author

run pipelinex

@zhiqiang-hhhh
Copy link
Contributor Author

run pipelinex_p0

@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 13e19270ecd5a03de5ade6a41049b1be1eb966f2, data reload: false

------ Round 1 ----------------------------------
q1	17647	5167	5245	5167
q2	2016	157	144	144
q3	10531	1106	1117	1106
q4	10171	767	851	767
q5	7804	2906	2923	2906
q6	214	131	133	131
q7	913	562	503	503
q8	9282	2001	1992	1992
q9	6806	6424	6368	6368
q10	8197	3045	3063	3045
q11	437	228	220	220
q12	384	234	231	231
q13	17997	3660	3598	3598
q14	241	216	205	205
q15	579	535	536	535
q16	444	389	381	381
q17	944	490	465	465
q18	7364	6655	6657	6655
q19	1563	1398	1408	1398
q20	725	354	341	341
q21	2769	2373	2402	2373
q22	387	327	345	327
Total cold run time: 107415 ms
Total hot run time: 38858 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5198	5124	5034	5034
q2	341	243	260	243
q3	3318	3291	3252	3252
q4	2101	2025	1952	1952
q5	5806	5785	5779	5779
q6	215	128	120	120
q7	2300	1897	1917	1897
q8	3355	3440	3433	3433
q9	8810	8722	8711	8711
q10	3791	3864	3839	3839
q11	583	481	484	481
q12	792	682	647	647
q13	7246	3167	3231	3167
q14	305	265	282	265
q15	590	537	545	537
q16	557	499	489	489
q17	1921	1791	1728	1728
q18	8715	8286	8272	8272
q19	1616	1583	1590	1583
q20	2196	1979	1969	1969
q21	5480	5276	5321	5276
q22	525	493	493	493
Total cold run time: 65761 ms
Total hot run time: 59167 ms

@doris-robot
Copy link

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit 13e19270ecd5a03de5ade6a41049b1be1eb966f2, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	928	358	344	344
query2	6426	1929	1880	1880
query3	6663	206	199	199
query4	28620	22296	22538	22296
query5	5978	587	524	524
query6	282	210	192	192
query7	4586	284	264	264
query8	229	206	195	195
query9	8171	2723	2683	2683
query10	457	247	269	247
query11	16166	15526	15520	15520
query12	138	80	76	76
query13	1662	325	328	325
query14	12145	7117	7247	7117
query15	256	195	206	195
query16	6342	285	278	278
query17	1786	502	494	494
query18	1922	273	271	271
query19	262	142	144	142
query20	83	79	78	78
query21	185	96	94	94
query22	4959	4724	4512	4512
query23	32128	31338	31197	31197
query24	11823	2798	2837	2798
query25	572	352	350	350
query26	1714	146	146	146
query27	2829	281	272	272
query28	6893	1946	1932	1932
query29	2065	385	407	385
query30	285	140	143	140
query31	948	767	765	765
query32	89	61	58	58
query33	729	278	269	269
query34	895	450	434	434
query35	918	743	766	743
query36	1317	1200	1156	1156
query37	184	76	76	76
query38	3364	3318	3253	3253
query39	1312	1301	1284	1284
query40	304	96	93	93
query41	37	36	40	36
query42	97	92	94	92
query43	518	496	508	496
query44	1040	703	720	703
query45	199	184	183	183
query46	1076	645	629	629
query47	1692	1541	1615	1541
query48	342	261	249	249
query49	1209	325	321	321
query50	738	346	340	340
query51	5402	5228	5347	5228
query52	90	90	81	81
query53	216	153	154	153
query54	1420	560	573	560
query55	99	89	87	87
query56	222	203	198	198
query57	1038	963	944	944
query58	234	209	209	209
query59	2748	2564	2668	2564
query60	249	240	228	228
query61	88	88	87	87
query62	658	443	482	443
query63	172	147	146	146
query64	5832	1773	1690	1690
query65	3333	3264	3270	3264
query66	1246	336	337	336
query67	15546	15317	15144	15144
query68	12874	555	536	536
query69	542	248	247	247
query70	1671	1530	1515	1515
query71	494	238	233	233
query72	5633	3606	3588	3588
query73	2966	312	305	305
query74	7019	6454	6346	6346
query75	5241	2268	2253	2253
query76	6323	1129	1153	1129
query77	656	257	287	257
query78	9074	9012	8506	8506
query79	1131	518	498	498
query80	659	379	379	379
query81	461	224	209	209
query82	203	105	116	105
query83	182	135	132	132
query84	247	56	56	56
query85	956	307	297	297
query86	377	361	402	361
query87	3582	3348	3365	3348
query88	2920	2242	2243	2242
query89	325	259	265	259
query90	1865	213	209	209
query91	129	92	94	92
query92	59	51	55	51
query93	1535	445	431	431
query94	934	185	189	185
query95	463	428	415	415
query96	613	314	314	314
query97	4307	4164	4174	4164
query98	209	194	190	190
query99	1139	817	828	817
Total cold run time: 296945 ms
Total hot run time: 178898 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 13e19270ecd5a03de5ade6a41049b1be1eb966f2, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5505	5135	5112	5112
q2	392	162	157	157
q3	1458	1217	1159	1159
q4	1089	775	823	775
q5	3178	3101	3132	3101
q6	223	137	142	137
q7	985	534	521	521
q8	2154	2227	2228	2227
q9	6679	6659	6649	6649
q10	3177	3180	3127	3127
q11	337	217	202	202
q12	381	239	237	237
q13	4417	3606	3632	3606
q14	247	205	212	205
q15	596	542	534	534
q16	438	433	444	433
q17	1051	555	502	502
q18	7074	6901	6855	6855
q19	1651	1536	1452	1452
q20	607	491	331	331
q21	2897	2507	2505	2505
q22	405	324	349	324
Total cold run time: 44941 ms
Total hot run time: 40151 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5101	5102	5062	5062
q2	352	240	231	231
q3	3390	3338	3298	3298
q4	2125	2062	2029	2029
q5	5947	5937	5936	5936
q6	233	128	130	128
q7	2371	1917	1934	1917
q8	3576	3652	3663	3652
q9	9024	9063	8976	8976
q10	3890	3895	3933	3895
q11	588	481	486	481
q12	800	692	696	692
q13	3889	3203	3195	3195
q14	293	278	272	272
q15	610	545	534	534
q16	564	528	489	489
q17	2024	1796	1799	1796
q18	8737	8413	8336	8336
q19	1742	1659	1697	1659
q20	2275	1994	1979	1979
q21	5772	5330	5364	5330
q22	580	478	493	478
Total cold run time: 63883 ms
Total hot run time: 60365 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.45 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.4 seconds inserted 10000000 Rows, about 352K ops/s
storage size: 17183887870 Bytes

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.63% (8612/23512)
Line Coverage: 28.68% (69987/244040)
Region Coverage: 27.67% (36225/130938)
Branch Coverage: 24.36% (18507/75964)
Coverage Report: http://coverage.selectdb-in.cc/coverage/13e19270ecd5a03de5ade6a41049b1be1eb966f2_13e19270ecd5a03de5ade6a41049b1be1eb966f2/report/index.html

@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38707 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 13e19270ecd5a03de5ade6a41049b1be1eb966f2, data reload: false

------ Round 1 ----------------------------------
q1	17678	4964	4884	4884
q2	2030	153	144	144
q3	10801	1133	1137	1133
q4	10527	781	830	781
q5	7796	3051	2935	2935
q6	205	122	123	122
q7	905	519	495	495
q8	9501	2026	2017	2017
q9	8317	6390	6422	6390
q10	8317	3063	3039	3039
q11	436	223	210	210
q12	386	235	236	235
q13	18000	3384	3391	3384
q14	236	210	202	202
q15	549	508	500	500
q16	488	393	391	391
q17	958	705	616	616
q18	7262	6823	6730	6730
q19	1594	1528	1520	1520
q20	725	303	328	303
q21	2755	2419	2368	2368
q22	362	308	310	308
Total cold run time: 109828 ms
Total hot run time: 38707 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4951	4960	4928	4928
q2	338	250	238	238
q3	3287	3281	3300	3281
q4	2110	2013	2026	2013
q5	5772	5785	5774	5774
q6	205	116	117	116
q7	2299	1901	1885	1885
q8	3363	3465	3489	3465
q9	8829	8798	8738	8738
q10	3837	3834	3860	3834
q11	542	448	450	448
q12	797	642	627	627
q13	3863	3188	3182	3182
q14	283	277	267	267
q15	545	512	508	508
q16	566	517	503	503
q17	1854	1864	1855	1855
q18	8584	8294	8308	8294
q19	1594	1645	1640	1640
q20	2200	1974	1935	1935
q21	5585	5434	5335	5335
q22	527	482	449	449
Total cold run time: 61931 ms
Total hot run time: 59315 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.65% (8637/23566)
Line Coverage: 28.67% (70194/244877)
Region Coverage: 27.62% (36305/131467)
Branch Coverage: 24.30% (18550/76328)
Coverage Report: http://coverage.selectdb-in.cc/coverage/13e19270ecd5a03de5ade6a41049b1be1eb966f2_13e19270ecd5a03de5ade6a41049b1be1eb966f2/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 184209 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 13e19270ecd5a03de5ade6a41049b1be1eb966f2, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	930	348	338	338
query2	6542	2117	2014	2014
query3	6711	224	225	224
query4	25716	22590	22601	22590
query5	5627	621	581	581
query6	349	202	202	202
query7	4631	304	298	298
query8	254	235	226	226
query9	8696	2896	2920	2896
query10	573	339	311	311
query11	16138	15565	15643	15565
query12	165	97	111	97
query13	1684	336	345	336
query14	12937	7391	7541	7391
query15	259	251	209	209
query16	6178	333	337	333
query17	1620	528	506	506
query18	1859	309	306	306
query19	252	157	157	157
query20	104	99	105	99
query21	187	91	92	91
query22	5279	5003	4857	4857
query23	32149	31424	31600	31424
query24	11861	3025	2907	2907
query25	533	368	365	365
query26	1503	174	164	164
query27	2885	317	311	311
query28	6992	2158	2109	2109
query29	1292	417	425	417
query30	286	146	149	146
query31	1033	837	819	819
query32	107	75	74	74
query33	776	325	312	312
query34	866	475	474	474
query35	968	865	840	840
query36	1436	1294	1271	1271
query37	202	82	80	80
query38	3489	3352	3359	3352
query39	1349	1286	1301	1286
query40	291	90	87	87
query41	40	35	36	35
query42	109	112	104	104
query43	582	540	541	540
query44	1282	790	814	790
query45	224	209	209	209
query46	1059	722	694	694
query47	1710	1635	1623	1623
query48	372	292	286	286
query49	1064	319	309	309
query50	737	369	327	327
query51	5422	5299	5300	5299
query52	107	98	101	98
query53	230	173	169	169
query54	1388	664	665	664
query55	103	107	101	101
query56	271	254	274	254
query57	1056	969	961	961
query58	292	278	286	278
query59	3135	3029	2913	2913
query60	311	260	297	260
query61	135	155	147	147
query62	596	479	493	479
query63	189	180	187	180
query64	4509	1793	1657	1657
query65	3413	3353	3345	3345
query66	1156	406	401	401
query67	15704	15691	15186	15186
query68	11642	538	542	538
query69	614	328	327	327
query70	1820	1658	1619	1619
query71	620	299	289	289
query72	5997	3500	3509	3500
query73	2217	336	343	336
query74	7055	6521	6496	6496
query75	5585	2316	2268	2268
query76	6711	1038	1128	1038
query77	785	349	350	349
query78	9136	8723	8621	8621
query79	1308	513	527	513
query80	539	375	345	345
query81	463	214	215	214
query82	224	103	105	103
query83	171	141	141	141
query84	258	57	60	57
query85	949	273	279	273
query86	441	414	445	414
query87	3614	3459	3420	3420
query88	2980	2482	2520	2482
query89	359	278	286	278
query90	1879	249	257	249
query91	123	94	98	94
query92	94	69	80	69
query93	1366	490	432	432
query94	856	254	246	246
query95	557	500	489	489
query96	646	333	343	333
query97	4312	4295	4259	4259
query98	231	210	205	205
query99	1105	852	838	838
Total cold run time: 294562 ms
Total hot run time: 184209 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.98 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.7 seconds inserted 10000000 Rows, about 361K ops/s
storage size: 17183850962 Bytes

@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38311 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2dc7f80ba43cd841d84708d9560ee725772a5f7f, data reload: false

------ Round 1 ----------------------------------
q1	17956	5008	4946	4946
q2	2049	149	138	138
q3	10791	1114	1139	1114
q4	10587	779	817	779
q5	7838	3025	3128	3025
q6	206	122	127	122
q7	892	505	484	484
q8	9308	1979	1982	1979
q9	6616	6441	6418	6418
q10	8249	3012	3047	3012
q11	424	222	205	205
q12	360	193	195	193
q13	18127	3440	3408	3408
q14	241	213	211	211
q15	561	517	508	508
q16	435	390	377	377
q17	939	496	528	496
q18	7308	6761	6651	6651
q19	1575	1286	1353	1286
q20	602	312	317	312
q21	2741	2346	2440	2346
q22	357	328	301	301
Total cold run time: 108162 ms
Total hot run time: 38311 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4987	4997	4906	4906
q2	315	201	201	201
q3	3306	3304	3294	3294
q4	2239	2250	2230	2230
q5	5800	5780	5798	5780
q6	193	117	121	117
q7	2315	1831	1840	1831
q8	3445	3566	3542	3542
q9	8828	8879	8801	8801
q10	3739	3822	3786	3786
q11	544	431	439	431
q12	783	606	600	600
q13	4100	3222	3190	3190
q14	289	248	273	248
q15	567	517	513	513
q16	516	465	492	465
q17	2046	2045	2008	2008
q18	8837	8250	8291	8250
q19	1620	1623	1603	1603
q20	2150	1933	1905	1905
q21	5910	5801	5574	5574
q22	581	481	489	481
Total cold run time: 63110 ms
Total hot run time: 59756 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.66% (8643/23579)
Line Coverage: 28.71% (70416/245275)
Region Coverage: 27.64% (36383/131640)
Branch Coverage: 24.33% (18601/76446)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2dc7f80ba43cd841d84708d9560ee725772a5f7f_2dc7f80ba43cd841d84708d9560ee725772a5f7f/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 178764 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2dc7f80ba43cd841d84708d9560ee725772a5f7f, data reload: false

query1	929	342	329	329
query2	6723	1928	1851	1851
query3	6721	213	211	211
query4	26036	22284	22159	22159
query5	5926	490	511	490
query6	272	194	197	194
query7	4620	285	278	278
query8	238	218	204	204
query9	9107	2761	2771	2761
query10	523	240	244	240
query11	16229	15325	15401	15325
query12	132	82	83	82
query13	1637	338	357	338
query14	12304	7198	7256	7198
query15	269	188	198	188
query16	6458	277	273	273
query17	1922	500	478	478
query18	1953	267	270	267
query19	284	146	145	145
query20	85	82	78	78
query21	183	92	90	90
query22	4923	4750	4885	4750
query23	32718	31234	31370	31234
query24	11046	2840	2825	2825
query25	534	343	346	343
query26	857	158	153	153
query27	2505	275	282	275
query28	6146	1914	1909	1909
query29	912	412	379	379
query30	281	144	144	144
query31	988	784	762	762
query32	92	62	59	59
query33	741	277	282	277
query34	920	449	443	443
query35	886	785	734	734
query36	1302	1214	1263	1214
query37	105	68	66	66
query38	3361	3302	3316	3302
query39	1358	1275	1276	1275
query40	197	95	95	95
query41	39	35	35	35
query42	99	91	96	91
query43	543	506	527	506
query44	1129	704	720	704
query45	212	184	182	182
query46	1053	683	678	678
query47	1657	1628	1588	1588
query48	415	317	313	313
query49	1107	324	318	318
query50	734	332	339	332
query51	5392	5267	5302	5267
query52	93	96	94	94
query53	217	158	148	148
query54	1437	570	600	570
query55	106	89	86	86
query56	218	200	201	200
query57	1027	945	960	945
query58	237	211	220	211
query59	2813	2750	2695	2695
query60	251	251	242	242
query61	84	83	82	82
query62	649	451	456	451
query63	175	162	158	158
query64	4524	1685	1705	1685
query65	3350	3285	3273	3273
query66	991	336	338	336
query67	15690	15143	15886	15143
query68	11198	551	532	532
query69	535	282	293	282
query70	1735	1522	1543	1522
query71	503	217	210	210
query72	4928	2825	2832	2825
query73	2138	326	319	319
query74	6947	6451	6468	6451
query75	4930	2278	2269	2269
query76	6265	886	1074	886
query77	693	280	300	280
query78	9826	8630	8417	8417
query79	1044	522	502	502
query80	565	342	366	342
query81	470	212	217	212
query82	195	88	94	88
query83	163	140	137	137
query84	265	57	56	56
query85	966	294	287	287
query86	406	410	408	408
query87	3513	3387	3423	3387
query88	2839	2314	2335	2314
query89	330	272	267	267
query90	1877	211	208	208
query91	165	135	134	134
query92	64	56	53	53
query93	1594	494	490	490
query94	781	194	201	194
query95	482	440	432	432
query96	646	331	323	323
query97	4315	4149	4181	4149
query98	208	196	188	188
query99	1139	859	849	849
Total cold run time: 288390 ms
Total hot run time: 178764 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.8 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.2 seconds inserted 10000000 Rows, about 471K ops/s
storage size: 17184032133 Bytes

@doris-robot
Copy link

ClickBench: Total hot run time: 30.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2dc7f80ba43cd841d84708d9560ee725772a5f7f, data reload: false

query1	0.07	0.05	0.06
query2	0.06	0.03	0.02
query3	0.26	0.11	0.11
query4	1.76	0.13	0.12
query5	0.52	0.52	0.50
query6	1.32	0.66	0.63
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.56	0.50	0.50
query10	0.56	0.54	0.55
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.60	0.60	0.59
query14	0.79	0.79	0.79
query15	0.82	0.80	0.80
query16	0.36	0.36	0.37
query17	0.97	0.99	0.96
query18	0.27	0.26	0.26
query19	1.85	1.74	1.77
query20	0.01	0.01	0.01
query21	15.45	0.58	0.55
query22	2.56	2.39	1.45
query23	17.17	0.90	0.80
query24	6.01	0.78	0.96
query25	1.98	0.15	0.15
query26	0.14	0.13	0.13
query27	0.14	0.15	0.14
query28	9.59	0.81	0.83
query29	12.58	3.16	3.17
query30	0.52	0.49	0.49
query31	2.77	0.36	0.36
query32	3.37	0.48	0.48
query33	3.25	3.22	3.27
query34	15.86	4.17	4.16
query35	4.17	4.18	4.17
query36	1.11	1.05	1.04
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.01
query40	0.15	0.13	0.13
query41	0.07	0.01	0.01
query42	0.03	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 108.14 s
Total hot run time: 30.34 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2dc7f80ba43cd841d84708d9560ee725772a5f7f with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       14.2 seconds inserted 10000000 Rows, about 704K ops/s

@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.74% (8663/23580)
Line Coverage: 28.77% (70568/245268)
Region Coverage: 27.70% (36460/131628)
Branch Coverage: 24.40% (18648/76440)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2dc7f80ba43cd841d84708d9560ee725772a5f7f_2dc7f80ba43cd841d84708d9560ee725772a5f7f/report/index.html

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.43 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.2 seconds inserted 10000000 Rows, about 471K ops/s
storage size: 17183509933 Bytes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zhiqiang-hhhh @doris-robot