Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore](agent) log the binary message size of agent tasks #43239 #43367

Merged
merged 1 commit into from
Nov 6, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Nov 6, 2024

cherry pick from #43239

@w41ter
Copy link
Contributor Author

w41ter commented Nov 6, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

TPC-H: Total hot run time: 49393 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3133205aabe87aa053bacf7d4f93d8a8e989f71b, data reload: false

------ Round 1 ----------------------------------
q1	17553	4418	4368	4368
q2	2067	152	152	152
q3	10447	1900	1887	1887
q4	10309	1235	1335	1235
q5	8487	3907	3928	3907
q6	239	122	124	122
q7	2039	1644	1618	1618
q8	9551	2736	2708	2708
q9	13747	10784	10147	10147
q10	8696	3499	3491	3491
q11	411	253	254	253
q12	471	297	298	297
q13	18365	3993	4029	3993
q14	357	317	328	317
q15	512	458	453	453
q16	536	458	461	458
q17	1152	983	970	970
q18	7202	6725	6887	6725
q19	1661	1582	1525	1525
q20	544	309	305	305
q21	4412	4127	4073	4073
q22	489	402	389	389
Total cold run time: 119247 ms
Total hot run time: 49393 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4322	4357	4308	4308
q2	320	220	223	220
q3	4174	4151	4138	4138
q4	2741	2745	2748	2745
q5	7198	7113	7116	7113
q6	238	119	120	119
q7	3226	2872	2919	2872
q8	4382	4472	4516	4472
q9	14221	13713	13868	13713
q10	4216	4284	4275	4275
q11	772	702	679	679
q12	1022	859	859	859
q13	6609	3751	3692	3692
q14	458	440	420	420
q15	498	462	452	452
q16	632	611	595	595
q17	3803	3850	3827	3827
q18	8756	8692	8816	8692
q19	1734	1695	1642	1642
q20	2364	2147	2085	2085
q21	8489	8456	8501	8456
q22	1002	952	980	952
Total cold run time: 81177 ms
Total hot run time: 76326 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 214963 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3133205aabe87aa053bacf7d4f93d8a8e989f71b, data reload: false

query1	930	420	384	384
query2	6535	2107	2093	2093
query3	6919	201	208	201
query4	23345	21924	21579	21579
query5	19736	6594	6468	6468
query6	285	222	236	222
query7	4360	300	304	300
query8	245	257	263	257
query9	3058	2662	2576	2576
query10	473	318	301	301
query11	15449	15164	15112	15112
query12	120	73	79	73
query13	1028	457	443	443
query14	17781	13288	13664	13288
query15	367	217	239	217
query16	6477	284	262	262
query17	1907	962	941	941
query18	901	319	324	319
query19	218	153	149	149
query20	98	97	98	97
query21	192	100	97	97
query22	5172	4975	5032	4975
query23	34341	33368	33501	33368
query24	6911	6339	6344	6339
query25	533	428	442	428
query26	1124	164	165	164
query27	2283	292	289	289
query28	6128	2258	2208	2208
query29	2827	2816	2656	2656
query30	249	172	169	169
query31	944	734	765	734
query32	74	62	61	61
query33	439	265	277	265
query34	871	485	479	479
query35	1163	918	958	918
query36	1267	1208	1221	1208
query37	92	66	62	62
query38	3062	2938	2934	2934
query39	1394	1334	1305	1305
query40	307	100	99	99
query41	42	39	40	39
query42	89	95	95	95
query43	655	646	614	614
query44	1157	718	718	718
query45	250	230	230	230
query46	1244	949	986	949
query47	1836	1679	2112	1679
query48	499	418	413	413
query49	642	374	384	374
query50	873	654	600	600
query51	4845	4705	4643	4643
query52	88	90	85	85
query53	234	190	188	188
query54	2666	2476	2519	2476
query55	81	79	85	79
query56	221	225	218	218
query57	1231	1240	1071	1071
query58	224	214	225	214
query59	3411	3571	3261	3261
query60	217	220	215	215
query61	99	98	99	98
query62	877	595	489	489
query63	212	186	180	180
query64	3550	1606	1542	1542
query65	3635	3569	3546	3546
query66	774	420	422	420
query67	15243	15707	17544	15707
query68	9981	631	651	631
query69	511	297	270	270
query70	1660	1386	1376	1376
query71	411	311	318	311
query72	6896	5004	4981	4981
query73	755	321	321	321
query74	6328	5817	5824	5817
query75	5328	3738	3769	3738
query76	5691	1161	1154	1154
query77	973	262	274	262
query78	12803	13118	13421	13118
query79	11714	633	634	633
query80	758	384	412	384
query81	491	238	237	237
query82	753	105	98	98
query83	189	140	137	137
query84	264	72	69	69
query85	796	341	336	336
query86	338	310	296	296
query87	3252	3083	3078	3078
query88	5057	2299	2289	2289
query89	422	296	333	296
query90	2572	223	218	218
query91	161	128	127	127
query92	60	55	52	52
query93	5644	582	549	549
query94	1183	216	212	212
query95	2032	2151	2056	2056
query96	663	334	319	319
query97	6637	6348	6474	6348
query98	227	216	209	209
query99	2735	858	1021	858
Total cold run time: 321785 ms
Total hot run time: 214963 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3133205aabe87aa053bacf7d4f93d8a8e989f71b, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.02
query3	0.26	0.05	0.05
query4	1.78	0.08	0.07
query5	0.53	0.54	0.52
query6	1.22	0.62	0.63
query7	0.02	0.01	0.02
query8	0.03	0.02	0.02
query9	0.52	0.49	0.48
query10	0.54	0.54	0.52
query11	0.12	0.09	0.09
query12	0.12	0.10	0.09
query13	0.62	0.61	0.61
query14	0.78	0.79	0.77
query15	0.80	0.77	0.76
query16	0.37	0.39	0.34
query17	1.00	0.98	0.95
query18	0.23	0.24	0.26
query19	1.94	1.89	1.85
query20	0.01	0.00	0.01
query21	15.47	0.54	0.54
query22	1.93	1.61	1.61
query23	16.75	0.98	0.86
query24	5.51	1.25	1.12
query25	0.36	0.13	0.05
query26	0.64	0.17	0.16
query27	0.04	0.04	0.04
query28	7.22	0.72	0.72
query29	12.65	2.41	2.22
query30	0.58	0.54	0.53
query31	2.80	0.39	0.37
query32	3.36	0.50	0.50
query33	3.11	3.06	3.12
query34	15.24	4.81	4.83
query35	4.87	4.90	4.86
query36	1.08	1.03	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 102.99 s
Total hot run time: 30.75 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3133205aabe87aa053bacf7d4f93d8a8e989f71b with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.1 seconds inserted 10000000 Rows, about 452K ops/s

@w41ter w41ter merged commit f0a9fed into apache:branch-2.0 Nov 6, 2024
22 of 24 checks passed
@w41ter w41ter deleted the pick_43239_to_origin_branch-2.0 branch November 6, 2024 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants