Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ut](datatype) Add BE UT for some basic data types and serializers #48788

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[ut](datatype) Add BE UT for some basic data types and serializers #48788

wants to merge 4 commits into from
+17,141 −378

Conversation

jacktengg
Copy link
Contributor

@jacktengg jacktengg commented Mar 6, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
DataTypeNumber, DataTypeDecimal, DataTypeString, DataTypeDateTimeV2

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@jacktengg jacktengg force-pushed the be-core-ut-datatypes branch from 6cb0295 to f8fe567 Compare March 6, 2025 17:03
@jacktengg
Copy link
Contributor Author

run buildall

@jacktengg jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from f25334b to b9b981b Compare March 6, 2025 17:06
@jacktengg
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32381 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9b981b162592571a03b6d3ef3921f35879ea833, data reload: false

------ Round 1 ----------------------------------
q1	17582	5217	5065	5065
q2	2050	315	166	166
q3	10445	1352	708	708
q4	10237	1040	542	542
q5	7638	2315	2408	2315
q6	189	169	138	138
q7	929	750	618	618
q8	9331	1389	1080	1080
q9	4955	4658	4630	4630
q10	6823	2305	1893	1893
q11	479	285	264	264
q12	356	355	217	217
q13	17780	3708	3031	3031
q14	232	233	208	208
q15	545	502	492	492
q16	615	611	582	582
q17	584	858	354	354
q18	6919	6555	6353	6353
q19	1900	961	546	546
q20	316	321	186	186
q21	2773	2190	2020	2020
q22	1072	1020	973	973
Total cold run time: 103750 ms
Total hot run time: 32381 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5201	5132	5113	5113
q2	239	328	231	231
q3	2134	2682	2292	2292
q4	1448	1845	1415	1415
q5	4276	4154	4140	4140
q6	209	164	122	122
q7	1908	1996	1754	1754
q8	2589	2707	2629	2629
q9	7261	7086	7267	7086
q10	2976	3151	2734	2734
q11	579	492	491	491
q12	689	806	634	634
q13	3541	3848	3252	3252
q14	277	288	284	284
q15	529	479	471	471
q16	628	686	645	645
q17	1139	1617	1339	1339
q18	7861	7721	7528	7528
q19	828	795	822	795
q20	1981	2027	1916	1916
q21	5494	4911	4811	4811
q22	1095	1013	1001	1001
Total cold run time: 52882 ms
Total hot run time: 50683 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184848 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9b981b162592571a03b6d3ef3921f35879ea833, data reload: false

query1	980	377	372	372
query2	6528	1977	1870	1870
query3	6797	234	220	220
query4	26236	23916	23148	23148
query5	4337	642	476	476
query6	306	191	189	189
query7	4598	508	291	291
query8	283	238	216	216
query9	8592	2571	2541	2541
query10	447	302	245	245
query11	15577	15056	14955	14955
query12	158	117	106	106
query13	1639	506	398	398
query14	9154	6082	6380	6082
query15	222	191	173	173
query16	7129	625	454	454
query17	923	713	565	565
query18	1942	409	310	310
query19	197	193	160	160
query20	122	114	115	114
query21	210	127	110	110
query22	4184	4157	4216	4157
query23	33763	32957	33178	32957
query24	7755	2410	2415	2410
query25	528	455	382	382
query26	1239	276	151	151
query27	2463	486	327	327
query28	4247	2413	2413	2413
query29	769	540	425	425
query30	286	216	188	188
query31	914	828	761	761
query32	72	65	64	64
query33	544	359	332	332
query34	785	846	502	502
query35	785	833	721	721
query36	939	971	869	869
query37	123	96	82	82
query38	4319	4094	4066	4066
query39	1467	1394	1407	1394
query40	206	115	105	105
query41	58	53	51	51
query42	122	103	102	102
query43	508	540	490	490
query44	1286	784	782	782
query45	175	171	163	163
query46	837	1016	638	638
query47	1755	1795	1705	1705
query48	368	399	295	295
query49	771	500	428	428
query50	682	736	398	398
query51	4179	4217	4172	4172
query52	106	99	98	98
query53	226	256	192	192
query54	480	485	443	443
query55	81	85	82	82
query56	257	268	244	244
query57	1115	1142	1071	1071
query58	246	243	237	237
query59	2669	2704	2566	2566
query60	295	264	249	249
query61	119	117	118	117
query62	799	761	699	699
query63	229	182	188	182
query64	4354	991	715	715
query65	4457	4359	4312	4312
query66	1141	409	312	312
query67	16012	15642	15243	15243
query68	8071	865	511	511
query69	457	296	267	267
query70	1195	1098	1060	1060
query71	394	290	262	262
query72	5617	3523	3607	3523
query73	711	683	345	345
query74	9198	9091	8942	8942
query75	3271	3203	2743	2743
query76	3273	1171	742	742
query77	546	368	277	277
query78	10063	10088	9346	9346
query79	1224	827	580	580
query80	606	518	458	458
query81	475	257	225	225
query82	195	126	95	95
query83	168	173	160	160
query84	287	94	71	71
query85	738	426	299	299
query86	371	321	296	296
query87	4486	4437	4289	4289
query88	2800	2182	2173	2173
query89	392	311	284	284
query90	1988	210	204	204
query91	139	148	110	110
query92	80	60	57	57
query93	1463	1058	584	584
query94	657	415	299	299
query95	356	259	250	250
query96	482	554	269	269
query97	3297	3394	3335	3335
query98	221	214	195	195
query99	1327	1406	1310	1310
Total cold run time: 269640 ms
Total hot run time: 184848 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9b981b162592571a03b6d3ef3921f35879ea833, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.63	0.10	0.11
query5	0.56	0.54	0.54
query6	1.20	0.73	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.61	0.53	0.52
query10	0.59	0.60	0.58
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.62	0.61	0.59
query14	2.66	2.71	2.67
query15	0.93	0.85	0.86
query16	0.38	0.38	0.38
query17	1.01	1.01	1.00
query18	0.21	0.21	0.19
query19	1.88	1.74	1.97
query20	0.02	0.01	0.02
query21	15.35	0.91	0.55
query22	0.77	1.33	0.78
query23	14.69	1.40	0.62
query24	6.96	1.70	1.06
query25	0.50	0.25	0.13
query26	0.56	0.16	0.13
query27	0.05	0.05	0.06
query28	9.61	0.84	0.45
query29	12.54	4.02	3.29
query30	0.24	0.09	0.07
query31	2.81	0.60	0.39
query32	3.23	0.56	0.46
query33	2.99	3.05	3.06
query34	15.82	5.17	4.50
query35	4.55	4.57	4.48
query36	0.67	0.49	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.02	0.03
Total cold run time: 104.84 s
Total hot run time: 31.06 s

@jacktengg jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from 5ec7ffc to 5365220 Compare March 7, 2025 02:19
@jacktengg
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32049 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5365220cc29e8d62c94e580ce33ce11e27fdd3a8, data reload: false

------ Round 1 ----------------------------------
q1	17636	5293	5053	5053
q2	2049	293	163	163
q3	10415	1255	739	739
q4	10200	1036	535	535
q5	7784	2403	2305	2305
q6	187	193	140	140
q7	902	789	637	637
q8	9329	1316	1034	1034
q9	4945	4508	4617	4508
q10	6803	2283	1877	1877
q11	475	275	253	253
q12	344	352	217	217
q13	17753	3704	3081	3081
q14	223	235	222	222
q15	523	488	478	478
q16	636	613	581	581
q17	583	832	363	363
q18	7128	6407	6225	6225
q19	1664	950	533	533
q20	309	327	193	193
q21	2859	2117	1926	1926
q22	1093	1017	986	986
Total cold run time: 103840 ms
Total hot run time: 32049 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5221	5135	5202	5135
q2	235	342	226	226
q3	2169	2671	2274	2274
q4	1434	1892	1382	1382
q5	4244	4094	4121	4094
q6	210	163	124	124
q7	1838	1850	1740	1740
q8	2645	2616	2502	2502
q9	7208	7257	7177	7177
q10	3005	3216	2763	2763
q11	567	511	472	472
q12	683	793	602	602
q13	3421	3897	3319	3319
q14	306	303	262	262
q15	528	476	477	476
q16	639	659	627	627
q17	1130	1621	1328	1328
q18	7689	7692	7426	7426
q19	838	772	877	772
q20	2061	2015	1876	1876
q21	5522	4927	4601	4601
q22	1087	1112	1049	1049
Total cold run time: 52680 ms
Total hot run time: 50227 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191794 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5365220cc29e8d62c94e580ce33ce11e27fdd3a8, data reload: false

query1	1391	989	980	980
query2	6196	1910	1924	1910
query3	11102	4557	4560	4557
query4	52740	25637	23129	23129
query5	5042	633	485	485
query6	337	210	191	191
query7	4874	504	288	288
query8	315	225	227	225
query9	5630	2552	2565	2552
query10	424	313	266	266
query11	15173	15049	14964	14964
query12	160	104	112	104
query13	1041	496	376	376
query14	10156	6296	6901	6296
query15	213	205	187	187
query16	7085	687	430	430
query17	1086	733	567	567
query18	1560	403	303	303
query19	200	187	152	152
query20	127	117	124	117
query21	202	129	122	122
query22	4311	4440	4443	4440
query23	34044	33258	33238	33238
query24	5643	2419	2437	2419
query25	504	478	419	419
query26	696	285	166	166
query27	1938	494	340	340
query28	2757	2471	2459	2459
query29	608	596	445	445
query30	277	224	192	192
query31	852	854	807	807
query32	78	67	64	64
query33	485	382	320	320
query34	767	856	512	512
query35	816	855	777	777
query36	954	995	903	903
query37	123	110	81	81
query38	4259	4309	4285	4285
query39	1485	1471	1447	1447
query40	207	114	98	98
query41	51	54	51	51
query42	123	110	101	101
query43	495	512	488	488
query44	1312	819	790	790
query45	180	173	173	173
query46	868	1037	642	642
query47	1840	1916	1763	1763
query48	376	414	303	303
query49	743	504	423	423
query50	716	822	422	422
query51	4377	4292	4310	4292
query52	107	111	95	95
query53	231	257	188	188
query54	485	512	416	416
query55	85	80	84	80
query56	255	259	259	259
query57	1145	1182	1134	1134
query58	242	253	242	242
query59	2662	2801	2805	2801
query60	295	275	262	262
query61	127	120	150	120
query62	759	733	683	683
query63	227	199	205	199
query64	1683	1031	775	775
query65	4565	4444	4496	4444
query66	760	403	284	284
query67	15741	15442	15290	15290
query68	6781	878	503	503
query69	530	294	262	262
query70	1229	1140	1111	1111
query71	503	298	268	268
query72	5578	3597	3769	3597
query73	1483	736	348	348
query74	9342	9108	8725	8725
query75	4046	3161	2710	2710
query76	4300	1182	748	748
query77	758	365	272	272
query78	10019	9995	9262	9262
query79	5837	811	576	576
query80	714	527	453	453
query81	495	254	221	221
query82	758	127	92	92
query83	218	176	153	153
query84	285	95	139	95
query85	762	346	301	301
query86	330	298	289	289
query87	4456	4388	4459	4388
query88	2836	2183	2232	2183
query89	460	321	278	278
query90	1986	203	204	203
query91	137	151	109	109
query92	78	63	56	56
query93	3063	1050	578	578
query94	676	411	381	381
query95	344	274	258	258
query96	497	552	278	278
query97	3347	3359	3273	3273
query98	218	209	204	204
query99	1436	1403	1286	1286
Total cold run time: 299810 ms
Total hot run time: 191794 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5365220cc29e8d62c94e580ce33ce11e27fdd3a8, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.55	0.54	0.56
query6	1.20	0.71	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.52
query10	0.57	0.62	0.59
query11	0.16	0.11	0.10
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.82	2.68	2.82
query15	0.92	0.86	0.86
query16	0.39	0.37	0.37
query17	1.05	1.04	1.04
query18	0.22	0.19	0.19
query19	1.97	1.80	1.92
query20	0.02	0.01	0.01
query21	15.35	0.91	0.57
query22	0.76	1.19	0.96
query23	14.72	1.37	0.60
query24	6.58	1.64	0.85
query25	0.51	0.24	0.12
query26	0.61	0.15	0.14
query27	0.05	0.06	0.05
query28	9.40	0.91	0.45
query29	12.54	4.07	3.33
query30	0.25	0.09	0.07
query31	2.83	0.57	0.38
query32	3.22	0.55	0.46
query33	3.06	3.01	2.98
query34	15.68	5.13	4.55
query35	4.56	4.60	4.54
query36	0.66	0.51	0.47
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 104.47 s
Total hot run time: 31.25 s

@jacktengg jacktengg force-pushed the be-core-ut-datatypes branch from 5365220 to 0a68e98 Compare March 7, 2025 07:09
@jacktengg jacktengg force-pushed the be-core-ut-datatypes branch from 0a68e98 to 92e0daa Compare March 7, 2025 07:09
@jacktengg
Copy link
Contributor Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jacktengg @hello-stephen @doris-robot