Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "[fix](move-memtable) only check missing tablets when commit i… #29352

Merged
merged 2 commits into from
Dec 31, 2023

Conversation

dataroaring
Copy link
Contributor

…nfo is not empty (#29326)"

This reverts commit 4634c72.

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
…nfo is not empty (apache#29326)"

This reverts commit 4634c72.
@dataroaring
Copy link
Contributor Author

run buildall

@dataroaring
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 738ad558696faaa46bcf1557a619bf3e776ab835, data reload: false

------ Round 1 ----------------------------------
q1	17668	5181	5133	5133
q2	2025	160	150	150
q3	10532	1067	1125	1067
q4	10317	785	806	785
q5	7790	2911	2980	2911
q6	211	129	132	129
q7	910	541	507	507
q8	9290	1999	2013	1999
q9	6855	6415	6416	6415
q10	8250	3053	3054	3053
q11	422	221	226	221
q12	397	235	230	230
q13	17997	3664	3654	3654
q14	251	213	211	211
q15	579	548	552	548
q16	462	394	392	392
q17	958	458	508	458
q18	7279	6648	6643	6643
q19	1591	1447	1376	1376
q20	669	327	330	327
q21	2805	2360	2413	2360
q22	363	332	321	321
Total cold run time: 107621 ms
Total hot run time: 38890 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5135	5150	5088	5088
q2	352	230	247	230
q3	3346	3278	3250	3250
q4	2098	2020	2059	2020
q5	5796	5769	5777	5769
q6	214	124	120	120
q7	2324	1923	1869	1869
q8	3417	3453	3480	3453
q9	8809	8750	8693	8693
q10	3802	3838	3832	3832
q11	600	475	514	475
q12	806	646	644	644
q13	6451	3204	3262	3204
q14	303	273	269	269
q15	603	553	547	547
q16	560	500	479	479
q17	1932	1742	1757	1742
q18	8641	8406	8252	8252
q19	1644	1598	1623	1598
q20	2196	1962	1976	1962
q21	5578	5354	5459	5354
q22	525	509	490	490
Total cold run time: 65132 ms
Total hot run time: 59340 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.62% (8614/23520)
Line Coverage: 28.69% (70022/244072)
Region Coverage: 27.68% (36244/130954)
Branch Coverage: 24.38% (18518/75968)
Coverage Report: http://coverage.selectdb-in.cc/coverage/738ad558696faaa46bcf1557a619bf3e776ab835_738ad558696faaa46bcf1557a619bf3e776ab835/report/index.html

@doris-robot
Copy link

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit 738ad558696faaa46bcf1557a619bf3e776ab835, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	930	361	347	347
query2	6415	1932	1983	1932
query3	6639	212	203	203
query4	28087	22550	22499	22499
query5	5838	534	520	520
query6	272	194	186	186
query7	4592	270	267	267
query8	232	208	201	201
query9	8212	2551	2637	2551
query10	415	247	243	243
query11	16246	15590	15652	15590
query12	140	83	80	80
query13	1635	332	338	332
query14	11642	7228	7358	7228
query15	244	199	196	196
query16	6383	287	272	272
query17	1830	519	506	506
query18	1918	279	266	266
query19	288	141	141	141
query20	84	82	84	82
query21	185	98	95	95
query22	4885	4756	4761	4756
query23	32162	31260	31290	31260
query24	11825	2840	2835	2835
query25	595	357	354	354
query26	1701	148	148	148
query27	2854	281	288	281
query28	6839	1965	1950	1950
query29	1706	416	386	386
query30	294	152	149	149
query31	963	776	785	776
query32	84	63	60	60
query33	723	283	286	283
query34	820	459	441	441
query35	882	775	764	764
query36	1283	1303	1220	1220
query37	108	81	80	80
query38	3393	3327	3277	3277
query39	1340	1297	1283	1283
query40	306	99	97	97
query41	39	35	35	35
query42	104	86	91	86
query43	556	493	500	493
query44	1080	707	724	707
query45	196	196	185	185
query46	1077	644	623	623
query47	1716	1611	1640	1611
query48	344	262	258	258
query49	1168	338	335	335
query50	742	334	339	334
query51	5379	5310	5377	5310
query52	93	88	95	88
query53	214	161	158	158
query54	1356	575	588	575
query55	103	92	98	92
query56	216	204	203	203
query57	1039	986	949	949
query58	229	213	221	213
query59	2823	2670	2648	2648
query60	255	240	249	240
query61	85	86	84	84
query62	638	480	475	475
query63	176	149	157	149
query64	5809	1745	1700	1700
query65	3356	3310	3291	3291
query66	1136	339	339	339
query67	15805	15365	15230	15230
query68	12226	534	542	534
query69	548	262	280	262
query70	1640	1562	1488	1488
query71	509	245	238	238
query72	5674	3594	3568	3568
query73	2701	349	326	326
query74	6986	6555	6513	6513
query75	4992	2283	2243	2243
query76	6298	1156	1188	1156
query77	670	307	273	273
query78	9117	8751	8553	8553
query79	1048	521	507	507
query80	562	381	381	381
query81	463	215	208	208
query82	212	111	103	103
query83	163	142	139	139
query84	253	57	54	54
query85	934	303	286	286
query86	391	368	382	368
query87	3562	3386	3346	3346
query88	2947	2259	2263	2259
query89	342	277	278	277
query90	1817	215	221	215
query91	122	96	92	92
query92	62	58	58	58
query93	1382	519	449	449
query94	777	193	196	193
query95	474	435	419	419
query96	617	316	318	316
query97	4296	4157	4165	4157
query98	219	199	195	195
query99	1110	833	881	833
Total cold run time: 293838 ms
Total hot run time: 180557 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.24 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.3 seconds inserted 10000000 Rows, about 353K ops/s
storage size: 17184147418 Bytes

Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 31, 2023
@dataroaring dataroaring merged commit b07ce17 into apache:master Dec 31, 2023
25 of 28 checks passed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
apache#29352)

* Revert "[fix](move-memtable) only check missing tablets when commit info is not empty (apache#29326)"

This reverts commit 4634c72.

* Revert "[fix](move-memtable) check missing tablets before commit (apache#29223)"

This reverts commit fb0ed8c.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants