Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](meta-mgr) Real-time parsing meta service endpoint to avoid rpc failed after config muted #45877

Merged
merged 1 commit into from
Dec 25, 2024

Conversation

TangSiyang2001
Copy link
Collaborator

What problem does this PR solve?

Problem Summary:

Release note

Original config::meta_service_endpoint is immutable. But it is changed to mutable now, so we should parse it in realtime to avoid rpc failure.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 24, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.80% (10095/26017)
Line Coverage: 29.79% (85163/285894)
Region Coverage: 28.91% (43489/150425)
Branch Coverage: 25.45% (22174/87134)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2af74679d0bf1bce54d74890d3fd6ea615d2ebcd_2af74679d0bf1bce54d74890d3fd6ea615d2ebcd/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32364 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2af74679d0bf1bce54d74890d3fd6ea615d2ebcd, data reload: false

------ Round 1 ----------------------------------
q1	17587	6149	6003	6003
q2	2048	292	162	162
q3	10433	1206	749	749
q4	10258	854	428	428
q5	8548	2160	1975	1975
q6	211	181	145	145
q7	888	753	610	610
q8	9239	1358	1170	1170
q9	5202	4889	4972	4889
q10	6829	2368	1938	1938
q11	474	275	262	262
q12	353	372	224	224
q13	17772	3564	2895	2895
q14	229	231	203	203
q15	569	508	500	500
q16	644	619	606	606
q17	578	845	321	321
q18	6834	6311	6332	6311
q19	2499	964	539	539
q20	295	314	182	182
q21	2747	2139	1950	1950
q22	361	332	302	302
Total cold run time: 104598 ms
Total hot run time: 32364 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6324	6165	6234	6165
q2	247	326	229	229
q3	2225	2629	2302	2302
q4	1396	1804	1348	1348
q5	4329	4710	4756	4710
q6	180	179	146	146
q7	2055	1992	1806	1806
q8	2626	2813	2670	2670
q9	7314	7318	7297	7297
q10	3082	3376	2770	2770
q11	583	501	485	485
q12	654	809	586	586
q13	3317	3754	3137	3137
q14	291	302	298	298
q15	582	517	505	505
q16	677	680	651	651
q17	1231	1739	1278	1278
q18	7536	7525	7311	7311
q19	857	1134	1083	1083
q20	1984	2008	1923	1923
q21	5734	5325	4914	4914
q22	625	625	571	571
Total cold run time: 53849 ms
Total hot run time: 52185 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197255 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2af74679d0bf1bce54d74890d3fd6ea615d2ebcd, data reload: false

query1	1311	931	927	927
query2	6484	2330	2299	2299
query3	11098	4749	4984	4749
query4	32460	23817	23392	23392
query5	3997	615	446	446
query6	284	185	175	175
query7	3973	482	299	299
query8	293	235	257	235
query9	9512	2740	2720	2720
query10	449	297	239	239
query11	18029	15639	15173	15173
query12	154	106	106	106
query13	1568	529	402	402
query14	10489	7627	7476	7476
query15	237	201	185	185
query16	7848	602	464	464
query17	1571	795	606	606
query18	2062	417	344	344
query19	208	197	159	159
query20	119	120	116	116
query21	207	127	124	124
query22	4848	4662	4588	4588
query23	34636	33303	33488	33303
query24	6582	2412	2337	2337
query25	512	484	484	484
query26	921	279	157	157
query27	2111	474	336	336
query28	5478	2477	2492	2477
query29	589	556	410	410
query30	209	192	146	146
query31	975	928	837	837
query32	71	58	57	57
query33	487	347	302	302
query34	760	857	509	509
query35	824	824	733	733
query36	1058	1097	973	973
query37	112	97	76	76
query38	4258	4380	4249	4249
query39	1505	1465	1458	1458
query40	207	113	97	97
query41	44	42	45	42
query42	120	105	108	105
query43	521	516	495	495
query44	1353	824	843	824
query45	175	173	167	167
query46	887	1068	655	655
query47	1999	2024	1963	1963
query48	377	434	353	353
query49	704	469	381	381
query50	642	672	387	387
query51	7210	7272	7144	7144
query52	100	99	99	99
query53	225	246	184	184
query54	488	484	396	396
query55	76	87	85	85
query56	247	275	227	227
query57	1265	1237	1174	1174
query58	224	221	236	221
query59	3341	3286	3215	3215
query60	259	257	250	250
query61	108	105	105	105
query62	843	806	777	777
query63	228	198	186	186
query64	3238	1010	655	655
query65	3364	3227	3242	3227
query66	889	412	303	303
query67	16553	15783	15509	15509
query68	9573	759	513	513
query69	491	308	247	247
query70	1206	1133	1146	1133
query71	428	291	251	251
query72	5870	3853	3896	3853
query73	719	754	362	362
query74	9904	9439	9034	9034
query75	4535	3154	2613	2613
query76	5570	1189	782	782
query77	1008	358	269	269
query78	11093	11289	9879	9879
query79	2689	862	612	612
query80	774	511	438	438
query81	489	272	235	235
query82	414	147	130	130
query83	187	160	152	152
query84	274	90	73	73
query85	726	346	301	301
query86	346	300	291	291
query87	4580	4482	4292	4292
query88	4243	2192	2182	2182
query89	403	325	368	325
query90	1872	184	180	180
query91	125	127	102	102
query92	65	55	48	48
query93	908	859	536	536
query94	646	382	304	304
query95	326	257	246	246
query96	481	617	272	272
query97	2830	2835	2666	2666
query98	217	201	200	200
query99	1706	1553	1440	1440
Total cold run time: 298674 ms
Total hot run time: 197255 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.41 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2af74679d0bf1bce54d74890d3fd6ea615d2ebcd, data reload: false

query1	0.03	0.03	0.05
query2	0.08	0.04	0.04
query3	0.23	0.07	0.06
query4	1.62	0.10	0.11
query5	0.42	0.42	0.40
query6	1.15	0.65	0.65
query7	0.03	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.50	0.53
query10	0.54	0.57	0.57
query11	0.14	0.10	0.10
query12	0.14	0.10	0.10
query13	0.60	0.61	0.59
query14	2.83	2.74	2.85
query15	0.91	0.83	0.82
query16	0.39	0.39	0.40
query17	1.07	1.05	0.96
query18	0.23	0.21	0.22
query19	1.96	1.82	1.99
query20	0.02	0.01	0.01
query21	15.36	0.92	0.57
query22	0.76	0.81	0.67
query23	15.29	1.46	0.57
query24	3.33	1.48	0.93
query25	0.22	0.12	0.17
query26	0.37	0.14	0.13
query27	0.04	0.04	0.03
query28	13.52	1.50	1.04
query29	12.60	4.08	3.41
query30	0.26	0.09	0.06
query31	2.82	0.59	0.38
query32	3.23	0.56	0.47
query33	3.11	3.08	3.03
query34	17.21	5.13	4.54
query35	4.52	4.52	4.54
query36	0.65	0.49	0.50
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.81 s
Total hot run time: 31.41 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 25, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit cab074f into apache:master Dec 25, 2024
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 25, 2024
…failed after config muted (#45877)


Original config::meta_service_endpoint is immutable. But it is changed
to mutable now, so we should parse it in realtime to avoid rpc failure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.x reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants