Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids)Add catalog/db/table filter info in SchemaScanNode #46864

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Jibing-Li
Copy link
Contributor

What problem does this PR solve?

While querying information_schema.tables, we need to filter the catalogs/dbs/tables if user specify them thought where statement. Before, we didn't pass the query predicate to BE. This pr pass the predicate to BE and filter the catalog/db/table in FrontendServiceImpl.listTableStatus

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Jibing-Li Jibing-Li marked this pull request as ready for review January 13, 2025 03:23
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32658 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fe02861c74d0aa83e9c0d954895af1e0ed75da8a, data reload: false

------ Round 1 ----------------------------------
q1	17627	6220	6075	6075
q2	2047	319	171	171
q3	10579	1337	725	725
q4	10288	895	431	431
q5	8734	2175	1961	1961
q6	204	183	150	150
q7	905	743	636	636
q8	9243	1358	1244	1244
q9	6034	5030	4881	4881
q10	6798	2301	1830	1830
q11	488	281	260	260
q12	344	362	220	220
q13	17774	3634	3086	3086
q14	240	228	218	218
q15	576	513	498	498
q16	639	628	599	599
q17	582	860	329	329
q18	7017	6480	6315	6315
q19	1547	965	539	539
q20	310	321	192	192
q21	2811	2192	1987	1987
q22	375	331	311	311
Total cold run time: 105162 ms
Total hot run time: 32658 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6267	6212	6225	6212
q2	248	335	242	242
q3	2292	2657	2353	2353
q4	1392	1789	1365	1365
q5	4363	4768	5018	4768
q6	182	177	138	138
q7	2101	1979	1856	1856
q8	2644	2778	2682	2682
q9	7356	7316	7185	7185
q10	3063	3361	2809	2809
q11	601	540	491	491
q12	675	750	609	609
q13	3467	3835	3232	3232
q14	290	322	291	291
q15	564	515	514	514
q16	654	686	645	645
q17	1226	1740	1264	1264
q18	7796	7480	7369	7369
q19	810	1005	1181	1005
q20	1950	2089	1970	1970
q21	5719	5136	4991	4991
q22	617	640	575	575
Total cold run time: 54277 ms
Total hot run time: 52566 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188654 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fe02861c74d0aa83e9c0d954895af1e0ed75da8a, data reload: false

query1	989	387	378	378
query2	6514	2407	2342	2342
query3	6712	215	216	215
query4	33568	23476	23595	23476
query5	4385	634	491	491
query6	299	207	174	174
query7	4633	502	311	311
query8	295	237	224	224
query9	9602	2684	2670	2670
query10	462	315	246	246
query11	17820	15208	15085	15085
query12	157	105	104	104
query13	1628	510	384	384
query14	9688	7367	7076	7076
query15	242	192	187	187
query16	8266	589	448	448
query17	1594	781	550	550
query18	2101	395	299	299
query19	215	177	149	149
query20	114	108	112	108
query21	209	120	99	99
query22	4392	4408	4240	4240
query23	34740	33095	33086	33086
query24	6708	2206	2229	2206
query25	508	461	369	369
query26	1197	279	154	154
query27	2065	459	329	329
query28	5163	2417	2395	2395
query29	721	546	408	408
query30	233	185	156	156
query31	976	859	767	767
query32	82	59	62	59
query33	513	350	295	295
query34	763	845	519	519
query35	830	798	723	723
query36	1013	1059	931	931
query37	126	99	79	79
query38	4165	4002	3976	3976
query39	1486	1441	1406	1406
query40	208	123	107	107
query41	59	59	65	59
query42	128	106	155	106
query43	514	539	505	505
query44	1358	814	808	808
query45	176	169	165	165
query46	851	1047	646	646
query47	1816	1807	1767	1767
query48	391	400	308	308
query49	776	496	412	412
query50	632	699	387	387
query51	6989	7018	6760	6760
query52	104	106	96	96
query53	222	254	194	194
query54	478	494	412	412
query55	83	77	78	77
query56	259	283	241	241
query57	1184	1149	1100	1100
query58	246	227	246	227
query59	3048	3102	3011	3011
query60	279	256	250	250
query61	121	116	130	116
query62	807	768	688	688
query63	221	194	193	193
query64	4452	1061	656	656
query65	3261	3152	3194	3152
query66	863	424	329	329
query67	15862	15750	15465	15465
query68	7782	701	515	515
query69	466	303	271	271
query70	1187	1159	1099	1099
query71	401	281	266	266
query72	6237	3853	3909	3853
query73	640	749	348	348
query74	10374	9074	8933	8933
query75	3269	3146	2658	2658
query76	3273	1168	784	784
query77	538	373	275	275
query78	9908	9838	9305	9305
query79	2994	795	609	609
query80	644	508	447	447
query81	484	279	238	238
query82	609	142	126	126
query83	173	171	158	158
query84	241	98	73	73
query85	784	369	295	295
query86	359	311	300	300
query87	4367	4400	4262	4262
query88	4774	2143	2120	2120
query89	458	327	292	292
query90	1823	193	190	190
query91	135	136	130	130
query92	71	58	53	53
query93	1687	871	541	541
query94	695	406	295	295
query95	330	272	255	255
query96	489	625	281	281
query97	2899	2829	2764	2764
query98	229	203	194	194
query99	1476	1555	1373	1373
Total cold run time: 290358 ms
Total hot run time: 188654 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.85 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fe02861c74d0aa83e9c0d954895af1e0ed75da8a, data reload: false

query1	0.03	0.04	0.04
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.11	0.10
query5	0.43	0.42	0.40
query6	1.13	0.66	0.65
query7	0.03	0.01	0.01
query8	0.04	0.04	0.04
query9	0.58	0.50	0.49
query10	0.56	0.56	0.57
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.79	2.72	2.83
query15	0.89	0.82	0.83
query16	0.40	0.39	0.37
query17	1.05	1.06	1.04
query18	0.22	0.22	0.21
query19	2.00	1.83	2.01
query20	0.01	0.01	0.02
query21	15.36	0.92	0.58
query22	0.75	0.80	0.64
query23	15.28	1.42	0.55
query24	3.03	1.52	1.62
query25	0.14	0.10	0.22
query26	0.29	0.13	0.14
query27	0.04	0.07	0.05
query28	14.45	1.45	1.04
query29	12.60	3.99	3.33
query30	0.25	0.09	0.06
query31	2.81	0.61	0.38
query32	3.22	0.55	0.45
query33	3.08	3.18	3.11
query34	16.79	5.05	4.49
query35	4.49	4.47	4.44
query36	0.64	0.48	0.47
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.12	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 106.67 s
Total hot run time: 31.85 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 40.35% (10514/26054)
Line Coverage: 31.10% (89086/286448)
Region Coverage: 30.22% (45562/150755)
Branch Coverage: 26.51% (23153/87344)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fe02861c74d0aa83e9c0d954895af1e0ed75da8a_fe02861c74d0aa83e9c0d954895af1e0ed75da8a/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants