Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [enhance](mtmv)MTMV no longer generate cache when replaying logs #44283 #44307

Merged
merged 1 commit into from
Nov 26, 2024
Merged

branch-3.0: [enhance](mtmv)MTMV no longer generate cache when replaying logs #44283 #44307

merged 1 commit into from
Nov 26, 2024

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #44283

@zddr
[enhance](mtmv)MTMV no longer generate cache when replaying logs (#44283
55a1534 
)

### What problem does this PR solve?

When replaying logs, cache will no longer be generated because the
catalog may not have been initialized or the dependent environment may
not be connected, causing it to freeze here and preventing FE from
starting

The cost is that after the materialized view is refreshed, the cache of
the follower node will be empty, and a cache will be generated when the
query is first used

Problem Summary:
MTMV no longer generate cache when replaying logs
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Nov 20, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40162 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55a1534e2ef08874195970f70fd13ecd4a7282ff, data reload: false

------ Round 1 ----------------------------------
q1	17567	7341	7251	7251
q2	2039	177	157	157
q3	10831	1037	1108	1037
q4	10536	744	683	683
q5	7741	2802	2753	2753
q6	233	144	145	144
q7	948	607	608	607
q8	9479	1901	1957	1901
q9	6544	6364	6370	6364
q10	6951	2310	2298	2298
q11	453	255	253	253
q12	406	211	202	202
q13	17787	2921	2981	2921
q14	248	206	211	206
q15	558	516	537	516
q16	664	593	591	591
q17	946	555	604	555
q18	7240	6618	6564	6564
q19	2912	1047	1090	1047
q20	475	193	189	189
q21	3969	3272	2945	2945
q22	1056	978	981	978
Total cold run time: 109583 ms
Total hot run time: 40162 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7329	7174	7248	7174
q2	313	229	230	229
q3	2872	2879	2827	2827
q4	1978	1759	1811	1759
q5	5596	5645	5647	5645
q6	214	138	139	138
q7	2190	1757	1742	1742
q8	3284	3510	3445	3445
q9	8652	8891	8792	8792
q10	3519	3505	3483	3483
q11	614	496	494	494
q12	814	589	622	589
q13	16543	3136	3110	3110
q14	297	267	268	267
q15	570	519	508	508
q16	719	662	665	662
q17	1842	1638	1557	1557
q18	8114	7714	7490	7490
q19	1974	1594	1506	1506
q20	2078	1881	1812	1812
q21	5438	5299	5142	5142
q22	1099	1034	987	987
Total cold run time: 76049 ms
Total hot run time: 59358 ms

@dataroaring dataroaring merged commit 1b06f02 into branch-3.0 Nov 26, 2024
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-44283-branch-3.0 branch November 26, 2024 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@doris-robot @dataroaring @zddr