-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IGNITE-19661 Fixed CDC data records logging to WAL for in-memory caches when disabled. #10766
base: master
Are you sure you want to change the base?
Conversation
# Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/management/cdc/CdcDeleteLostSegmentsTask.java
@@ -102,69 +115,138 @@ protected CdcDeleteLostSegmentsJob(CdcDeleteLostSegmentLinksCommandArg arg, bool | |||
try { | |||
lock.tryLock(1); | |||
|
|||
try (Stream<Path> cdcFiles = Files.list(walCdcDir.toPath())) { | |||
Set<File> delete = new HashSet<>(); | |||
Long lostSgmnt = findLastLostSegement(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lostSgmnt -> lastSegBeforeSkip
findLastLostSegement -> findLastSegmentBeforeSkip
Be aware about typo in segEment in method name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
|
||
AtomicLong lastSgmnt = new AtomicLong(-1); | ||
if (lostSgmnt != null) | ||
deleteAll(lostSgmnt); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
deleteAll -> deleteAllUntil or
deleteAllPrevious(lostSgmnt + 1);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
|
||
AtomicLong lastSgmnt = new AtomicLong(-1); | ||
if (lostSgmnt != null) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (seg != null) -> this check can be moved to the method.
It seem, that will simplify the code.
.reversed()) // Sort by segment index. | ||
.forEach(path -> { | ||
long idx = FileWriteAheadLogManager.segmentIndex(path); | ||
Long cdcDisableSgmnt = findLastCdcDisableSegment(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
findLastCdcDisableSegment -> findLastSegmentWithCdcDisabledRecord.
cdcDisableSgmnt -> lastSegWithDisabledRec
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
} | ||
if (lostSgmnt != null || cdcDisableSgmnt != null) | ||
resetWalState(); | ||
else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
brace can be omitted
else if (log.isDebugEnabled)
log.debug("Here!");
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
} | ||
|
||
/** */ | ||
private void resetWalState() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
deleteCdcWalState
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
@@ -515,8 +521,18 @@ public void setFileIOFactory(FileIOFactory ioFactory) { | |||
name, oldVal, newVal)); | |||
} | |||
|
|||
if (newVal != null && newVal) | |||
if (newVal != null && newVal) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few linew below you check both values - oldVal && !newVal
.
Here you check only newVal
.
Does it mean that disable record can be recorded twice? When oldVal == newVal
So, it seems you should keep logic of both check consistent.
if (!inMemoryCdc || rec instanceof CdcDisableRecord || !cdcDisabled.getOrDefault(false)) | ||
return false; | ||
|
||
LT.warn(log, "Logging CDC data records to WAL skipped. The '" + CDC_DISABLED + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LT.warn effectively creates an object to check timeout of logging on every invocation
IgniteBiTuple<Class<? extends Throwable>, String> tup =
e != null && !byMsg ? F.<Class<? extends Throwable>, String>t(e.getClass(), e.getMessage()) :
F.<Class<? extends Throwable>, String>t(null, longMsg);
So we shouldn't call it for every DataRecord that will be logged on every entry change.
Let's log changes of cdcDisabled state only.
SonarCloud Quality Gate failed. 1 Bug 0.0% Coverage The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Catch issues before they fail your Quality Gate with our IDE extension SonarLint |
…led.
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
The Contribution Checklist
The description explains WHAT and WHY was made instead of HOW.
The following pattern must be used:
IGNITE-XXXX Change summary
whereXXXX
- number of JIRA issue.(see the Maintainers list)
the
green visa
attached to the JIRA ticket (see TC.Bot: Check PR)Notes
If you need any help, please email [email protected] or ask anу advice on http://asf.slack.com #ignite channel.