Add target search concurrency to TieredMergePolicy #13430

carlosdelest · 2024-05-27T18:38:30Z

Adds a minimum number of segments to TieredMergePolicy. This allows to ensure that a minimum search concurrency parallelism is achievable using inter-segment parallelism.

jpountz

Thanks for looking into it!

Currently, it seems to support this new option the same way that it handles the max merged segment size, ie. by greedily merging segments together if the merged segment can be just as large as the target max merged segment size?

Intuitively, this doesn't feel right to me as this boils down to merging more aggressively when minNumSegments is configured, while I'd expect the opposite: less aggressive merging in order to preserve search concurrency. I'm curious to get thoughts on a different approach that would consist of:

Update the calculation of allowedSegCount to take this new option into account, by allowing at least minNumSegments if there is a single tier, or minNumSegments-1 segments on the highest tier otherwise (since the previous tier would have as many docs as a segment on the highest tier).
Update doFindMerge to simply ignore candidate merges that have more than allowedDocCount docs.

I'm curious to get @mikemccand 's thoughts.

Separately, I don't think the minNumSegments is good. This suggests that the merge policy will ensure the index has at least this number of segments, which is not what it is doing. Maybe call in targetSearchConcurrenty or targetNumSlices instead to better reflect the connection with search?

carlosdelest · 2024-05-29T10:46:28Z

Hi @jpountz , thanks for your thoughts!

Let me check whether I got your suggestion right:

There are at most 2 segment tiers. The first has minNumSegments at most, the second minNumSegments-1 to ensure we maintain minNumSegments total segments at the most.
The max number of docs allowed in a segment is (maxDocs - numDeletes) / minNumSegments. That avoids creating larger segments than necessary so we ensure there's minNumSegments segments at a minimum.

Does it align with your thoughts?

Some questions I have:

segsPerTier is not taken into account if minNumSegments is specified, correct?
In case segments reach maxMergedSegmentBytes, should we fall back to the previous segsPerTier behaviour?

I see that minNumSegments is not a good name with this approach - I think targetSearchConcurrency makes sense to be explicit.

Thanks!

jpountz · 2024-05-29T12:20:16Z

There are at most 2 segment tiers.

Well, there can be more tiers, but since tiers have exponential sizes (e.g. if you merge factor is 10, each tier has segments that are 10x bigger than the previous tier) it's almost certainly fine to ignore segments on tiers beyond the 2nd higher tier, they would account to very few documents compared with the first 2 tiers.

segsPerTier is not taken into account if minNumSegments is specified, correct?

If minNumSegments is bigger than segsPerTier, then the highest tier should indeed allow for up to minNumSegments (or maybe minNumSegments-1 if there are multiple tiers, though it may introduce complexity for little value, maybe it's better/easier to keep things simple and allow for minNumSegments on the highest tier all the time). But lower tiers should still aim for segsPerTier segments.

In case segments reach maxMergedSegmentBytes, should we fall back to the previous segsPerTier behaviour?

I can understood this question in different ways, so I'll try to clarify how I think these two parameters should interact:

Even if minNumSegments is configured, we should still honor maxMergedSegmentBytes and not create segments that are bigger than that.
If your merge policy has segsPerTier = 10, maxMergedSegmentMB = 5GB, floorSegmentBytes = 100MB, minNumSegments = 12, and the total index size is 15GB, it should aim for allowedSegCount=23 segments (10 100MB segments ~=1GB, 14 1GB segments). So even though we could have created a segment of the maximum merged size by merging 10 1GB segments, we did not do it because then we would have ended up with a segment that had more than maxDoc/12 documents (assuming all docs contribute equally to the index size).

mikemccand · 2024-05-29T13:03:25Z

Thank you for tackling this @carlosdelest! What a hairy challenge ... TMP really is its own little baby chess engine, with many things it is trying to optimize towards (what the ML world now calls "multi-objective"), playing against a worthy opponent (the indexing application that keeps indexing different sized docs, doing some deletions/updates, refresh/commit, etc., producing all kinds of exotic segments). And of course in addition to the app exotically birthing new segments, TMP itself is causing new big segments to appear when each merge completes! This really is a game theory problem, heh.

+1 for the name targetSearchConcurrency -- it makes it clear this is about optimizing the "inter-segment concurrent search latency" in the resulting index, and, I think it also makes it clear that maxMergedSegmentBytes still wins. I think simply aiming for the largest tier to allow targetSearchConcurrency segment count when that exceeds segsPerTier is a good approach. After all it is those biggest segments that you really need for concurrency since searching them takes the most CPU / latency -- the lower tiers (1/10th, 1/100th, ... the size of the top tier) will be super fast by comparison and not the long pole in each query's latency.

…t tier

carlosdelest · 2024-05-29T19:10:55Z

Thanks @mikemccand and @jpountz for your explanations! This is much clearer now.

I've updated the PR, LMKWYT about the approach!

I'm looking into effective ways of testing in terms of ensuring that the last tier contains the target number of segments for search concurrency. Any suggestions on how can I check that the last tier eventually contains the expected number of segments?

jpountz · 2024-05-29T20:48:19Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

@@ -93,6 +93,7 @@ public class TieredMergePolicy extends MergePolicy {
  private double segsPerTier = 10.0;
  private double forceMergeDeletesPctAllowed = 10.0;
  private double deletesPctAllowed = 20.0;
+  private int targetSearchConcurrency = -1;


Should it default to 1? I expect it to yield the same behavior as today?

Yes - I iterated a bit on this but now it's equivalent. Thanks!

jpountz · 2024-05-29T20:53:17Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

@@ -257,6 +258,25 @@ public double getSegmentsPerTier() {
    return segsPerTier;
  }

+  /**
+   * Sets the target search concurrency. This allows merging to ensure that there are at least
+   * targetSearchConcurrency segments on the top tier. This setting can be overriden by force


Let's rather say that this prevents creating segments that are bigger than maxDoc/targetSearchConcurrency, which in-turn makes the work parallelizable into targetSearchConcurrency slices of similar doc counts?

I think it's also worth clarifying that this makes merging less aggressive, ie. high values of targetSearchConcurrency result in indexes that do less merging and have more segments.

Makes sense, updating.

jpountz · 2024-05-29T21:01:09Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

@@ -409,6 +433,8 @@ public MergeSpecification findMerges(
    // allowedSegCount may occasionally be less than segsPerTier
    // if segment sizes are below the floor size
    allowedSegCount = Math.max(allowedSegCount, segsPerTier);
+    // Override with the targetSearchConcurrency if it is greater
+    allowedSegCount = Math.max(allowedSegCount, targetSearchConcurrency);


I would also do a change like this one a few lines above:

diff --git a/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java b/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java index 208aa297287..4126601e51e 100644 --- a/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java +++ b/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java @@ -392,13 +392,14 @@ public class TieredMergePolicy extends MergePolicy { allowedDelCount = Math.max(0, allowedDelCount); final int mergeFactor = (int) Math.min(maxMergeAtOnce, segsPerTier); + final int maxNumSegmentsOnHighestTier = Math.max(segsPerTier, targetSearchConcurrency); // Compute max allowed segments in the index long levelSize = Math.max(minSegmentBytes, floorSegmentBytes); long bytesLeft = totIndexBytes; double allowedSegCount = 0; while (true) { final double segCountLevel = bytesLeft / (double) levelSize; - if (segCountLevel < segsPerTier || levelSize == maxMergedSegmentBytes) { + if (segCountLevel <= maxNumSegmentsOnHighestTier || levelSize == maxMergedSegmentBytes) { allowedSegCount += Math.ceil(segCountLevel); break; }

Correct, now the highest tier needs to take that into account. Thanks! 👍

jpountz · 2024-05-29T21:10:56Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

          break;
        }

-        final MergeScore score = score(candidate, hitTooLarge, segInfosSizes);
+        final MergeScore score = score(candidate, hitTooLarge || hitMaxDocs, segInfosSizes);


I don't think that we want to do this. hitTooLarge is considered a good thing by this merge policy. It means that we found a merge that reached the maximum merged segment size. It's great because if we perform this merge, then this segment will not be eligible again for merging (unless it gets many deletes), so we'll essentially be done with it. So these merges are given a very good score by assuming that they have a perfect skew.

hitMaxDocs is different, we don't want to prioritize unbalanced merges that produce segments of hitMaxDocs documents? It's better to keep prioritizing balanced small segment merges as usual?

My expectation for handling allowedMaxDoc is that we would just never score any merge that has more than allowedMaxDoc documents and force the merge policy to select one of the candidate merges that produce fewer documents than that.

I see - I was thinking in doing the same with docs than with bytes, but I see they're quite different. Docs is more a hint to maintain the number of segments we want, but bytes provides a hard limit on size and also score guidance on segments to merge.

jpountz · 2024-05-29T21:17:45Z

lucene/core/src/test/org/apache/lucene/index/TestTieredMergePolicy.java

@@ -431,6 +447,128 @@ public void testForcedMergesRespectSegSize() throws Exception {
    dir.close();
  }

+  public void testForcedMergesRespectsTargetSearchConcurrency() throws Exception {


You may want to check out BaseMergePolicyTestCase#testSimulateAppendOnly. It simulates merging without actually indexing docs, which allows simulating many many rounds of merging in very little time, while comparing how some parameters (like targetSearchConcurrency) affect merging decisions and metrics such as write amplification through merges.

I've given it a try - LMKWYT!

… to byte merges

carlosdelest · 2024-05-30T10:38:00Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

@@ -257,6 +258,25 @@ public double getSegmentsPerTier() {
    return segsPerTier;
  }

+  /**
+   * Sets the target search concurrency. This allows merging to ensure that there are at least
+   * targetSearchConcurrency segments on the top tier. This setting can be overriden by force


Makes sense, updating.

carlosdelest · 2024-05-30T10:38:19Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

@@ -93,6 +93,7 @@ public class TieredMergePolicy extends MergePolicy {
  private double segsPerTier = 10.0;
  private double forceMergeDeletesPctAllowed = 10.0;
  private double deletesPctAllowed = 20.0;
+  private int targetSearchConcurrency = -1;


Yes - I iterated a bit on this but now it's equivalent. Thanks!

carlosdelest · 2024-05-30T10:41:00Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

@@ -409,6 +433,8 @@ public MergeSpecification findMerges(
    // allowedSegCount may occasionally be less than segsPerTier
    // if segment sizes are below the floor size
    allowedSegCount = Math.max(allowedSegCount, segsPerTier);
+    // Override with the targetSearchConcurrency if it is greater
+    allowedSegCount = Math.max(allowedSegCount, targetSearchConcurrency);


Correct, now the highest tier needs to take that into account. Thanks! 👍

carlosdelest · 2024-05-30T10:48:44Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

          break;
        }

-        final MergeScore score = score(candidate, hitTooLarge, segInfosSizes);
+        final MergeScore score = score(candidate, hitTooLarge || hitMaxDocs, segInfosSizes);


I see - I was thinking in doing the same with docs than with bytes, but I see they're quite different. Docs is more a hint to maintain the number of segments we want, but bytes provides a hard limit on size and also score guidance on segments to merge.

carlosdelest · 2024-05-31T09:18:55Z

lucene/core/src/test/org/apache/lucene/index/TestTieredMergePolicy.java

@@ -431,6 +447,128 @@ public void testForcedMergesRespectSegSize() throws Exception {
    dir.close();
  }

+  public void testForcedMergesRespectsTargetSearchConcurrency() throws Exception {


I've given it a try - LMKWYT!

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

jpountz · 2024-06-04T07:23:54Z

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

+          int segDocCount = segSizeDocs.maxDoc - segSizeDocs.delCount;
+          if (docCountThisMerge + segDocCount > allowedDocCount) {
+            // We don't want to merge segments that will produce more documents than allowedDocCount
+            continue;


I think we should break here, not continue. continue allows producing merges of segments whose sizes are not adjacent, I don't think we should allow this for the doc count condition, as this potentially makes merging run in quadratic time.

We probably also should produce a singleton merge here in case candidate.isEmpty() (see what we're doing for too large segments below). Most likely this suggested merge will not be selected because it will have a low score, though it could end up selected if it reclaims lots of deletes and we don't have better merges.

Makes sense, thanks!

lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java

jpountz · 2024-06-04T07:49:43Z

lucene/core/src/test/org/apache/lucene/index/TestTieredMergePolicy.java

    // below we make the assumption that segments that reached the max segment
    // size divided by 2 don't need merging anymore
    int mergeFactor = (int) Math.min(tmp.getSegmentsPerTier(), tmp.getMaxMergeAtOnce());
    while (true) {
      final double segCountLevel = bytesLeft / (double) levelSizeBytes;
-      if (segCountLevel < tmp.getSegmentsPerTier() || levelSizeBytes >= maxMergedSegmentBytes / 2) {
+      if (segCountLevel < maxNumSegmentsOnHighestTier


Should it allow maxNumSegmentsOnHighestTier for consistency with the code under main? Though I don't expect it to make a big difference since segCountLevel is a double.

Suggested change

if (segCountLevel < maxNumSegmentsOnHighestTier

if (segCountLevel <= maxNumSegmentsOnHighestTier

Good catch - updating

jpountz · 2024-06-04T07:51:44Z

lucene/core/src/test/org/apache/lucene/index/TestTieredMergePolicy.java

@@ -111,11 +114,29 @@ protected void assertSegmentInfos(MergePolicy policy, SegmentInfos infos) throws
      }
    }

+    // There can be more segments if we can't merge docs - they are balanced between segments
+    int maxDocsPerSegment = tmp.getMaxAllowedDocs(infos.totalMaxDoc());


should we subtract deletions to totalMaxDoc?

Yes, we should. Thanks!

jpountz · 2024-06-04T07:55:52Z

lucene/core/src/test/org/apache/lucene/index/TestTieredMergePolicy.java

+      }
+      numEligible++;
+    }
+    boolean eligibleDocsMerge = numEligible > 1;


I wonder if the condition can be made simpler. Would it be ok to check the sum of the doc counts of the two smallest segments? If it's greater than the allowed doc count, then no merging is allowed, otherwise this 2-segments merge would have been a legal merge?

Correct, applied the change.

…ments

jpountz · 2024-06-24T14:35:22Z

I spent some time digging into the new behavior introduced by this PR, as I'd like to get it over the finish line. :) I am getting disappointing results for the write amplification. This seems to be due to the fact that we compute the segment count as if segments could magically be close to the doc count limit, but in practice this is not the case, and it forces TieredMergePolicy to perform sub-optimal merging.

I tried to play with other approaches, and one that seems to perform closer to what I would expect consists of removing the targetConcurrency biggest segments from the list of eligible segments, in a similar way to how we remove too big segments. And then let the merge policy do merging on the long tail of smaller segments.

jpountz · 2024-06-24T16:34:19Z

@carlosdelest I took the freedom of pushing some changes to your fork, I hope you don't mind!

jpountz · 2024-06-24T16:41:15Z

FWIW I have been playing with BaseMergePolicyTestCase's doTestSimulateAppendOnly and doTestSimulateUpdates. With this change, I'm observing a number of segments in the order of targetSearchConcurrency on the highest tier while increasing the segment count by a bit less targetSearchConcurrency on average and without affecting the write amplification significantly.

This adds the same `targetSearchConcurrency` parameter to `LogMergePolicy` that apache#13430 is adding to `TieredMergePolicy`. The implementation is simpler here since `LogMergePolicy` is constrained to only merging adjacent segments. The downside is that this option results is significantly more write amplification on this merge policy compared with `TieredMergePolicy`, which is allowed to merge non-adjacent segments. From simulating merging on an index that gets appended 555 equal-size segments (like nightly benchmarks) and a `targetSearchConcurrency` of 8, I get the following results: - 19.1 segments in the index on average, vs. 11.1 with a `targetSearchConcurrency` of 1. - Maximum number of segments in the index of 28, vs. 22 with a `targetSearchConcurrency` of 1. - Write amplification through merging of 3.6 instead of 2.9. For comparison, 3.6 is about the write amplification that you get with a merge factor of 7 otherwise. - The resulting index has 24 segments, including 11 segments on the highest tier, vs. 15 segments and 5 segments on the highest tier with a `targetSearchConcurrency` of 1.

carlosdelest · 2024-07-02T14:55:17Z

Thanks for taking a look into this @jpountz !

Your approach makes sense to me, and addresses some of the problems I was having with bigger segments not being merged because of the number of documents, and smaller segments being discarded.

The test checks makes sense as well. I've added setting target search concurrency for testSimulateUpdates()

Is there any other work / checks we should be doing for this?

jpountz

I just reviewed the change, it looks good to me. I also ran more simulations for write amplification, and the results look good to me:

Scenario 1: 555 flushes of exactly 60065-document segments (like nightlies) where each doc contributes 5kB to the index. No updates/deletions.

Merge policy	Write amplification	Average number of segments	Max number of segments
TieredMergePolicy defaults	1.99	33.62	65
TieredMergePolicy targetSearchConcurrency=4	2.05	33.53	65
TieredMergePolicy targetSearchConcurrency=8	2.19	34.31	65
TieredMergePolicy targetSearchConcurrency=16	2.47	38.56	67

Scenario 2: 1000 flushes of random(1, 50k)-document segments where each doc contributes 5kB to the index. No updates/deletions.

Merge policy	Write amplification	Average number of segments	Max number of segments
TieredMergePolicy defaults	2.86	34.6	54
TieredMergePolicy targetSearchConcurrency=4	2.93	36.0	56
TieredMergePolicy targetSearchConcurrency=8	2.98	38.8	58
TieredMergePolicy targetSearchConcurrency=16	3.24	43.6	61

Scenario 3: 1000 flushes of random(1, 50k)-document segments where each doc contributes 5kB to the index. 50% of the documents in every flush are updates, ie. each flush contributes N/2 new documents and N/2 updated documents.

Merge policy	Write amplification	Average number of segments	Max number of segments
TieredMergePolicy defaults	3.13	28.8	46
TieredMergePolicy targetSearchConcurrency=4	3.12	28.9	46
TieredMergePolicy targetSearchConcurrency=8	3.31	30.7	46
TieredMergePolicy targetSearchConcurrency=16	3.66	33.5	51

One thing worth noting is that it becomes easier to enforce the target search concurrency as more segments get added to the index

@mikemccand would you like to take a look before I merge?

As a follow-up, I think we should enable this feature by default? Maybe only in Lucene 10? With a value such as 8?

I just pushed a minor improvement that ignores the limit on the doc count if we're still under the floor segment size, as it felt silly to apply this constraint in this case.

Is there any other work / checks we should be doing for this?

I would suggest improving LuceneTestCase#newTieredMergePolicy(Random) should randomly set the target concurrency on the merge policy. And remove the call to the setter to the BaseMergePolicyTestCase#testSimulate* test cases?

carlosdelest · 2024-07-16T16:13:21Z

I would suggest improving LuceneTestCase#newTieredMergePolicy(Random) should randomly set the target concurrency on the merge policy. And remove the call to the setter to the BaseMergePolicyTestCase#testSimulate* test cases?

Done in 2d80315

… in append only tests

jpountz · 2024-07-17T07:15:45Z

lucene/test-framework/src/java/org/apache/lucene/tests/util/LuceneTestCase.java

+    if (rarely(r)) {
+      tmp.setTargetSearchConcurrency(TestUtil.nextInt(r, 2, 20));
+    } else {
+      tmp.setTargetSearchConcurrency(TestUtil.nextInt(r, 10, 50));


I would have expected the opposite: that we commonly set a reasonable target search concurrency, and rarely set a high value?

I see - I was mimicking the segments per tier approach here, but I guess that makes sense. Changing it

This adds the same `targetSearchConcurrency` parameter to `LogMergePolicy` that #13430 is adding to `TieredMergePolicy`.

carlosdelest added 6 commits May 27, 2024 16:34

Add targetSearchConcurrency

ab0a13a

Some renaming and fixes

ecf2035

Some renaming and fixes

6bf94d2

minNumSegments can be overriden by force merge

36ad550

Add minNumSegments to tests

585c927

Merge remote-tracking branch 'origin/main'

c922f47

carlosdelest changed the title ~~Add minimum number of segments to TieredMergePolicy~~ WIP - Add minimum number of segments to TieredMergePolicy May 27, 2024

carlosdelest added 4 commits May 28, 2024 11:46

Add forceMerge semantics

f74616c

WIP - test

31e132f

Take forced merges into account

2d6173d

Minor fixes, adding tests

22436c0

jpountz reviewed May 28, 2024

View reviewed changes

carlosdelest added 2 commits May 29, 2024 16:21

Rename minNumSegments -> targetSearchConcurrency

7c4397e

Implement target search concurrency as number of segments for the las…

289d07d

…t tier

jpountz reviewed May 30, 2024

View reviewed changes

carlosdelest added 4 commits May 30, 2024 12:53

PR feedback

d4d89f1

Removed some changes for not making doc merges so aggressive compared…

bfdb58b

… to byte merges

Focus testing on append only merges for target search concurrency

f240644

Removing some useless changes from main

500fe32

carlosdelest commented May 31, 2024

View reviewed changes

jpountz reviewed Jun 4, 2024

View reviewed changes

carlosdelest and others added 4 commits June 4, 2024 19:09

Better handling of num docs overflow

1b91298

Simplify testing

299ac18

Refactor singleton merges

e058a9f

Merge branch 'main' into carlosdelest/tiered-merge-policy-min-num-seg…

9ab6fcd

…ments

jpountz added 2 commits June 24, 2024 18:28

Tune merging logic for targetSearchConcurrency.

d44de4c

Add CHANGES entry.

66d62e9

jpountz marked this pull request as ready for review June 24, 2024 16:34

jpountz changed the title ~~WIP - Add minimum number of segments to TieredMergePolicy~~ Add target search concurrency to TieredMergePolicy Jun 24, 2024

jpountz added this to the 9.12.0 milestone Jun 24, 2024

jpountz mentioned this pull request Jun 25, 2024

Add a targetSearchConcurrency parameter to LogMergePolicy. #13517

Merged

carlosdelest added 3 commits July 2, 2024 15:42

Add proper credit to Adrien on this :)

ca4c19c

Small refactoring for getMaxAllowedDocs

2c18e5f

Add target search concurrency checks in testSimulateUpdates()

1b9b60b

Minor improvement to corner case.

3db835a

jpountz approved these changes Jul 16, 2024

View reviewed changes

Add target search concurrency to all tests randomly

2d80315

Remove check for target search concurrency, which already takes place…

8de0212

… in append only tests

jpountz reviewed Jul 17, 2024

View reviewed changes

Change target search concurrency for rare test occurrences

898e2e8

jpountz merged commit 22ca695 into apache:main Jul 17, 2024
3 checks passed

jpountz pushed a commit that referenced this pull request Jul 17, 2024

Add target search concurrency to TieredMergePolicy (#13430)

3ee3769

goankur pushed a commit to goankur/lucene that referenced this pull request Jul 18, 2024

Add target search concurrency to TieredMergePolicy (apache#13430)

dea5f28

jpountz added a commit that referenced this pull request Jul 18, 2024

Add a targetSearchConcurrency parameter to LogMergePolicy. (#13517)

9f04086

This adds the same `targetSearchConcurrency` parameter to `LogMergePolicy` that #13430 is adding to `TieredMergePolicy`.

jpountz added a commit that referenced this pull request Jul 18, 2024

Add a targetSearchConcurrency parameter to LogMergePolicy. (#13517)

bbcd65f

This adds the same `targetSearchConcurrency` parameter to `LogMergePolicy` that #13430 is adding to `TieredMergePolicy`.

cpoerschke mentioned this pull request Aug 12, 2024

document Lucene 9.12's TieredMergePolicyFactory.targetSearchConcurrency parameter apache/solr#2640

Draft

ChrisHegarty mentioned this pull request Sep 23, 2024

TestTieredMergePolicy.testSimulateAppendOnly fails with AssertionError #13818

Closed

	if (segCountLevel < maxNumSegmentsOnHighestTier
	if (segCountLevel <= maxNumSegmentsOnHighestTier

Add target search concurrency to TieredMergePolicy #13430

Add target search concurrency to TieredMergePolicy #13430

Conversation

carlosdelest commented May 27, 2024

jpountz left a comment

Choose a reason for hiding this comment

carlosdelest commented May 29, 2024

jpountz commented May 29, 2024

mikemccand commented May 29, 2024

carlosdelest commented May 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpountz commented Jun 24, 2024

jpountz commented Jun 24, 2024

jpountz commented Jun 24, 2024

carlosdelest commented Jul 2, 2024

jpountz left a comment

Choose a reason for hiding this comment

carlosdelest commented Jul 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment