add a feature that flattens custom result index when enabled #1401

jackiehanyang · 2025-01-15T02:46:11Z

Description

wip - sending implementation out first.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang · 2025-01-24T00:47:09Z

resolved
~~non related IT test case failed:~~
REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsWithCustomResultIndex" -Dtests.seed=764AE145CE6907EA -Dtests.security.manager=false -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-419 -Dtests.timezone=NZ -Druntime.java=21
org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testSearchTopAnomalyResultsWithCustomResultIndex FAILED
org.opensearch.client.ResponseException: method [POST], host [http://127.0.0.1:44973/], URI [/_plugins/_anomaly_detection/detectors], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"status_exception","reason":"Fail to create detector"}],"type":"status_exception","reason":"Fail to create detector"},"status":500}
at __randomizedtesting.SeedInfo.seed([764AE145CE6907EA:A9D4336811E01C9A]:0)
at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:501)
at app//org.opensearch.client.RestClient.performRequest(RestClient.java:384)
at app//org.opensearch.client.RestClient.performRequest(RestClient.java:359)
at app//org.opensearch.timeseries.TestHelpers.makeRequest(TestHelpers.java:231)
at app//org.opensearch.timeseries.TestHelpers.makeRequest(TestHelpers.java:204)
at app//org.opensearch.ad.AnomalyDetectorRestTestCase.createAnomalyDetector(AnomalyDetectorRestTestCase.java:133)
at app//org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsWithCustomResultIndex(AnomalyDetectorRestApiIT.java:2069)

src/main/java/org/opensearch/ad/settings/AnomalyDetectorSettings.java

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/ad/transport/ADResultBulkTransportAction.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java

Signed-off-by: Jackie Han <[email protected]>

kaituo · 2025-01-28T18:23:06Z

bwc test failure is similar to opensearch-project/OpenSearch#15234

=== Standard error of node `node{::adBwcCluster#twoThirdsUpgradedClusterTask-0}` ===
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:393) [opensearch-2.19.0.jar:2.19.0]
?  	... 41 more
?  Caused by: java.lang.IllegalStateException: unexpected byte [0xac]
?  	at org.opensearch.core.common.io.stream.StreamInput.readBoolean(StreamInput.java:596) ~[opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.common.io.stream.StreamInput.readOptionalBoolean(StreamInput.java:606) ~[opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.search.internal.ShardSearchRequest.<init>(ShardSearchRequest.java:255) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.RequestHandlerRegistry.newRequest(RequestHandlerRegistry.java:87) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.newRequest(NativeMessageHandler.java:316) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleRequest(NativeMessageHandler.java:271) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleMessage(NativeMessageHandler.java:146) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.messageReceived(NativeMessageHandler.java:126) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.messageReceivedFromPipeline(InboundHandler.java:120) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:112) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:768) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.forwardFragments(InboundBytesHandler.java:137) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.doHandleBytes(InboundBytesHandler.java:77) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:124) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:113) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:95) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
?  	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
?  	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
?  	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) ~[?:?]
?  	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?]
?  	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
?  	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
?  	at java.lang.Thread.run(Thread.java:1583) ~[?:?]
? WARN ][r.suppressed             ] [adBwcCluster0-2] path: /_opendistro/_anomaly_detection/detectors/yHNUrJQBrgHv3CV3UwAC/_start, params: {detectorID=yHNUrJQBrgHv3CV3UwAC}
?  org.opensearch.OpenSearchStatusException: Fail to start detector
?  	at org.opensearch.timeseries.util.RestHandlerUtils.lambda$wrapRestActionListener$2(RestHandlerUtils.java:243) [opensearch-anomaly-detection-2.19.0.0.jar:2.19.0.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.timeseries.rest.handler.IndexJobActionHandler.lambda$onGetJobForWrite$10(IndexJobActionHandler.java:303) [opensearch-anomaly-detection-2.19.0.0.jar:2.19.0.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.timeseries.task.TaskManager.lambda$getAndExecuteOnLatestTasks$17(TaskManager.java:605) [opensearch-anomaly-detection-2.19.0.0.jar:2.19.0.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.support.TransportAction$1.onFailure(TransportAction.java:124) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.action.ActionListener$5.onFailure(ActionListener.java:277) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.raisePhaseFailure(AbstractSearchAsyncAction.java:802) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:775) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:395) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:815) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:548) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:316) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:75) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:766) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TransportService$9.handleException(TransportService.java:1741) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1527) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.lambda$handleException$5(NativeMessageHandler.java:454) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleException(NativeMessageHandler.java:452) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handlerResponseError(NativeMessageHandler.java:444) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleMessage(NativeMessageHandler.java:172) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.messageReceived(NativeMessageHandler.java:126) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.messageReceivedFromPipeline(InboundHandler.java:120) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:112) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:768) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.forwardFragments(InboundBytesHandler.java:137) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.doHandleBytes(InboundBytesHandler.java:77) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:124) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:113) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:95) [transport-netty4-client-2.19.0.jar:2.19.0]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) [netty-handler-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.117.Final.jar:4.1.117.Final]
?  	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
?   ? last 40 non error or warning messages from /__w/anomaly-detection/anomaly-detection/build/testclusters/adBwcCluster0-2/logs/opensearch.stdout.log ?
? [2025-01-28T09:53:13,046][DEBUG][o.o.c.c.C.CoordinatorPublication] [adBwcCluster0-2] publishing version 63 to [PublicationTarget{discoveryNode={adBwcCluster0-0}{yxG8heptSiK2eK9S9Ks46A}{mFsI26s-QTuJOWcGDLRAjQ}{127.0.0.1}{127.0.0.1:37589}{dimr}{upgraded=true, testattr=test, shard_indexing_pressure_enabled=true}, state=NOT_STARTED, ackIsPending=true}, PublicationTarget{discoveryNode={adBwcCluster0-1}{th_t3o4ySIqKWHrFfQg35g}{IePje0osQvilETYvk6-fFA}{127.0.0.1}{127.0.0.1:45277}{dimr}{upgraded=true, testattr=test, shard_indexing_pressure_enabled=true}, state=NOT_STARTED, ackIsPending=true}, PublicationTarget{discoveryNode={adBwcCluster0-2}{jpEavNb_SKyKsfA6FVKT0A}{SalIDjFNQGC9PjW8hGFlLQ}{127.0.0.1}{127.0.0.1:33951}{dimr}{testattr=test, shard_indexing_pressure_enabled=true}, state=NOT_STARTED, ackIsPending=true}]
? [2025-01-28T09:53:13,046][DEBUG][o.o.c.c.PublicationTransportHandler] [adBwcCluster0-2] received diff cluster state version [63] with uuid [_1E3aVONQ_6Q0Oj7pl9LVg], diff size [589]

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

Signed-off-by: Jackie Han <[email protected]>

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

Signed-off-by: Jackie Han <[email protected]>

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java

…ult index Signed-off-by: Jackie Han <[email protected]>

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/ad/transport/ADResultBulkTransportAction.java

Signed-off-by: Jackie Han <[email protected]>

codecov · 2025-01-29T10:32:01Z

Codecov Report

Attention: Patch coverage is 78.46154% with 42 lines in your changes missing coverage. Please review.

Project coverage is 81.54%. Comparing base (926b2e1) to head (8a9429d).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...opensearch/timeseries/indices/IndexManagement.java	60.00%	11 Missing and 3 partials ⚠️
.../rest/handler/AbstractTimeSeriesActionHandler.java	86.27%	11 Missing and 3 partials ⚠️
...g/opensearch/timeseries/util/RestHandlerUtils.java	36.36%	6 Missing and 1 partial ⚠️
...transport/ADSingleStreamResultTransportAction.java	0.00%	2 Missing ⚠️
...ort/ForecastSingleStreamResultTransportAction.java	0.00%	2 Missing ⚠️
...n/java/org/opensearch/timeseries/model/Config.java	75.00%	1 Missing and 1 partial ⚠️
.../forecast/ratelimit/ForecastResultWriteWorker.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1401      +/-   ##
============================================
- Coverage     81.58%   81.54%   -0.05%     
- Complexity     5826     5857      +31     
============================================
  Files           533      533              
  Lines         23522    23675     +153     
  Branches       2367     2375       +8     
============================================
+ Hits          19191    19305     +114     
- Misses         3175     3204      +29     
- Partials       1156     1166      +10

Flag	Coverage Δ
plugin	`81.54% <78.46%> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...a/org/opensearch/ad/indices/ADIndexManagement.java	`84.61% <100.00%> (+1.28%)`	⬆️
.../java/org/opensearch/ad/model/AnomalyDetector.java	`88.85% <ø> (ø)`
.../opensearch/ad/ratelimit/ADResultWriteRequest.java	`100.00% <100.00%> (ø)`
...g/opensearch/ad/ratelimit/ADResultWriteWorker.java	`100.00% <100.00%> (ø)`
.../opensearch/ad/ratelimit/ADSaveResultStrategy.java	`96.66% <100.00%> (+0.11%)`	⬆️
...arch/ad/transport/ADResultBulkTransportAction.java	`94.11% <100.00%> (+2.68%)`	⬆️
...java/org/opensearch/forecast/model/Forecaster.java	`80.81% <ø> (+1.74%)`	⬆️
...forecast/ratelimit/ForecastResultWriteRequest.java	`50.00% <100.00%> (ø)`
...forecast/ratelimit/ForecastSaveResultStrategy.java	`96.55% <100.00%> (+0.12%)`	⬆️
...opensearch/timeseries/constant/CommonMessages.java	`97.50% <ø> (ø)`
... and 11 more

... and 10 files with indirect coverage changes

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

Signed-off-by: Jackie Han <[email protected]>

opensearch-trigger-bot · 2025-01-29T19:47:11Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/anomaly-detection/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/anomaly-detection/backport-2.x
# Create a new branch
git switch --create backport/backport-1401-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e33ff46cd1e6d8c0607708f139b1424a612360c7
# Push it to GitHub
git push --set-upstream origin backport/backport-1401-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/anomaly-detection/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1401-to-2.x.

…rch-project#1401) * add a feature that flattens custom result index when enabled Signed-off-by: Jackie Han <[email protected]> * clean up Signed-off-by: Jackie Han <[email protected]> * add IT Signed-off-by: Jackie Han <[email protected]> * cleanup Signed-off-by: Jackie Han <[email protected]> * address comments Signed-off-by: Jackie Han <[email protected]> * update IT Signed-off-by: Jackie Han <[email protected]> * update IT Signed-off-by: Jackie Han <[email protected]> * clean up Signed-off-by: Jackie Han <[email protected]> * add more IT Signed-off-by: Jackie Han <[email protected]> * add more IT Signed-off-by: Jackie Han <[email protected]> * add more IT Signed-off-by: Jackie Han <[email protected]> * address comments * address comments Signed-off-by: Jackie Han <[email protected]> * address comments Signed-off-by: Jackie Han <[email protected]> * utlizing a node state manager when writing results into flattened result index Signed-off-by: Jackie Han <[email protected]> * build flatten resuilt index enabled into the ResultWriteRequest Signed-off-by: Jackie Han <[email protected]> * unbind ingest pipeline with flattened result index when it's disabled Signed-off-by: Jackie Han <[email protected]> * test * address comments * cleanup Signed-off-by: Jackie Han <[email protected]> * make flatten result index use detector name Signed-off-by: Jackie Han <[email protected]> --------- Signed-off-by: Jackie Han <[email protected]> Signed-off-by: Jackie Han <[email protected]>

…nabled (#1401) (#1405) * add a feature that flattens custom result index when enabled (#1401) * add a feature that flattens custom result index when enabled Signed-off-by: Jackie Han <[email protected]> * clean up Signed-off-by: Jackie Han <[email protected]> * add IT Signed-off-by: Jackie Han <[email protected]> * cleanup Signed-off-by: Jackie Han <[email protected]> * address comments Signed-off-by: Jackie Han <[email protected]> * update IT Signed-off-by: Jackie Han <[email protected]> * update IT Signed-off-by: Jackie Han <[email protected]> * clean up Signed-off-by: Jackie Han <[email protected]> * add more IT Signed-off-by: Jackie Han <[email protected]> * add more IT Signed-off-by: Jackie Han <[email protected]> * add more IT Signed-off-by: Jackie Han <[email protected]> * address comments * address comments Signed-off-by: Jackie Han <[email protected]> * address comments Signed-off-by: Jackie Han <[email protected]> * utlizing a node state manager when writing results into flattened result index Signed-off-by: Jackie Han <[email protected]> * build flatten resuilt index enabled into the ResultWriteRequest Signed-off-by: Jackie Han <[email protected]> * unbind ingest pipeline with flattened result index when it's disabled Signed-off-by: Jackie Han <[email protected]> * test * address comments * cleanup Signed-off-by: Jackie Han <[email protected]> * make flatten result index use detector name Signed-off-by: Jackie Han <[email protected]> --------- Signed-off-by: Jackie Han <[email protected]> Signed-off-by: Jackie Han <[email protected]> * spotless apply Signed-off-by: Jackie Han <[email protected]> * update recency_emphasis to be greater than 1 in test cases Signed-off-by: Jackie Han <[email protected]> --------- Signed-off-by: Jackie Han <[email protected]> Signed-off-by: Jackie Han <[email protected]>

add a feature that flattens custom result index when enabled

41d5949

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang requested review from jmazanec15, jngz-es, kaituo, saratvemulapalli, ohltyler, vamshin, VijayanB, ylwu-amzn, amitgalitz, sean-zheng-amazon, dbwiddis, owaiskazi19 and joshpalis as code owners January 15, 2025 02:46

opensearch-trigger-bot bot added the backport 2.x label Jan 15, 2025

jackiehanyang force-pushed the ingestPipelineFlatten branch 3 times, most recently from dc1c604 to 9f0dcf6 Compare January 15, 2025 03:10

clean up

352075b

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang force-pushed the ingestPipelineFlatten branch from d0d60f0 to 352075b Compare January 15, 2025 21:19

add IT

b131f9a

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang force-pushed the ingestPipelineFlatten branch from c947793 to bf552b5 Compare January 23, 2025 23:50

cleanup

9fced72

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang force-pushed the ingestPipelineFlatten branch from bf552b5 to 9fced72 Compare January 23, 2025 23:52

kaituo reviewed Jan 24, 2025

View reviewed changes

jackiehanyang force-pushed the ingestPipelineFlatten branch 2 times, most recently from 2fd6bc2 to 8b12fd7 Compare January 27, 2025 07:22

address comments

3ce0734

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang force-pushed the ingestPipelineFlatten branch from 8b12fd7 to 3ce0734 Compare January 27, 2025 08:07

jackiehanyang force-pushed the ingestPipelineFlatten branch from ad3d99c to c844922 Compare January 28, 2025 09:44

kaituo reviewed Jan 28, 2025

View reviewed changes

jackiehanyang added 2 commits January 28, 2025 11:58

address comments

d4b9262

address comments

781d2da

Signed-off-by: Jackie Han <[email protected]>

kaituo reviewed Jan 28, 2025

View reviewed changes

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java Show resolved Hide resolved

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java Outdated Show resolved Hide resolved

address comments

a623fcf

Signed-off-by: Jackie Han <[email protected]>

kaituo reviewed Jan 28, 2025

View reviewed changes

jackiehanyang mentioned this pull request Jan 29, 2025

[DOC] Flatten Custom Result Index in Anomaly Detection opensearch-project/documentation-website#9132

Closed

4 tasks

utlizing a node state manager when writing results into flattened res…

cf8bed4

…ult index Signed-off-by: Jackie Han <[email protected]>

kaituo reviewed Jan 29, 2025

View reviewed changes

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java Outdated Show resolved Hide resolved

src/main/java/org/opensearch/ad/transport/ADResultBulkTransportAction.java Outdated Show resolved Hide resolved

build flatten resuilt index enabled into the ResultWriteRequest

d557c50

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang force-pushed the ingestPipelineFlatten branch from 590fb5a to d557c50 Compare January 29, 2025 06:18

jackiehanyang added 4 commits January 28, 2025 22:54

unbind ingest pipeline with flattened result index when it's disabled

b5240df

Signed-off-by: Jackie Han <[email protected]>

test

ffd7d94

address comments

05f56a3

cleanup

64cc3bc

Signed-off-by: Jackie Han <[email protected]>

jackiehanyang mentioned this pull request Jan 29, 2025

add ingest pipeline and indices related permissions for anomaly_full_access role opensearch-project/security#5065

Merged

5 tasks

kaituo reviewed Jan 29, 2025

View reviewed changes

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java Outdated Show resolved Hide resolved

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java Outdated Show resolved Hide resolved

make flatten result index use detector name

8a9429d

Signed-off-by: Jackie Han <[email protected]>

kaituo approved these changes Jan 29, 2025

View reviewed changes

jackiehanyang merged commit e33ff46 into opensearch-project:main Jan 29, 2025
19 of 20 checks passed

opensearch-trigger-bot bot added the backport-failed label Jan 29, 2025

jackiehanyang mentioned this pull request Jan 29, 2025

[Backport 2.x] add a feature that flattens custom result index when enabled (#1401) #1405

Merged

This was referenced Jan 30, 2025

add documentation for AD flatten result index feature opensearch-project/documentation-website#9140

Merged

Add release notes for 2.19.0 #1412

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add a feature that flattens custom result index when enabled #1401

add a feature that flattens custom result index when enabled #1401

jackiehanyang commented Jan 15, 2025

jackiehanyang commented Jan 24, 2025 •

edited

Loading

kaituo commented Jan 28, 2025

codecov bot commented Jan 29, 2025 •

edited

Loading

opensearch-trigger-bot bot commented Jan 29, 2025

add a feature that flattens custom result index when enabled #1401

add a feature that flattens custom result index when enabled #1401

Conversation

jackiehanyang commented Jan 15, 2025

Description

Related Issues

Check List

jackiehanyang commented Jan 24, 2025 • edited Loading

kaituo commented Jan 28, 2025

codecov bot commented Jan 29, 2025 • edited Loading

Codecov Report

opensearch-trigger-bot bot commented Jan 29, 2025

jackiehanyang commented Jan 24, 2025 •

edited

Loading

codecov bot commented Jan 29, 2025 •

edited

Loading