Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Blog] Reducing AD False Positives Through Algorithmic Improvements #3524

Merged
merged 7 commits into from
Feb 5, 2025

Conversation

kaituo
Copy link
Contributor

@kaituo kaituo commented Dec 26, 2024

Description

This PR contains the content for publishing the blog on Reducing AD False Positives Through Algorithmic Improvements.

Issues Resolved

#3523

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

…es Through Algorithmic Improvements

This PR contains the content for publishing the blog on  Reducing AD False Positives Through Algorithmic Improvements.

Signed-off-by: Kaituo Li <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
@kolchfa-aws kolchfa-aws self-assigned this Jan 8, 2025
@kolchfa-aws
Copy link
Collaborator

@kaituo Doc review complete: could you please review my changes and verify that they preserve technical accuracy? I changed some of the terms (like "alert once suppression" -> "grouped alerting") for more intrinsic understanding. Also, could you please change the headings in the images so they match the new headings? And please use sentence case for headings (first word capitalized and the others lowercase, unless it's a proper noun). Thanks!

@pajuric
Copy link

pajuric commented Jan 16, 2025

@kaituo - Have you incorporated Fanit's suggested edits into the PR?

Signed-off-by: Kaituo Li <[email protected]>
@kaituo
Copy link
Contributor Author

kaituo commented Feb 3, 2025

@pajuric Just incorporated @kolchfa-aws and Nina Mishra's feedback.

@kolchfa-aws Please take a look.

Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
@kaituo
Copy link
Contributor Author

kaituo commented Feb 4, 2025

@kolchfa-aws The doc review looks good.

Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolchfa-aws Editorial review complete. Please see my comments and changes and let me know if you have any questions. Thanks!


![network_zoom_in](/assets/media/blog-images/2024-12-26-Reducing-false-positives-through-algorithmic-improvements/network_zoom_in_1.png){:class="img-centered"}
When excluding the first 20% of data as a probation period, precision improves across datasets. This adjustment highlights how RCF becomes more accurate after observing enough historical data. The results of the last 80% of the dataset are summarized in the following table.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 203, first sentence: "of the data" => "of the dataset" (to match the last sentence)?

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
@kolchfa-aws
Copy link
Collaborator

@pajuric Please update the meta/keywords for this blog, and it'll be ready to publish.

- technical-posts
has_math: true
has_science_table: true
meta_keywords: anomaly detection, false positives, algorithmic improvements, OpenSearch, nab benchmark
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the meta with the following:
meta_keywords: anomaly detection, OpenSearch RCF algorithm, Random Cut Forest, value vectors, Numenta Anomaly Benchmark, algorithmic improvements in OpenSearch

meta_description: Explore how OpenSearch 2.17 enhanced the Random Cut Forest algorithm, making is possible to reduce false positives by 94.3% while maintaining detection accuracy. Get benchmark results and read practical studies.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

authors:
- kaituo
- kolchfa
date: 2024-12-26
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the publish date to today: 2025-02-05

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@pajuric
Copy link

pajuric commented Feb 5, 2025

@nateynateynate @krisfreedain - Please help push this live.

@pajuric
Copy link

pajuric commented Feb 5, 2025

@peterzhuamazon - Would you be available to help us push this live?

@peterzhuamazon peterzhuamazon merged commit 224fc31 into opensearch-project:main Feb 5, 2025
5 checks passed
@peterzhuamazon
Copy link
Member

I will update the time from 2024-02-05 to 2025-02-05 soon before production.

@peterzhuamazon
Copy link
Member

#3625

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants