Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub-Issue#2778: Added CloudWatchLogsSink #3083

Conversation

MaGonzalMayedo
Copy link
Contributor

@MaGonzalMayedo MaGonzalMayedo commented Jul 31, 2023

Description

This change implements the main Sink entry point for the CloudWatchLogs Sink. It extends the plugin by adding a single sink class alongside a test.

Issues Resolved

This PR will work towards resolving CWL-Sink for Data-Prepper [Issue https://github.com/https://github.com/https://github.com/https://github.com//issues/2778]
Link: #2778 (comment)

Check List

  • New functionality includes testing.
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

graytaylor0 and others added 30 commits July 3, 2023 12:56
…pensearch-project#2910)

Create Elasticsearch client, implement search and pit apis for ElasticsearchAccessor

Signed-off-by: Taylor Gray <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Added Config Files for CloudWatchLogs Sink.

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Added default params for back_off and log_send_interval alongside test cases for ThresholdConfig.

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…ith AwsConfig.

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…mer and params to AwsConfig

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…tive mapping to tests CwlSink

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…t the plugin use case more

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…, made fixes to gradle file to include catalog

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…sionLimitException

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…d fixes to variables and retry logic for InterruptExceptions

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Marcos Gonzalez Mayedo and others added 17 commits July 20, 2023 12:21
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…ervice, and fixed Log calling methods

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…ce and Dispatcher and modified backOffTimeBase

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by:Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…, and added logString pass in parameter for staging log events.

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…ure (performance enhancement)

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…ition of Uploader, introduced simple multithread tests for CloudWatchLogsService

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…tests to the CloudWatchLogsService class

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…d scale, and refactoring changes for better code syntax (renaming, refactoring methods for conciseness, etc...)

Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
…rch/dataprepper/plugins/sink/client/CloudWatchLogsDispatcher.java

Co-authored-by: Mark Kuhn <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Signed-off-by: Marcos Gonzalez Mayedo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants