[azure_logs] Add Custom Azure Logs input package

[zmoog]

Proposed commit message

Add a new input package to collect custom logs from Event Hub.

This input package is a thin wrapper on the azure-eventhub input that allows users to receive JSON log events from an event hub and transform them using custom pipelines and mappings.

Users should use this input package instead of generic integration in the Azure Logs integration.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• Add pipeline override
• Replace the logo with the one used in all input packages
• Write the docs (overview, requirements, setup)
• How to deal with routing

Reviewer's Checklist

Please review the following items and, ideally, leave your thoughts in the comments.

• Routing: I could not find a way to add an OOTB pipeline to route logs to other data streams. Do you know a method?
• Logo: all the input packages use the same logo. Should we adopt the same logo or use something Azure-specific?
• Data model: Currently, the input package stores the log events sent by the input without any additional change. The log event contains the JSON document as a string in the message field. This gives users all the options to transform the log event in the custom pipeline. Do you have any suggestions?
• Ownership: I currently set the package ownership to elastic/obs-ds-hosted-services. Should we set the ownership to elastic/obs-infraobs-integrations?

Related issues

• Relates Azure Event Hub Input Package #7821

Screenshots

[zmoog]

cc @lalit-satapathy for the ownership topic in the reviewer's checklist

[Kavindu-Dodan]

nit - should we add observability here. AWS logs do have this ¹

Footnotes

1. https://github.com/elastic/integrations/blob/main/packages/aws_logs/manifest.yml#L7-L10 ↩

[zmoog]

Good point! Adding it.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: de5fd1a

History

• 💚 Build #17751 succeeded 1f2a92e
• 💚 Build #17733 succeeded 8a3a010
• 💚 Build #17723 succeeded 86962f7
• 💚 Build #17690 succeeded c11a733
• 💚 Build #17682 succeeded 234f06e
• 💚 Build #17662 succeeded 5718f2c

cc @zmoog

[elastic-sonarqube]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube