Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logsdb telemetry #115994

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add logsdb telemetry #115994

wants to merge 5 commits into from

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Oct 31, 2024

This PR adds telemetry for logsdb. However, this change only tracks the count of indices using logsdb and those that use synthetic source. Additional stats, such as shard, indexing, and search stats, will be added in a follow-up, as they require reaching out to data nodes.

Copy link
Contributor

Documentation preview:

@dnhatn dnhatn changed the title Logsdb usage Add logsdb telemetry Oct 31, 2024
@dnhatn dnhatn marked this pull request as ready for review October 31, 2024 04:32
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine
Copy link
Collaborator

Hi @dnhatn, I've created a changelog YAML for you.

@dnhatn dnhatn added the auto-backport Automatically create backport pull requests when merged label Oct 31, 2024
Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I think we also need to update the mapping the telemetry repo. Similar to this elastic/telemetry#3415

}

@Override
protected void masterOperation(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this is the best place for this, but can we maybe in a follow up also track the number of indices without index mode that use synthetic source, the number of tsdb indices and how many of those tsdb indices use synthetic source? If you think this is a good idea. Maybe we should in this change rename the indices_count and indices_with_synthetic_source fields in the response to be something like logsdb_index_count, logsdb_with_synthetic_source_index_count?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I considered this as well, but these fields don't seem to fit within logsdb. I think we'll need to add two separate usages: one for time_series and another for source_mode. I’ll add them in follow-up PRs.

{
	"logsdb": {
		"enabled": true,
		"available": true,
		"indices_count": 1,
		"indices_with_synthetic_source": 1
	}
}

ActionListener<XPackUsageFeatureResponse> listener
) {
int numIndices = 0;
int numSyntheticSources = 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe rename to numLogsdbIndices and numLogsdbIndicesWithSynthticSource?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed 5dc58e6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged >enhancement :StorageEngine/Logs You know, for Logs Team:StorageEngine v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants