Skip to content

Commit

Permalink
Merge pull request #18736 from newrelic/rhs-cardinality-ui
Browse files Browse the repository at this point in the history
data-and-apis: Add cardinality UI page
  • Loading branch information
rhetoric101 authored Sep 30, 2024
2 parents 96c6c96 + 5624dc1 commit c41f6e6
Show file tree
Hide file tree
Showing 9 changed files with 161 additions and 2 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
---
title: 'Cardinality management'
metaDescription: 'You can manage your data cardinality from the New Relic UI.'
freshnessValidatedDate: 2024-09-24
---

<Callout title="preview">
We're still working on this feature, but we'd love for you to try it out!

This feature is currently provided as part of a preview program pursuant to our [pre-release policies](/docs/licenses/license-information/referenced-policies/new-relic-pre-release-policy).
</Callout>

As an administrator at New Relic, one of your tasks is to make sure you don't go beyond your limit of [GB Ingested](/docs/licenses/license-information/product-definitions/new-relic-one-pricing-definitions/#gb-ingested). One tool for tracking your ingest is the Cardinality Management UI. You'll find helpful graphs and tables explaining your ingest so you can take action if you're approaching limits.

## Where can you manage cardinality? [#where-manage]

In the New Relic UI, you can find this with other administration tools. Click your name in the lower left and go to **Administration > Cardinality Management**.

## Account Cardinality Breakdown [#account-breakdown]

If you're in an organization with multiple accounts, you can see a breakdown of cardinality by each account. As with most capabilities within New Relic, you can change the account you are currently viewing by selecting the account picker in the top right corner. The UI will display information about the selected account, as long as you have access to that account.

### Usage over time [#usage-over-time]

On the opening page of **Cardinality Management**, you can view the **Usage over time** chart:

<img
style={{ align: 'left',maxWidth: '90%' }}
title="Screenshot showing the usage over time chart"
alt="Screenshot showing the usage over time chart"
src="/images/apis-and-data_screenshot-crop_usage-over-time.webp"
/>

This shows following:

* The limit that is currently at 5 million account cardinality.
* The current cardinality usage for the account, which is in the 2 to 4 million range. If there's no change in the value, then a new data point isn't reported. This can appear as an empty data point, but it's not data loss. This behavior may change as the feature is finalized.


<Callout variant="tip">
You can modify this timeseries by changing the time-picker in the top right corner. Also, the time-picker only affects the timeseries and does not affect the **Metrics** table under the timeseries.
</Callout>

### Metrics table [#metrics-table]

In the **Metrics** table, you can see the breakdown of account cardinality by metric name:

<img
style={{ align: 'left',maxWidth: '90%' }}
title="Screenshot showing the metrics table"
alt="Screenshot showing the metrics table"
src="/images/apis-and-data_screenshot-crop_metrics-table.webp"
/>

This table shows the top 20 highest cardinality metrics in an account. The table provides the following details:
* The name of the metric
* The current overall usage of the metric (the ratio of current usage to current limit)
* The actual values of the current cardinality
* The current limit of that metric name

<Callout variant="tip">
All metrics have a default metric limit of 100,000.
</Callout>

## Metric cardinality breakdown [#metric-cardinality-breakdown]

You can view a deeper layer of granularity in your metrics by clicking on any metric name. You can view a breakdown of the specific metric's cardinality details.

<img
style={{ align: 'left',maxWidth: '90%' }}
title="Screenshot showing the Cardinality Management page"
alt="Screenshot showing the Cardinality Management page"
src="/images/apis-and-data_screenshot-crop_metric-cardinality-breakdown.webp"
/>

### Metric and account cardinality bar charts [#bar-charts]

In this metric-specific view, you can see the current cardinality usage as two bar charts, one for **Metric cardinality budget** and one for **Account cardinality budget** (the overall account usage).

## Usage over time (for metrics) [#usage-over-time-metrics]

Below the bar charts is a chart of a timeseries query showing the metric cardinality usage versus the limit of that metric. Again, you can change the time-picker to view different time periods of this individual metric's cardinality data.

<Callout variant="tip">
For any metric that has exceeded its cardinality limit (usually 100,000) New Relic will still attempt to populate information on the metric, but it will be slightly slower since it uses RAW data instead of using aggregated data.
</Callout>

### Attributes table [#attributes-table]

Under the timeseries, a table shows the attributes that are powering the metric cardinality. Note that the numbers usually don’t add up to 100% because cardinality is calculated as a combination of several attributes. However, it is a good indicator of what your highest contributing attributes are.

<img
style={{ align: 'left',maxWidth: '90%' }}
title="Screenshot showing attributes behind metrics"
alt="Screenshot showing attributes behind metrics"
src="/images/apis-and-data_screenshot-crop_metric-attributes.webp"
/>

This information is only for the last day (since UTC midnight), which is when cardinality resets (both account and metric).

You can also view specific information about an attribute, which can help you make decisions about whether or not that attribute is useful. You can see a sample of the attribute values by clicking the menu on the right side of the attribute row.


## Account pruning rules [#account-pruning-rules]

A common way to keep account cardinality under its limit is to implement pruning rules for metrics and attributes you don't consider important.

### View account pruning rules [#view-pruning-rules]

You can view pruning rules by clicking into a metric and then clicking **View Pruning Rules**. This will bring up a scrollable list of all pruning rules associated with an account. You can also delete pruning rules associated with an account in this view:

<img
style={{ align: 'left',maxWidth: '90%' }}
title="Screenshot showing the link to pruning rules on the Attributes page"
alt="Screenshot showing the link to pruning rules on the Attributes page"
src="/images/apis-and-data_screenshot-crop_view-pruning-rules.webp"
/>


### Create a pruning rule [#create-pruning-rule]

You can create a pruning rule based on attributes within the UI. To create a pruning rule:

1. Select the checkboxes next to the attributes you want to prune.
<img
style={{ align: 'left',maxWidth: '90%' }}
title="Screenshot showing how to select attributes for pruning rule"
alt="Screenshot showing how to select attributes for pruning rule"
src="/images/apis-and-data_screenshot-crop_select-attributes-for-pruning-rule.webp"
/>
2. Click **Create pruning rule**.
3. In the pop-up with information about how the pruning rule affects the metric's cardinality, enter a rule name or description.
<img
style={{ align: 'left' }}
title="Screenshot of page to name the pruning rule"
alt="Screenshot of page to name the pruning rule"
src="/images/apis-and-data_screenshot-crop_name-pruning-rule.webp"
/>
4. Click **Submit** to generate the pruning rule.

It's important to note that pruning rules are instantaneous. The information will be pruned instantly after creating the rule, so make sure you really want to prune the data.

If you create a pruning rule, you can still delete the pruning rule after it's made, but before it's deleted, the rule may prune a little bit of data.

### Delete a pruning rule [#delete-pruning-rule]

To delete a pruning rule:

1. Click **View All Pruning Rules**.
2. Scroll to the relevant pruning rule, and then click **Delete**.

## What's next [#whats-next]

If you want to learn more about high cardinality, see [Understand and query high cardinality metrics](/docs/data-apis/ingest-apis/metric-api/NRQL-high-cardinality-metrics/).

8 changes: 6 additions & 2 deletions src/nav/telemetry-data-platform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,12 @@ pages:
path: /docs/data-apis/ingest-apis/metric-api/metric-api-limits-restricted-attributes
- title: 'Troubleshoot: Metric API'
path: /docs/data-apis/ingest-apis/metric-api/troubleshoot-nrintegrationerror-events
- title: Understand and query high cardinality metrics
path: /docs/data-apis/ingest-apis/metric-api/NRQL-high-cardinality-metrics
- title: Cardinality
pages:
- title: Cardinality management
path: /docs/data-apis/ingest-apis/metric-api/cardinality-management
- title: Understand and query high cardinality metrics
path: /docs/data-apis/ingest-apis/metric-api/NRQL-high-cardinality-metrics
- title: Security data API
path: /docs/data-apis/ingest-apis/security-data-api
- title: Telemetry SDKs
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

0 comments on commit c41f6e6

Please sign in to comment.