From f9fd89b1470ae2a5b4ba3d5f3a0170a1dc28ea0b Mon Sep 17 00:00:00 2001 From: Dani Vansia <41837012+vansia43@users.noreply.github.com> Date: Tue, 8 Oct 2024 13:41:24 -0400 Subject: [PATCH 1/4] Add Resource Reports VQL docs --- docs/vql.md | 354 +---------------------------------- docs/vql_cost_report.md | 362 ++++++++++++++++++++++++++++++++++++ docs/vql_resource_report.md | 293 +++++++++++++++++++++++++++++ sidebars.js | 13 +- 4 files changed, 673 insertions(+), 349 deletions(-) create mode 100644 docs/vql_cost_report.md create mode 100644 docs/vql_resource_report.md diff --git a/docs/vql.md b/docs/vql.md index 8aa9624c..1889c1b9 100644 --- a/docs/vql.md +++ b/docs/vql.md @@ -15,10 +15,10 @@ import TabItem from '@theme/TabItem'; The Vantage Query Language (VQL) is a SQL-like language for filtering cloud cost data. It includes a normalized schema across cloud providers and basic filter syntax for creating complex filters. VQL is currently available as part of the [Vantage API](https://vantage.readme.io/reference/general) as well as within the [Vantage Terraform provider](/terraform). -- **API**. On supported endpoints, such as `/cost_reports`, you can use VQL in the `filter` parameter to programmatically create and fetch Cost Reports. +- **API**. On supported endpoints, such as `/cost_reports`, you can use VQL in the `filter` parameter to programmatically create and fetch Cost Reports. There are two VQL scopes currently available for Cost Reports and Resource Reports. - **Terraform provider**. In supported Terraform resources, such as `vantage_saved_filter` or `vantage_cost_report`, you can use VQL for the `filter` argument to create filtered Cost Reports or saved filters. -The below examples show how to use VQL on both the API as well as on the Terraform provider. This example creates a [saved filter](/cost_reports#saved-filters) in your Vantage account using VQL. +The below examples show how to use VQL on both the API and on the Terraform provider. This example creates a [saved filter](/cost_reports#saved-filters) in your Vantage account using VQL. @@ -79,351 +79,9 @@ VQL syntax closely resembles the `WHERE` clause of a SQL query. You can construc --- -### Schema +## Scopes -VQL comprises two namespaces, `costs` and `tags`, which represent the available [filters](/cost_reports#filtering-cost-reports) on Cost Reports in the Vantage console. To reference a filter, use the following syntax: `namespace.field` (e.g., `costs.provider` or `tags.name`). +VQL has a separate scope for each type of report—meaning that you cannot use VQL statements for Cost Reports when querying Resource Reports. See the following pages below to learn more about each VQL scope. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NamespaceFieldVQL Example
costsproviderProviders example
allocationCost allocation example
regionRegion example
marketplaceMarketplace example
account_idAccount ID example
provider_account_idProvider account ID example
serviceService example
categoryCategory example
subcategorySubcategory example
resource_idResource example
charge_typeCharge Type example
tagsname
value
- -:::note -Availability of the fields listed above varies among different cloud providers. For a comprehensive list of available fields per provider, see the [Data Dictionary](/data_dictionary). -::: - -### Keywords - -VQL includes a set of keywords to create complex filter conditions. These keywords function similar to their SQL equivalents. - -| Keyword | Description | VQL Sample | Explanation | -| ------- | ------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `AND` | Logical AND operator | `costs.provider = 'aws' AND costs.service = 'EC2'` | This example filters AWS costs for the EC2 service, where both conditions must be true. | -| `OR` | Logical OR operator | `costs.provider = 'azure' OR costs.provider = 'aws'` | This example retrieves costs from either Azure or AWS. At least one condition must be true. | -| `IN` | Used to compare against an array list | `costs.provider = 'azure' AND costs.account_id IN ('account-1', 'account-2')` | This example filters based on a list of account IDs, returning data for the specified accounts

You can also use `IN` along with a special syntax for filtering by multiple tags. See [Filter by Multiple Tags](/vql#multiple-tags) for details. | -| `LIKE` | Performs string comparisons | `costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value LIKE '%prod%'` | This example selects data where the tag value contains `prod`, such as `production-1`.
Note that at this time, `LIKE` is not compatible with `costs.account_id`, `costs.provider_account_id`, `costs.region`, and `costs.service`. | -| `NOT` | Represents negation | `costs.provider = 'aws' AND costs.region NOT IN ('us-east-1', 'us-east-2')` | This example filters out data from both specified regions, providing all AWS costs _not_ in these regions. Use `NOT IN` to specify a list of single or multiple values.

You can also use the `!=` or `<>` operators for "is not."

`costs.provider = 'aws' AND costs.region != 'us-east-1'`

You can use `NOT LIKE` to perform string comparisons:

`costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value NOT LIKE '%prod%'` | - -With these keywords, you can construct complex filter conditions in VQL, providing flexibility and precision when querying and analyzing cloud cost data. - -### Syntax - -You can think of VQL in its current iteration as the `WHERE` clause of a SQL query. By combining the schema and keywords above with parentheses, you can form complex filter operations, such as: - -```sql -costs.provider = 'mongo' AND costs.allocation = 1.0 AND (costs.service = 'REALM' AND costs.resource_id IN ('s3')) OR (costs.provider = 'aws' AND costs.allocation = 1.0 AND costs.account_id IN ('123456798')) -``` - -## VQL Examples {#examples} - -The following examples cover common use cases for VQL. - -### Combining Providers {#providers} - -:::note -To reference a [Custom Provider](/connecting_custom_providers) in VQL queries, navigate to the [**Integrations** page](https://console.vantage.sh/settings/custom_providers). Use the displayed **Provider ID** (e.g., `custom_provider:accss_crdntl_123a45bfdaf38765`). -::: - -Filter for provider costs associated with either MongoDB Atlas or AWS. - -```sql -costs.provider = 'mongo' OR costs.provider = 'aws' -``` - -### Cost Allocation {#cost-allocation} - -Set cost allocation to `0.5`. - -```sql -costs.provider = 'gcp' AND costs.allocation = 0.5 -``` - -### Costs from a List of Regions {#region} - -Filter for Snowflake costs in two regions. Note that you will need to use the region code, such as `us-east-1` in the case of AWS, or `AWS_US_EAST_1` in the case of Snowflake, below. - -```sql -costs.provider = 'snowflake' AND costs.region IN ('AWS_US_EAST_1', 'AWS_US_EAST_2') -``` - -### Get Marketplace Transactions {#marketplace} - -Retrieve costs associated with the AWS Marketplace. - -```sql -costs.provider = 'aws' AND costs.marketplace = true -``` - -### Costs by Account ID {#account} - -Costs for a specific set of services and account ID. - -```sql -costs.provider = 'aws' AND costs.account_id = '123456758' AND costs.service IN ('Amazon Relational Database', 'Amazon Elastic Compute Cloud - Compute') -``` - -### Costs by Provider Account ID {#provider-account} - -The following example represents costs from a specific AWS billing account or costs from a specific Azure subscription. - -```sql -(costs.provider = 'aws' AND costs.provider_account_id = 'abcd1234') OR (costs.provider = 'azure' AND costs.provider_account_id = 'abcd1234') -``` - -### Per-Resource Costs and Costs by Service {#resource} - -Resource costs require both `provider` and `service` in addition to the `resource_id`. - -```sql -costs.provider = 'aws' AND costs.service = 'Amazon Relational Database Service' AND costs.resource_id = 'arn:aws:rds:us-east-1:123456789:db:primary-01' -``` - -#### Multiple Resource IDs - -```sql -costs.provider = 'aws' AND costs.service = 'Amazon Relational Database Service' AND costs.resource_id IN ('arn:aws:rds:us-east-1:123456789:db:primary-01', 'arn:aws:rds:us-east-1:123456789:db:primary-02') -``` - -### Costs by Specific Category {#category} - -Filter costs to see a specific cost category. Category costs require both `provider` and `service` as well as `category`. - -```sql -costs.provider = 'fastly' AND costs.service = 'CDN' AND costs.category = 'Data Transfer' -``` - -### Costs by Specific Subcategory {#subcategory} - -Filter costs by a specific service and subcategory. Subcategory costs require both `provider` and `service` as well as `subcategory`. - -```sql -costs.provider = 'aws' AND costs.service = 'AWS Certificate Manager' AND costs.subcategory = 'USE1-PaidPrivateCA' -``` - -### Cost by Charge Type {#charge-type} - -Filter costs by a specific charge type. - -```sql -costs.provider = 'aws' AND costs.charge_type = 'Usage' -``` - -### Filter by Tag {#tags} - -#### Filter by Single Tag - -Filter costs based on a specific tag, such as `environment`, with the value `production`, in AWS. - -```sql -costs.provider = 'aws' AND tags.name = 'environment' AND tags.value = 'production' -``` - -#### Filter for Multiple Values from a Single Tag Key - -If you want to filter for multiple tag values that are tied to one tag key (e.g., costs tagged with the `team` tag of `mobile` and `data`), use the below syntax. - -```sql -costs.provider = 'aws' AND tags.name = 'team' AND tags.value IN ('mobile', 'data') -``` - -#### Filter by Multiple Tags {#multiple-tags} - -If you want to filter for resources that have more than one tag associated, you can use the syntax shown in the example below. - -```sql -costs.provider = 'aws' AND (tags.name, tags.value) IN (('environment', 'staging'), ('team', 'engineering')) -``` - -This example filters for resources that are tagged with the `environment` tag with a value of `staging` as well as the `team` tag with a value of `engineering`. This filter is the same as creating the following manual filter in the console. - -
- Filter by multiple tags in the console -
- -#### Filter for Matching Tags Using `LIKE` - -```sql -costs.provider = 'azure' AND tags.name = 'environment' AND tags.value LIKE '%prod%' -``` - -### Filter for Untagged Resources {#untagged} - -On providers that have a **Not Tagged**/**Not Labeled** filter option in the console, you can use the below VQL to see untagged resources. This example looks for untagged resources in a multi-cloud environment. - -```sql -(costs.provider = 'aws' AND tags.name = NULL) OR (costs.provider = 'azure' AND tags.name = NULL) OR (costs.provider = 'gcp' AND tags.name = NULL) -``` - -## Troubleshooting - -If you are receiving an error when trying to complete a query, check the following troubleshooting tips below. - -- Each provider exposes certain names to the API. Those names are normalized within the schema. Check the [Data Dictionary](/data_dictionary) for normalized field names. -- Query parameter values should be wrapped in single quotes. -
Click to view examples - - :::tip This works - - ```sql - costs.provider='aws' - ``` - - ::: - :::caution This does not work - - ```sql - costs.provider="aws" - ``` - - ::: - -
- -- Currently, there is a limitation where `AND` and `OR` are not supported together in a single "query group." -
Click to view examples - - :::tip This works - - ```sql - (costs.provider = 'aws' AND tags.name = 'environment' AND tags.value = 'dev') OR (costs.provider = 'aws' AND tags.name = 'environment' AND tags.value = 'prod') - ``` - - ::: - :::caution This does not work - - ```sql - costs.provider = 'aws' AND ((tags.name = 'environment' AND tags.value = 'dev') OR (tags.name = 'environment' AND tags.value = 'prod')) - ``` - - ::: - -
- -- The `costs.provider` field is required on every call. -
Click to view examples - - :::tip This works - - ```sql - costs.provider = 'fastly' AND costs.service = 'CDN' - ``` - - ::: - :::caution This does not work - - ```sql - costs.service = 'CDN' - ``` - - ::: - -
- -- Resource costs require both provider and service in addition to the resource ID. -
Click to view examples - - :::tip This works - - ```sql - costs.provider = 'aws' AND costs.service = 'Amazon Relational Database Service' AND costs.resource_id = 'arn:aws:rds:us-east-1:123456789:db:primary-01' - ``` - - ::: - :::caution This does not work - - ```sql - costs.provider = 'aws' AND costs.resource_id = 'arn:aws:rds:us-east-1:123456789:db:primary-01' - ``` - - ::: - -
- -- Category and subcategory costs also require provider and service. -
Click to view examples - - :::tip These work - - ```sql - costs.provider = 'fastly' AND costs.service = 'CDN' AND costs.category = 'Data Transfer' - ``` - - ```sql - costs.provider = 'aws' AND costs.service = 'AWS Certificate Manager' AND costs.subcategory = 'USE1-PaidPrivateCA' - ``` - - ::: - - :::caution These do not work - - ```sql - costs.provider = 'fastly' AND costs.category = 'Data Transfer' - ``` - - ```sql - costs.provider = 'aws' AND costs.subcategory = 'USE1-PaidPrivateCA' - ``` - - ::: - -
+- [VQL for Cost Reports](/vql_cost_reports) +- [VQL for Resource Reports](/vql_resource_reports) diff --git a/docs/vql_cost_report.md b/docs/vql_cost_report.md new file mode 100644 index 00000000..4c23209c --- /dev/null +++ b/docs/vql_cost_report.md @@ -0,0 +1,362 @@ +--- +id: vql_cost_report +title: VQL for Cost Reports +description: This page walks through the Vantage Query Language, or VQL, for use within the Cost Reports scope. +keywords: + - VQL + - Vantage Query Language + - Terraform +--- + +# VQL for Cost Reports + +This page describes how to use VQL when querying Cost Reports in API or using the Terraform Provider. + +## Schema + +VQL comprises two namespaces, `costs` and `tags`, which represent the available [filters](/cost_reports#filtering-cost-reports) on Cost Reports in the Vantage console. To reference a filter, use the following syntax: `namespace.field` (e.g., `costs.provider` or `tags.name`). + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NamespaceFieldVQL Example
costsproviderProviders example
allocationCost allocation example
regionRegion example
marketplaceMarketplace example
account_idAccount ID example
provider_account_idProvider account ID example
serviceService example
categoryCategory example
subcategorySubcategory example
resource_idResource example
charge_typeCharge Type example
tagsname
value
+ +:::note +Availability of the fields listed above varies among different cloud providers. For a comprehensive list of available fields per provider, see the [Data Dictionary](/data_dictionary). +::: + +## Keywords + +VQL includes a set of keywords to create complex filter conditions. These keywords function similar to their SQL equivalents. + +| Keyword | Description | VQL Sample | Explanation | +| ------- | ------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `AND` | Logical AND operator | `costs.provider = 'aws' AND costs.service = 'EC2'` | This example filters AWS costs for the EC2 service, where both conditions must be true. | +| `OR` | Logical OR operator | `costs.provider = 'azure' OR costs.provider = 'aws'` | This example retrieves costs from either Azure or AWS. At least one condition must be true. | +| `IN` | Used to compare against an array list | `costs.provider = 'azure' AND costs.account_id IN ('account-1', 'account-2')` | This example filters based on a list of account IDs, returning data for the specified accounts

You can also use `IN` along with a special syntax for filtering by multiple tags. See [Filter by Multiple Tags](/vql#multiple-tags) for details. | +| `LIKE` | Performs string comparisons | `costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value LIKE '%prod%'` | This example selects data where the tag value contains `prod`, such as `production-1`.
Note that at this time, `LIKE` is not compatible with `costs.account_id`, `costs.provider_account_id`, `costs.region`, and `costs.service`. | +| `NOT` | Represents negation | `costs.provider = 'aws' AND costs.region NOT IN ('us-east-1', 'us-east-2')` | This example filters out data from both specified regions, providing all AWS costs _not_ in these regions. Use `NOT IN` to specify a list of single or multiple values.

You can also use the `!=` or `<>` operators for "is not."

`costs.provider = 'aws' AND costs.region != 'us-east-1'`

You can use `NOT LIKE` to perform string comparisons:

`costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value NOT LIKE '%prod%'` | + +With these keywords, you can construct complex filter conditions in VQL, providing flexibility and precision when querying and analyzing cloud cost data. + +## Syntax + +You can think of VQL in its current iteration as the `WHERE` clause of a SQL query. By combining the schema and keywords above with parentheses, you can form complex filter operations, such as: + +```sql +costs.provider = 'mongo' AND costs.allocation = 1.0 AND (costs.service = 'REALM' AND costs.resource_id IN ('s3')) OR (costs.provider = 'aws' AND costs.allocation = 1.0 AND costs.account_id IN ('123456798')) +``` + +## VQL Examples {#examples} + +The following examples cover common use cases for VQL. + +### Combining Providers {#providers} + +:::note +To reference a [Custom Provider](/connecting_custom_providers) in VQL queries, navigate to the [**Integrations** page](https://console.vantage.sh/settings/custom_providers). Use the displayed **Provider ID** (e.g., `custom_provider:accss_crdntl_123a45bfdaf38765`). +::: + +Filter for provider costs associated with either MongoDB Atlas or AWS. + +```sql +costs.provider = 'mongo' OR costs.provider = 'aws' +``` + +### Cost Allocation {#cost-allocation} + +Set cost allocation to `0.5`. + +```sql +costs.provider = 'gcp' AND costs.allocation = 0.5 +``` + +### Costs from a List of Regions {#region} + +Filter for Snowflake costs in two regions. Note that you will need to use the region code, such as `us-east-1` in the case of AWS, or `AWS_US_EAST_1` in the case of Snowflake, below. + +```sql +costs.provider = 'snowflake' AND costs.region IN ('AWS_US_EAST_1', 'AWS_US_EAST_2') +``` + +### Get Marketplace Transactions {#marketplace} + +Retrieve costs associated with the AWS Marketplace. + +```sql +costs.provider = 'aws' AND costs.marketplace = true +``` + +### Costs by Account ID {#account} + +Costs for a specific set of services and account ID. + +```sql +costs.provider = 'aws' AND costs.account_id = '123456758' AND costs.service IN ('Amazon Relational Database', 'Amazon Elastic Compute Cloud - Compute') +``` + +### Costs by Provider Account ID {#provider-account} + +The following example represents costs from a specific AWS billing account or costs from a specific Azure subscription. + +```sql +(costs.provider = 'aws' AND costs.provider_account_id = 'abcd1234') OR (costs.provider = 'azure' AND costs.provider_account_id = 'abcd1234') +``` + +### Per-Resource Costs and Costs by Service {#resource} + +Resource costs require both `provider` and `service` in addition to the `resource_id`. + +```sql +costs.provider = 'aws' AND costs.service = 'Amazon Relational Database Service' AND costs.resource_id = 'arn:aws:rds:us-east-1:123456789:db:primary-01' +``` + +#### Multiple Resource IDs + +```sql +costs.provider = 'aws' AND costs.service = 'Amazon Relational Database Service' AND costs.resource_id IN ('arn:aws:rds:us-east-1:123456789:db:primary-01', 'arn:aws:rds:us-east-1:123456789:db:primary-02') +``` + +### Costs by Specific Category {#category} + +Filter costs to see a specific cost category. Category costs require both `provider` and `service` as well as `category`. + +```sql +costs.provider = 'fastly' AND costs.service = 'CDN' AND costs.category = 'Data Transfer' +``` + +### Costs by Specific Subcategory {#subcategory} + +Filter costs by a specific service and subcategory. Subcategory costs require both `provider` and `service` as well as `subcategory`. + +```sql +costs.provider = 'aws' AND costs.service = 'AWS Certificate Manager' AND costs.subcategory = 'USE1-PaidPrivateCA' +``` + +### Cost by Charge Type {#charge-type} + +Filter costs by a specific charge type. + +```sql +costs.provider = 'aws' AND costs.charge_type = 'Usage' +``` + +### Filter by Tag {#tags} + +#### Filter by Single Tag + +Filter costs based on a specific tag, such as `environment`, with the value `production`, in AWS. + +```sql +costs.provider = 'aws' AND tags.name = 'environment' AND tags.value = 'production' +``` + +#### Filter for Multiple Values from a Single Tag Key + +If you want to filter for multiple tag values that are tied to one tag key (e.g., costs tagged with the `team` tag of `mobile` and `data`), use the below syntax. + +```sql +costs.provider = 'aws' AND tags.name = 'team' AND tags.value IN ('mobile', 'data') +``` + +#### Filter by Multiple Tags {#multiple-tags} + +If you want to filter for resources that have more than one tag associated, you can use the syntax shown in the example below. + +```sql +costs.provider = 'aws' AND (tags.name, tags.value) IN (('environment', 'staging'), ('team', 'engineering')) +``` + +This example filters for resources that are tagged with the `environment` tag with a value of `staging` as well as the `team` tag with a value of `engineering`. This filter is the same as creating the following manual filter in the console. + +
+ Filter by multiple tags in the console +
+ +#### Filter for Matching Tags Using `LIKE` + +```sql +costs.provider = 'azure' AND tags.name = 'environment' AND tags.value LIKE '%prod%' +``` + +### Filter for Untagged Resources {#untagged} + +On providers that have a **Not Tagged**/**Not Labeled** filter option in the console, you can use the below VQL to see untagged resources. This example looks for untagged resources in a multi-cloud environment. + +```sql +(costs.provider = 'aws' AND tags.name = NULL) OR (costs.provider = 'azure' AND tags.name = NULL) OR (costs.provider = 'gcp' AND tags.name = NULL) +``` + +## Troubleshooting + +If you are receiving an error when trying to complete a query, check the following troubleshooting tips below. + +- Each provider exposes certain names to the API. Those names are normalized within the schema. Check the [Data Dictionary](/data_dictionary) for normalized field names. +- Query parameter values should be wrapped in single quotes. +
Click to view examples + + :::tip This works + + ```sql + costs.provider='aws' + ``` + + ::: + :::caution This does not work + + ```sql + costs.provider="aws" + ``` + + ::: + +
+ +- Currently, there is a limitation where `AND` and `OR` are not supported together in a single "query group." +
Click to view examples + + :::tip This works + + ```sql + (costs.provider = 'aws' AND tags.name = 'environment' AND tags.value = 'dev') OR (costs.provider = 'aws' AND tags.name = 'environment' AND tags.value = 'prod') + ``` + + ::: + :::caution This does not work + + ```sql + costs.provider = 'aws' AND ((tags.name = 'environment' AND tags.value = 'dev') OR (tags.name = 'environment' AND tags.value = 'prod')) + ``` + + ::: + +
+ +- The `costs.provider` field is required on every call. +
Click to view examples + + :::tip This works + + ```sql + costs.provider = 'fastly' AND costs.service = 'CDN' + ``` + + ::: + :::caution This does not work + + ```sql + costs.service = 'CDN' + ``` + + ::: + +
+ +- Resource costs require both provider and service in addition to the resource ID. +
Click to view examples + + :::tip This works + + ```sql + costs.provider = 'aws' AND costs.service = 'Amazon Relational Database Service' AND costs.resource_id = 'arn:aws:rds:us-east-1:123456789:db:primary-01' + ``` + + ::: + :::caution This does not work + + ```sql + costs.provider = 'aws' AND costs.resource_id = 'arn:aws:rds:us-east-1:123456789:db:primary-01' + ``` + + ::: + +
+ +- Category and subcategory costs also require provider and service. +
Click to view examples + + :::tip These work + + ```sql + costs.provider = 'fastly' AND costs.service = 'CDN' AND costs.category = 'Data Transfer' + ``` + + ```sql + costs.provider = 'aws' AND costs.service = 'AWS Certificate Manager' AND costs.subcategory = 'USE1-PaidPrivateCA' + ``` + + ::: + + :::caution These do not work + + ```sql + costs.provider = 'fastly' AND costs.category = 'Data Transfer' + ``` + + ```sql + costs.provider = 'aws' AND costs.subcategory = 'USE1-PaidPrivateCA' + ``` + + ::: + +
diff --git a/docs/vql_resource_report.md b/docs/vql_resource_report.md new file mode 100644 index 00000000..e0e5b0d7 --- /dev/null +++ b/docs/vql_resource_report.md @@ -0,0 +1,293 @@ +--- +id: vql_resource_report +title: VQL for Resource Reports +description: This page walks through the Vantage Query Language, or VQL, for use in Resource Reports. +keywords: + - VQL + - Vantage Query Language + - Terraform +--- + +# VQL for Resource Reports + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +## Resource Reports VQL Schema + +VQL for Resources Reports comprises two namespaces: `resources` and `tags`, which represent the available [filters](/active_resources#create-a-resource-report) on Resource Reports in the Vantage console. To reference a filter, use the following syntax: `namespace.field` (e.g., `resources.region` or `tags.name`). The following fields are available within these namespaces. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NamespaceFieldVQL Example
resourcesresources.providerProviders example
resources.regionRegion example
resources.account_idAccount example
resources.provider_account_idBilling Account example
resources.typeResource Type example
resources.labelLabel example
resources.uuidUUID (AWS ARN) example
resources.metadataCategory example
tagsname
value
+ +:::note +Availability of the fields listed above varies among different cloud providers. See the [Resource Reports documentation](/active_resources#resource-report-filters) for a full list of filter fields available per provider. +::: + +### Keywords + +VQL includes a set of keywords to create complex filter conditions. These keywords function similar to their SQL equivalents. + +| Keyword | Description | VQL Sample | Explanation | +| ------- | ------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `AND` | Logical AND operator | `costs.provider = 'aws' AND costs.service = 'EC2'` | This example filters AWS costs for the EC2 service, where both conditions must be true. | +| `OR` | Logical OR operator | `costs.provider = 'azure' OR costs.provider = 'aws'` | This example retrieves costs from either Azure or AWS. At least one condition must be true. | +| `IN` | Used to compare against an array list | `costs.provider = 'azure' AND costs.account_id IN ('account-1', 'account-2')` | This example filters based on a list of account IDs, returning data for the specified accounts

You can also use `IN` along with a special syntax for filtering by multiple tags. See [Filter by Multiple Tags](/vql#multiple-tags) for details. | +| `LIKE` | Performs string comparisons | `costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value LIKE '%prod%'` | This example selects data where the tag value contains `prod`, such as `production-1`.
Note that at this time, `LIKE` is not compatible with `costs.account_id`, `costs.provider_account_id`, `costs.region`, and `costs.service`. | +| `NOT` | Represents negation | `costs.provider = 'aws' AND costs.region NOT IN ('us-east-1', 'us-east-2')` | This example filters out data from both specified regions, providing all AWS costs _not_ in these regions. Use `NOT IN` to specify a list of single or multiple values.

You can also use the `!=` or `<>` operators for "is not."

`costs.provider = 'aws' AND costs.region != 'us-east-1'`

You can use `NOT LIKE` to perform string comparisons:

`costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value NOT LIKE '%prod%'` | + +With these keywords, you can construct complex filter conditions in VQL, providing flexibility and precision when querying and analyzing cloud cost data. + +## VQL Examples {#examples} + +The following examples cover common use cases for VQL. + +### Combining Providers {#providers} + +Filter for provider resources associated with either AWS or GCP. + +```sql +(resources.provider = 'aws') OR (resources.provider = 'gcp') +``` + +### Costs from a List of Regions {#region} + +Filter for Snowflake costs in two regions. Note that you will need to use the region code, such as `us-east-1`. + +```sql +resources.provider = 'aws' AND (resources.region = 'us-east-1' OR resources.region = 'us-west-1') +``` + +### Costs by Account ID {#account} + +Costs for a specific set of resource types and account ID. + +```sql +resources.provider = 'gcp' AND (resources.account_id = 'user-proj-1234') AND (resources.type = 'google_compute_disk' OR resources.type = 'google_compute_instance') +``` + +### Costs by Billing Account {#billing-account} + +Costs for a specific billing account. + +```sql +resources.provider = 'aws' AND (resources.provider_account_id = '11111111111') +``` + +### Costs by Resource Type {#resource-type} + +Filter costs to see a specific resource type. In the example below, the query is looking for any resource that is NOT an AWS CloudFront Distribution. Resource types are represented like `aws_cloudfront_distribution`. Expand the box below for a list of all available resource types and their VQL equivalents. + +```sql +resources.provider = 'aws' AND (resources.type != 'aws_cloudfront_distribution') +``` +
VQL +| Provider | VQL Representation | Friendly Name | +|------------|--------------------------------------|-----------------------------------------| +| AWS | aws_batch_job_definition | Batch Job Definition | +| AWS | aws_mq_broker | MQ Broker | +| AWS | aws_cloudfront_distribution | CloudFront Distribution | +| AWS | aws_cloudtrail | CloudTrail | +| AWS | aws_codebuild_project | CodeBuild Project | +| AWS | aws_codepipeline | CodePipeline | +| AWS | aws_config_config_rule | Config Rule | +| AWS | aws_ecr_repository | ECR Repository | +| AWS | aws_ecs_service | ECS Service | +| AWS | aws_docdb_cluster_instance | DocumentDB Cluster Instance | +| AWS | aws_route53_zone | Route 53 Zone | +| AWS | aws_dynamodb_table | DynamoDB Table | +| AWS | aws_ec2_reserved_instance | EC2 Reserved Instance | +| AWS | aws_eks_cluster | EKS Cluster | +| AWS | aws_elasticache_cluster | ElastiCache Cluster | +| AWS | aws_efs_file_system | EFS File System | +| AWS | aws_elasticsearch_domain | Elasticsearch Domain | +| AWS | aws_lambda_function | Lambda Function | +| AWS | aws_glacier_vault | Glacier Vault | +| AWS | aws_globalaccelerator_accelerator | Global Accelerator | +| AWS | aws_glue_job | Glue Job | +| AWS | aws_internet_gateway | Internet Gateway | +| AWS | aws_eip | Elastic IP | +| AWS | aws_msk_cluster | MSK Cluster | +| AWS | aws_kms_key | KMS Key | +| AWS | aws_lb | Load Balancer | +| AWS | aws_cloudwatch_log_group | CloudWatch Log Group | +| AWS | aws_mediaconnect_flow | MediaConnect Flow | +| AWS | aws_mediaconvert_job | MediaConvert Job | +| AWS | aws_medialive_channel | MediaLive Channel | +| AWS | aws_media_package_channel | MediaPackage Channel | +| AWS | aws_media_package_vod_asset | MediaPackage VOD Asset | +| AWS | aws_media_store_container | MediaStore Container | +| AWS | aws_media_tailor_channel | MediaTailor Channel | +| AWS | aws_media_tailor_playback_configuration | MediaTailor Playback Configuration | +| AWS | aws_nat_gateway | NAT Gateway | +| AWS | aws_network_interface | Network Interface | +| AWS | aws_outposts_outpost | Outposts Outpost | +| AWS | aws_ec2_managed_prefix_list | EC2 Managed Prefix List | +| AWS | aws_db_instance | RDS Instance | +| AWS | aws_rds_reserved_instance | RDS Reserved Instance | +| AWS | aws_db_snapshot | RDS Snapshot | +| AWS | aws_redshift_cluster | Redshift Cluster | +| AWS | aws_route53_resolver_query_log_config | Route 53 Resolver Query Log Config | +| AWS | aws_route_table | Route Table | +| AWS | aws_s3_bucket | S3 Bucket | +| AWS | aws_sagemaker_model | SageMaker Model | +| AWS | aws_savings_plan | Savings Plan | +| AWS | aws_secretsmanager_secret | Secrets Manager Secret | +| AWS | aws_sns_topic | SNS Topic | +| AWS | aws_sqs_queue | SQS Queue | +| AWS | aws_subnet | Subnet | +| AWS | aws_ecs_task_definition | ECS Task Definition | +| AWS | aws_transfer_server | Transfer Server | +| AWS | aws_ec2_transit_gateway | EC2 Transit Gateway | +| AWS | aws_instance | EC2 Instance | +| AWS | aws_instance_snapshot | EC2 Instance Snapshot | +| AWS | aws_ebs_volume | EBS Volume | +| AWS | aws_vpc | VPC | +| AWS | aws_vpc_endpoint | VPC Endpoint | +| AWS | aws_flow_log | Flow Log | +| AWS | aws_vpc_peering_connection | VPC Peering Connection | +| AWS | aws_vpn_gateway | VPN Gateway | +| AWS | aws_wafv2_web_acl | WAFv2 Web ACL | +| AWS | aws_workspaces_workspace | WorkSpaces Workspace | +| Azure | azurerm_application_gateway | Application Gateway | +| Azure | azurerm_application_insights | Application Insights | +| Azure | azurerm_app_service_plan | App Service Plan | +| Azure | azurerm_firewall | Firewall | +| Azure | azurerm_snapshot | Snapshot | +| Azure | azurerm_container_registry | Container Registry | +| Azure | azurerm_cosmosdb_account | CosmosDB Account | +| Azure | azurerm_databricks_workspace | Databricks Workspace | +| Azure | azurerm_managed_disk | Managed Disk | +| Azure | azurerm_dns_zone | DNS Zone | +| Azure | azurerm_sql_elasticpool | SQL Elastic Pool | +| Azure | azurerm_express_route_circuit | ExpressRoute Circuit | +| Azure | azurerm_lb | Load Balancer | +| Azure | azurerm_log_analytics_workspace | Log Analytics Workspace | +| Azure | azurerm_logic_app_workflow | Logic App Workflow | +| Azure | azurerm_kubernetes_cluster | Kubernetes Cluster | +| Azure | azurerm_nat_gateway | NAT Gateway | +| Azure | azurerm_postgresql_flexible_server | PostgreSQL Flexible Server | +| Azure | azurerm_postgresql_server | PostgreSQL Server | +| Azure | azurerm_powerbi_dedicated_capacity | Power BI Dedicated Capacity | +| Azure | azurerm_private_endpoint | Private Endpoint | +| Azure | azurerm_public_ip | Public IP | +| Azure | azurerm_recovery_services_vault | Recovery Services Vault | +| Azure | azurerm_redis_cache | Redis Cache | +| Azure | azurerm_security_center_pricing | Security Center Pricing | +| Azure | azurerm_sql_database | SQL Database | +| Azure | azurerm_sql_managed_instance | SQL Managed Instance | +| Azure | azurerm_storage_account | Storage Account | +| Azure | azurerm_synapse_workspace | Synapse Workspace | +| Azure | azurerm_virtual_machine | Virtual Machine | +| Azure | azurerm_virtual_machine_scale_set | Virtual Machine Scale Set | +| Azure | azurerm_virtual_network_gateway | Virtual Network Gateway | +| Google | google_alloydb_backup | AlloyDB Backup | +| Google | google_alloydb_cluster | AlloyDB Cluster | +| Google | google_alloydb_instance | AlloyDB Instance | +| Google | google_app_engine_service | App Engine Service | +| Google | google_bigquery_dataset | BigQuery Dataset | +| Google | google_bigtable_instance | Bigtable Instance | +| Google | google_compute_disk | Compute Disk | +| Google | google_compute_instance | Compute Instance | +| Google | google_container_cluster | Container Cluster | +| Google | google_dataflow_job | Dataflow Job | +| Google | google_firestore_database | Firestore Database | +| Google | google_cloudfunctions_function | Cloud Functions Function | +| Google | google_logging_project_bucket_config | Logging Project Bucket Config | +| Google | google_redis_instance | Redis Instance | +| Google | google_cloud_run_service | Cloud Run Service | +| Google | google_secret_manager_secret | Secret Manager Secret | +| Google | google_spanner_instance | Spanner Instance | +| Google | google_sql_database_instance | SQL Database Instance | +| Google | google_storage_bucket | Storage Bucket | +
+ +### Costs by Label {#label} + +Resource costs by specific label. + +```sql +resources.provider = 'aws' AND resources.label = '123456' +``` + +### Costs for Specific ARN {#uuid} + +The UUID is the unique identifier for the resource. In the case of AWS resources, this is the ARN. The below example shows a query for resources that contain specific text within the ARN. + +```sql +resources.provider = 'aws' AND resources.uuid LIKE '%arn:aws:s3:::my-bucket%' +``` + +### Resource Metadata Costs {#metadata} + +Resource metadata costs require both `provider` and `type` as well as `metadata`. + +```sql +resources.provider = 'aws' AND (resources.type = 'aws_cloudfront_distribution' AND resources.metadata->>'domain' = 'vantage.sh') +``` + +### Filter by Tag {#tags} + +Filter resources based on a specific tag, such as `terraform`, with the value `true`, in AWS. + +```sql +resources.provider = 'aws' AND (tags.name = 'terraform' AND tags.value = 'true') +``` + +### Filter for Untagged Resources {#untagged} + +On providers that have a **Not Tagged**/**Not Labeled** filter option in the console, you can use the below VQL to see untagged resources. This example looks for untagged resources in a multi-cloud environment. + +```sql +resources.provider = 'gcp' AND tags.name = NULL) OR (resources.provider = 'aws' AND tags.name = NULL +``` \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index dbb620ff..2f979898 100644 --- a/sidebars.js +++ b/sidebars.js @@ -75,7 +75,18 @@ module.exports = { items: [ "api", "terraform", - "vql", + { + type: "category", + label: "Vantage Query Language (VQL)", + link: { + type: 'doc', + id: "vql", + }, + items: [ + "vql_cost_report", + "vql_resource_report", + ], + }, "data_dictionary", ], }, From de4537f4d0b259578bbccc02cea9d378261bdd4e Mon Sep 17 00:00:00 2001 From: Dani Vansia <41837012+vansia43@users.noreply.github.com> Date: Tue, 8 Oct 2024 15:19:28 -0400 Subject: [PATCH 2/4] Add additional edits --- docs/data_dictionary.md | 2 +- docs/vql.md | 6 ++--- docs/vql_resource_report.md | 50 ++++++++++++++++++++----------------- 3 files changed, 31 insertions(+), 27 deletions(-) diff --git a/docs/data_dictionary.md b/docs/data_dictionary.md index de9983f8..551f94e4 100644 --- a/docs/data_dictionary.md +++ b/docs/data_dictionary.md @@ -1,6 +1,6 @@ --- id: data_dictionary -title: Data Dictionary +title: Data Dictionary (Cost Reports) description: The page is a data dictionary of values used for creating VQL statements in the Vantage API and Terraform provider. keywords: - Data dictionary diff --git a/docs/vql.md b/docs/vql.md index e4eefce2..29fa0794 100644 --- a/docs/vql.md +++ b/docs/vql.md @@ -16,7 +16,7 @@ import TabItem from '@theme/TabItem'; The Vantage Query Language (VQL) is a SQL-like language for filtering cloud cost data. It includes a normalized schema across cloud providers and basic filter syntax for creating complex filters. VQL is currently available as part of the [Vantage API](https://vantage.readme.io/reference/general) as well as within the [Vantage Terraform provider](/terraform). :::tip -On [Cost Reports](/cost_reports), you can create a filter and click **View as VQL** to see the filter represented in VQL. +On Cost and Resource Reports, you can create a filter and click **View as VQL** to see the filter represented in VQL.
Show as VQL button on a Cost Report @@ -24,7 +24,7 @@ On [Cost Reports](/cost_reports), you can create a filter and click **View as VQ ::: -- **API**. On supported endpoints, such as `/cost_reports`, you can use VQL in the `filter` parameter to programmatically create and fetch Cost Reports. +- **API**. On supported endpoints, such as `/cost_reports` and `/resource_reports`, you can use VQL in the `filter` parameter to programmatically create and fetch Cost Reports. - **Terraform provider**. In supported Terraform resources, such as `vantage_saved_filter` or `vantage_cost_report`, you can use VQL for the `filter` argument to create filtered Cost Reports or saved filters. The below examples show how to use VQL on both the API and the Terraform provider. This example creates a [saved filter](/cost_reports#saved-filters) in your Vantage account using VQL. @@ -75,7 +75,7 @@ VQL uses a schema that organizes filters into namespaces. These namespaces compr -VQL includes keywords, like `AND`, `OR`, `IN`, `LIKE`, and `NOT`, to create complex and precise filter conditions. +VQL includes keywords, like `AND`, `OR`, `IN`, `LIKE`, and `NOT`, to create complex and precise filter conditions. These keywords vary per report type/scope. diff --git a/docs/vql_resource_report.md b/docs/vql_resource_report.md index e0e5b0d7..483c4460 100644 --- a/docs/vql_resource_report.md +++ b/docs/vql_resource_report.md @@ -27,37 +27,37 @@ VQL for Resources Reports comprises two namespaces: `resources` and `tags`, whic - resources + resources resources.provider Providers example - resources.region + region Region example - resources.account_id + account_id Account example - resources.provider_account_id + provider_account_id Billing Account example - resources.type + type Resource Type example - resources.label + label Label example - resources.uuid + uuid UUID (AWS ARN) example - resources.metadata - Category example + metadata + Metadata example tags @@ -71,22 +71,23 @@ VQL for Resources Reports comprises two namespaces: `resources` and `tags`, whic :::note -Availability of the fields listed above varies among different cloud providers. See the [Resource Reports documentation](/active_resources#resource-report-filters) for a full list of filter fields available per provider. +Availability of the fields listed above varies among different cloud providers. See the [Resource Reports documentation](/active_resources#resource-report-filters) for a full list of filter fields available per provider. ::: ### Keywords VQL includes a set of keywords to create complex filter conditions. These keywords function similar to their SQL equivalents. -| Keyword | Description | VQL Sample | Explanation | -| ------- | ------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `AND` | Logical AND operator | `costs.provider = 'aws' AND costs.service = 'EC2'` | This example filters AWS costs for the EC2 service, where both conditions must be true. | -| `OR` | Logical OR operator | `costs.provider = 'azure' OR costs.provider = 'aws'` | This example retrieves costs from either Azure or AWS. At least one condition must be true. | -| `IN` | Used to compare against an array list | `costs.provider = 'azure' AND costs.account_id IN ('account-1', 'account-2')` | This example filters based on a list of account IDs, returning data for the specified accounts

You can also use `IN` along with a special syntax for filtering by multiple tags. See [Filter by Multiple Tags](/vql#multiple-tags) for details. | -| `LIKE` | Performs string comparisons | `costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value LIKE '%prod%'` | This example selects data where the tag value contains `prod`, such as `production-1`.
Note that at this time, `LIKE` is not compatible with `costs.account_id`, `costs.provider_account_id`, `costs.region`, and `costs.service`. | -| `NOT` | Represents negation | `costs.provider = 'aws' AND costs.region NOT IN ('us-east-1', 'us-east-2')` | This example filters out data from both specified regions, providing all AWS costs _not_ in these regions. Use `NOT IN` to specify a list of single or multiple values.

You can also use the `!=` or `<>` operators for "is not."

`costs.provider = 'aws' AND costs.region != 'us-east-1'`

You can use `NOT LIKE` to perform string comparisons:

`costs.provider = 'gcp' AND tags.name = 'environment' AND tags.value NOT LIKE '%prod%'` | +| Keyword | Description | VQL Sample | Explanation | +| -------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `AND` | Logical AND operator | `resources.provider = 'aws' AND resources.label = '123456'` | This example filters AWS resources, with a specific associated label, where both conditions must be true. | +| `OR` | Logical OR operator | `(resources.provider = 'aws') OR (resources.provider = 'gcp')` | This example retrieves resources from either AWS or GCP. At least one condition must be true. | +| `LIKE` and `NOT LIKE` | Performs string comparisons | `resources.provider = 'aws' AND resources.uuid LIKE '%arn:aws:s3:::my-bucket%''` | This example selects data where the resource ARN contains `arn:aws:s3:::my-bucket`, such as `arn:aws:s3:::my-bucket-123`.

This same query also works for `NOT LIKE` where data does not contain a particular string `(resources.provider = 'aws' AND resources.uuid NOT LIKE '%arn:aws:s3:::my-bucket%').` | +| `!=` | Represents negation, "is not" | `resources.provider = 'azure' AND (resources.type != 'azurerm_public_ip' AND resources.type != 'azurerm_kubernetes_cluster')` | This example filters out data from two specified resource types, providing all Azure resources that are _not_ these types.| +| `<`, `>`, `<=`, `>=` | Mathematical operators for numerical queries | `resources.provider = 'azure' AND (resources.type = 'azurerm_virtual_machine' AND resources.metadata->>'virtual_machine_size' > '7')` | This example looks for Virtual Machines that have a size greater than 7. | +| `->>` | This operator is used only when constructing queries related to metadata | `resources.provider = 'aws' AND (resources.type = 'aws_instance' AND resources.metadata->>'architecture' = 'x86_64')` | This example looks for EC2 instances with an architecture of `x86_64`. | -With these keywords, you can construct complex filter conditions in VQL, providing flexibility and precision when querying and analyzing cloud cost data. +With these operators and keywords, you can construct complex filter conditions in VQL, providing flexibility and precision when querying and analyzing cloud cost data. ## VQL Examples {#examples} @@ -102,7 +103,7 @@ Filter for provider resources associated with either AWS or GCP. ### Costs from a List of Regions {#region} -Filter for Snowflake costs in two regions. Note that you will need to use the region code, such as `us-east-1`. +Filter for AWS costs in two regions. Note that you will need to use the region code, such as `us-east-1`. ```sql resources.provider = 'aws' AND (resources.region = 'us-east-1' OR resources.region = 'us-west-1') @@ -131,7 +132,9 @@ Filter costs to see a specific resource type. In the example below, the query is ```sql resources.provider = 'aws' AND (resources.type != 'aws_cloudfront_distribution') ``` -
VQL + +
Resource Type VQL Representations + | Provider | VQL Representation | Friendly Name | |------------|--------------------------------------|-----------------------------------------| | AWS | aws_batch_job_definition | Batch Job Definition | @@ -250,6 +253,7 @@ resources.provider = 'aws' AND (resources.type != 'aws_cloudfront_distribution') | Google | google_spanner_instance | Spanner Instance | | Google | google_sql_database_instance | SQL Database Instance | | Google | google_storage_bucket | Storage Bucket | +
### Costs by Label {#label} @@ -270,7 +274,7 @@ resources.provider = 'aws' AND resources.uuid LIKE '%arn:aws:s3:::my-bucket%' ### Resource Metadata Costs {#metadata} -Resource metadata costs require both `provider` and `type` as well as `metadata`. +Resource metadata costs require both `provider` and `type` as well as `metadata`. Metadata uses a specific syntax (e.g., `resources.metadata->>'domain' = 'vantage.sh'`). ```sql resources.provider = 'aws' AND (resources.type = 'aws_cloudfront_distribution' AND resources.metadata->>'domain' = 'vantage.sh') @@ -286,8 +290,8 @@ resources.provider = 'aws' AND (tags.name = 'terraform' AND tags.value = 'true') ### Filter for Untagged Resources {#untagged} -On providers that have a **Not Tagged**/**Not Labeled** filter option in the console, you can use the below VQL to see untagged resources. This example looks for untagged resources in a multi-cloud environment. +On providers that have a **Not Tagged** filter option in the console, you can use the below VQL to see untagged resources. This example looks for untagged resources in a multi-cloud environment. ```sql resources.provider = 'gcp' AND tags.name = NULL) OR (resources.provider = 'aws' AND tags.name = NULL -``` \ No newline at end of file +``` From 5d0c655663358534897572bba86a084e7968c233 Mon Sep 17 00:00:00 2001 From: Dani Vansia <41837012+vansia43@users.noreply.github.com> Date: Tue, 8 Oct 2024 15:40:40 -0400 Subject: [PATCH 3/4] Fix broken links --- docs/vql.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/vql.md b/docs/vql.md index 29fa0794..268efaff 100644 --- a/docs/vql.md +++ b/docs/vql.md @@ -92,5 +92,5 @@ VQL syntax closely resembles the `WHERE` clause of a SQL query. You can construc VQL has a separate scope for each type of report—meaning that you cannot use VQL statements for Cost Reports when querying Resource Reports. See the following pages below to learn more about each VQL scope. -- [VQL for Cost Reports](/vql_cost_reports) -- [VQL for Resource Reports](/vql_resource_reports) +- [VQL for Cost Reports](/vql_cost_report) +- [VQL for Resource Reports](/vql_resource_report) From 0b29416a37dd69007974eab476ac33859c1131eb Mon Sep 17 00:00:00 2001 From: Dani Vansia <41837012+vansia43@users.noreply.github.com> Date: Thu, 10 Oct 2024 12:09:52 -0400 Subject: [PATCH 4/4] Final updates --- docs/vql_cost_report.md | 4 ++-- docs/vql_resource_report.md | 14 ++++++++++---- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/vql_cost_report.md b/docs/vql_cost_report.md index 4c23209c..44550347 100644 --- a/docs/vql_cost_report.md +++ b/docs/vql_cost_report.md @@ -12,9 +12,9 @@ keywords: This page describes how to use VQL when querying Cost Reports in API or using the Terraform Provider. -## Schema +## Cost Reports VQL Schema -VQL comprises two namespaces, `costs` and `tags`, which represent the available [filters](/cost_reports#filtering-cost-reports) on Cost Reports in the Vantage console. To reference a filter, use the following syntax: `namespace.field` (e.g., `costs.provider` or `tags.name`). +VQL comprises two namespaces: `costs` and `tags`, which represent the available [filters](/cost_reports#filtering-cost-reports) on Cost Reports in the Vantage console. To reference a filter, use the following syntax: `namespace.field` (e.g., `costs.provider` or `tags.name`). diff --git a/docs/vql_resource_report.md b/docs/vql_resource_report.md index 483c4460..44eb8308 100644 --- a/docs/vql_resource_report.md +++ b/docs/vql_resource_report.md @@ -13,6 +13,12 @@ keywords: import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; +This page describes how to use VQL when querying Resource Reports in API or using the Terraform Provider. + +:::tip +If you need help constructing a VQL query, navigate to the [**Resource Reports**](https://console.vantage.sh/resources) page and click **New Resource Report**. From the top left, open the **Filters** menu. Create a filter and click the **View as VQL** button at the top of the **Filters** menu to see a filter's VQL representation. You can copy this syntax to use within your API calls. +::: + ## Resource Reports VQL Schema VQL for Resources Reports comprises two namespaces: `resources` and `tags`, which represent the available [filters](/active_resources#create-a-resource-report) on Resource Reports in the Vantage console. To reference a filter, use the following syntax: `namespace.field` (e.g., `resources.region` or `tags.name`). The following fields are available within these namespaces. @@ -28,7 +34,7 @@ VQL for Resources Reports comprises two namespaces: `resources` and `tags`, whic - + @@ -82,12 +88,12 @@ VQL includes a set of keywords to create complex filter conditions. These keywor | -------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AND` | Logical AND operator | `resources.provider = 'aws' AND resources.label = '123456'` | This example filters AWS resources, with a specific associated label, where both conditions must be true. | | `OR` | Logical OR operator | `(resources.provider = 'aws') OR (resources.provider = 'gcp')` | This example retrieves resources from either AWS or GCP. At least one condition must be true. | -| `LIKE` and `NOT LIKE` | Performs string comparisons | `resources.provider = 'aws' AND resources.uuid LIKE '%arn:aws:s3:::my-bucket%''` | This example selects data where the resource ARN contains `arn:aws:s3:::my-bucket`, such as `arn:aws:s3:::my-bucket-123`.

This same query also works for `NOT LIKE` where data does not contain a particular string `(resources.provider = 'aws' AND resources.uuid NOT LIKE '%arn:aws:s3:::my-bucket%').` | +| `LIKE` and `NOT LIKE` | Performs string comparisons | `resources.provider = 'aws' AND resources.uuid LIKE '%arn:aws:s3:::my-bucket%''` | This example selects data where the resource ARN contains `arn:aws:s3:::my-bucket`, such as `arn:aws:s3:::my-bucket-123`.

This same query also works for `NOT LIKE` where data does not contain a particular string: `resources.provider = 'aws' AND resources.uuid NOT LIKE '%arn:aws:s3:::my-bucket%'`. | | `!=` | Represents negation, "is not" | `resources.provider = 'azure' AND (resources.type != 'azurerm_public_ip' AND resources.type != 'azurerm_kubernetes_cluster')` | This example filters out data from two specified resource types, providing all Azure resources that are _not_ these types.| | `<`, `>`, `<=`, `>=` | Mathematical operators for numerical queries | `resources.provider = 'azure' AND (resources.type = 'azurerm_virtual_machine' AND resources.metadata->>'virtual_machine_size' > '7')` | This example looks for Virtual Machines that have a size greater than 7. | | `->>` | This operator is used only when constructing queries related to metadata | `resources.provider = 'aws' AND (resources.type = 'aws_instance' AND resources.metadata->>'architecture' = 'x86_64')` | This example looks for EC2 instances with an architecture of `x86_64`. | -With these operators and keywords, you can construct complex filter conditions in VQL, providing flexibility and precision when querying and analyzing cloud cost data. +With these operators and keywords, you can construct complex filter conditions in VQL. ## VQL Examples {#examples} @@ -127,7 +133,7 @@ resources.provider = 'aws' AND (resources.provider_account_id = '11111111111') ### Costs by Resource Type {#resource-type} -Filter costs to see a specific resource type. In the example below, the query is looking for any resource that is NOT an AWS CloudFront Distribution. Resource types are represented like `aws_cloudfront_distribution`. Expand the box below for a list of all available resource types and their VQL equivalents. +Filter costs to see a specific resource type. In the example below, the query is looking for any AWS resource that is _not_ an AWS CloudFront Distribution. Resource types are represented like `aws_cloudfront_distribution`. Expand the box below for a list of all available resource types and their VQL equivalents. ```sql resources.provider = 'aws' AND (resources.type != 'aws_cloudfront_distribution')
resourcesresources.providerprovider Providers example