Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] AI-recommended Expectations #10913

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/docusaurus/docs/cloud/deploy/deploy_gx_agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ The GX Agent serves as an intermediary between GX Cloud and your organization's

A local deployment of the GX Agent will allow you to test GX Cloud setup or processes from a single machine before moving to a shared production deployment. Alternatively, you can run the GX Agent in your deployment environment and leverage GX Cloud while connecting to Data Sources using your organization's environment and infrastructure, for enhanced control and security.

## Limitations

Agent-enabled deployments do not support [AI-recommended Expectations](/cloud/expectations/manage_expectations.md#generate-ai-recommended-expectations-beta). To be able to generate AI-recommended Expectations, use a [fully-hosted deployment](/cloud/deploy/deployment_patterns.md).

## Prerequisites

Expand Down
21 changes: 21 additions & 0 deletions docs/docusaurus/docs/cloud/expectations/manage_expectations.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,27 @@ To clear the Expectation condition, click the clear button located on the right-
When you [create a new Data Asset](/cloud/data_assets/manage_data_assets.md#add-a-data-asset-from-an-existing-data-source), you can choose to automatically generate Expectations that detect column changes in that Data Asset.
:::

## Generate AI-recommended Expectations (BETA)

To accelerate test coverage, you can generate AI-recommended Expectations for a Data Asset. These will be personalized based on an analysis of a sample of your data.

Keep the following requirements and limitations in mind when working with AI-recommended Expectations:
- Only [Snowflake Data Sources](/cloud/connect/connect_snowflake.md) are supported at this time.
- Your organization must be using a [fully-hosted deployment](/cloud/deploy/deployment_patterns.md).
- The analytical AI assesses only a sample of your data.
- AI-recommended Expectations won’t include [dynamic parameters](#dynamic-parameters) or [Expectation conditions](#expectation-condition). But, you can edit the generated Expectations to make them use these.

:::caution Avoid sensitive personal data
This feature sends small data samples and/or aggregate statistics to an AI model for analysis. Avoid using it with sensitve personal data.
:::

To add AI-recommended Expectations:
1. In GX Cloud, click **Data Assets**.
2. In the **Data Assets** list, click the Data Asset name.
3. Click **Generate Expectations**.
4. Optional. Run a Validation. See [Run a Validation](/cloud/validations/manage_validations.md#run-a-validation).
5. Optional. [Edit](#edit-an-expectation) or [delete](#delete-an-expectation) AI-generated Expectations based on the insights you get from running a Validation and your data quality needs.


## Optional. Define a Batch

Expand Down
2 changes: 1 addition & 1 deletion docs/docusaurus/docs/cloud/schedules/manage_schedules.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: 'Manage schedules'
description: Create and manage schedules for Validations in GX Cloud.
---

Use a schedule to automate data quality checks with GX-managed Expectations. When you add your first Expectation in the GX Cloud UI for a Data Asset, including when you choose to auto-generate Expectations to detect schema changes, we enable a default schedule for that Asset's GX-managed Expectations. By default, GX-managed Expectations are scheduled to run every 24 hours. The first run will be at the start of the next hour after you add your first Expectation in the Cloud UI. You can keep the default schedule, edit it, or disable it.
Use a schedule to automate data quality checks with GX-managed Expectations. When you add your first Expectation in the GX Cloud UI for a Data Asset, including when you choose to auto-generate Expectations, we enable a default schedule for that Asset's GX-managed Expectations. By default, GX-managed Expectations are scheduled to run every 24 hours. The first run will be at the start of the next hour after you add your first Expectation in the Cloud UI. You can keep the default schedule, edit it, or disable it.

:::note Schedules are for GX-managed Expectations only
To automate data quality checks for [API-managed Expectations](/cloud/expectations/manage_expectations.md#gx-managed-vs-api-managed-expectations), use an [orchestrator](/cloud/connect/connect_airflow.md).
Expand Down
Loading