Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example to show how bundle variables work #21

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions knowledge_base/bundle_variables/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.databricks
65 changes: 65 additions & 0 deletions knowledge_base/bundle_variables/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# bundle_variables
Copy link
Contributor

@juliacrawf-db juliacrawf-db Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a real title would be nice. Given it's a KB topic, how about "How to validate bundle variables"? Or maybe that's not quite right... "How to validate and set variable values"?


Example to show how bundle variables work.

We will focus on the behavior of the variables section alone. To do so, we exclusively use
the `validate` command, and use its JSON output mode to focus on the variables section.
Comment on lines +3 to +6
Copy link
Contributor

@juliacrawf-db juliacrawf-db Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example to show how bundle variables work.
We will focus on the behavior of the variables section alone. To do so, we exclusively use
the `validate` command, and use its JSON output mode to focus on the variables section.
This example shows how to validate and set values for the custom variables defined for a bundle using the `bundle validate` command and its JSON output mode.

Copy link
Contributor

@juliacrawf-db juliacrawf-db Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is the main intent of this KB, but another sentence could also be added about showing how to set variable defaults. (Update: I tinkered with that suggestion.)


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any prerequisites worth mentioning? It might be a good standard just to list the supported CLI version for any KB?

## Usage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This singular heading feels unnecessary and also doesn't seem like quite the right word. I'd either remove it (acceptable for a KB e.g., https://kb.databricks.com/en_US/clusters/calculate-number-of-cores) or break up the content into at least two headings. (I'd lean towards the former.)


Configure the workspace to use:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Configure the workspace to use:
First, configure your workspace:

```sh
export DATABRICKS_CONFIG_PROFILE="<workspace profile>"
```

Try to run validate without arguments:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Try to run validate without arguments:
Next, run `bundle validate` using the JSON output mode and `jq` to retrieve just the variables node.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure how this command was "without arguments" but maybe I'm misunderstanding the intent there. Also, I'm not sure if it's necessary to state that last bit as most devs know all about jq, but it feels most complete.

```sh
databricks bundle validate --output json | jq .variables
```

Because the configuration includes a variable definition without a value, it returns an error
saying that the value must be defined:
Comment on lines +20 to +21
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Because the configuration includes a variable definition without a value, it returns an error
saying that the value must be defined:
Because the configuration includes a variable definition without a value, it returns the following error:

```
Error: no value assigned to required variable "no_default".
```

Assign a value for this variable by either:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Assign a value for this variable by either:
To assign a value to this variable, you can either:


1. Adding a `--var` flag to all bundle commands:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Adding a `--var` flag to all bundle commands:
- Add a `--var` flag to all bundle commands:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These aren't steps (numbered list) but a list of options (bulleted list). However, I like making it really clear (see the next comment)...

```sh
databricks bundle validate --var no_default="injected value"
```
Comment on lines +29 to +31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indent these to fit under the bullet.


2. Configuring an environment variable:
Copy link
Contributor

@juliacrawf-db juliacrawf-db Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. Configuring an environment variable:
OR
- Configure an environment variable:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(But this might look weird, or not follow our style)

```sh
export BUNDLE_VAR_no_default="injected value"
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indent this to fit under the bullet.


Now, try to run validate again, and observe that it passes:
Copy link
Contributor

@juliacrawf-db juliacrawf-db Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Now, try to run validate again, and observe that it passes:
Now run validate again, and observe that it succeeds in outputting valid JSON:

```sh
databricks bundle validate --var no_default="injected value" --output json | jq .variables
```

Example output:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example output:

```json
{
"no_default": {
"description": "This is a variable with no default value",
"value": "injected value"
},
"with_default": {
"description": "This is a variable with a default value",
"value": "hello"
},
"with_default_in_targets": {
"default": "value_in_development",
"description": "This is a variable with its default value defined in the targets section",
"value": "value_in_development"
}
}
```

What we've seen:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
What we've seen:
In summary:

* The `no_default` variable is defined with the value we injected.
* The `with_default` variable is defined with the default value.
* The `with_default_in_targets` variable is defined with the value from the targets section.
24 changes: 24 additions & 0 deletions knowledge_base/bundle_variables/databricks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
bundle:
name: bundle_variables

variables:
no_default:
description: This is a variable with no default value

with_default:
description: This is a variable with a default value
value: hello

with_default_in_targets:
description: This is a variable with its default value defined in the targets section

Copy link
Contributor

@juliacrawf-db juliacrawf-db Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pedantic, I know, but can we add periods to the end of the sentences on all of these descriptions?

targets:
development:
default: true

variables:
with_default_in_targets: value_in_development

production:
variables:
with_default_in_targets: value_in_production