Adds `databricks_volume` as data source #3211

karolusz · 2024-02-02T21:38:44Z

Changes

Adds databricks_volume data source.

The data source takes the volume's full_name ~~or catalog_name, schema_name, name~~ as user input.

I tried to use the schema customization from common.customizable_schema.go when defining data source's parameters. The following changes were made in common/resource.go as none of the function wrapping around
genericDatabricksData and the genericDatabricksData itself was not allowing to pass a customization function:

~~- genericDatabricksData now takes a schema customization function for customizing otherFields which is overlaid on top SDK schema.~~
~~- a new function WorkspaceDataWithCustomParams was added that wraps genericDatabricksData for cases where a schema customization function is desired.~~
~~- WorkspaceData, WorkspaceDataWithParams, AccountData, AccountDataWithParams were adjusted to pass NoCustomize to the updated genericDatabricksData~~

The above changes were taken out because of #3207

Tests

make test run locally
relevant change in docs/ folder
covered with integration tests in internal/acceptance
relevant acceptance tests are passing
using Go SDK

alexott

Please add docs as well

catalog/data_volume.go

codecov-commenter · 2024-02-03T10:06:17Z

Codecov Report

Attention: Patch coverage is 90.90909% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 82.56%. Comparing base (6cfcb2b) to head (5d21bc8).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3211      +/-   ##
==========================================
+ Coverage   82.55%   82.56%   +0.01%     
==========================================
  Files         186      188       +2     
  Lines       19131    19177      +46     
==========================================
+ Hits        15794    15834      +40     
- Misses       2411     2415       +4     
- Partials      926      928       +2

Files	Coverage Δ
common/resource.go	`74.64% <100.00%> (ø)`
provider/provider.go	`93.93% <100.00%> (+0.05%)`	⬆️
catalog/data_volume.go	`90.00% <90.00%> (ø)`

... and 2 files with indirect coverage changes

karolusz · 2024-02-04T23:40:30Z

Added docs, tests and resolved review comments.

karolusz · 2024-02-05T08:29:44Z

Formatting fixed.

karolusz · 2024-02-14T21:28:55Z

Small refactor to take advantage of #3207 and avoid having to add new functions in common/resource.go

Tests and docs added.

karolusz · 2024-02-19T09:45:25Z

@alexott the requested changes were addressed. Please let me know if you see anything else outstanding

alexott

In general looks good, just one comment

common/resource.go

…cks into feature/data-volume

karolusz · 2024-04-04T21:33:49Z

Update: Merged the latest changes from main and made adjustments for the latest version of GO SDK.

catalog/data_volume.go

alexott

relevant integration test has passed

alexott · 2024-05-15T05:34:07Z

@nkvuong @mgyucht I think that we need to discuss the unified approach to the data sources implementation - the metastore, catalog, and table sources are exposing data as a nested object metastore_info, catalog_info, etc., while storage credential, external location and volume data sources are putting the data as flat structure...

tanmay-db

We discussed this in the slack thread (mentioning here for visibility) but since most of the data sources use nested structure, I think we should use that here as well for consistency.

karolusz · 2024-07-02T04:02:37Z

We discussed this in the slack thread (mentioning here for visibility) but since most of the data sources use nested structure, I think we should use that here as well for consistency.

The changes were made to comply with the approach of having data sources as nested structures.

alexott

Small changes are still requiired

catalog/data_volume.go

common/resource.go

docs/data-sources/volume.md

internal/acceptance/data_volume_test.go

alexott

integration tests are failing... I think that it's ok to remove that check to match test in databricks_table data source: https://github.com/databricks/terraform-provider-databricks/blob/main/internal/acceptance/data_table_test.go#L11

internal/acceptance/data_volume_test.go

alexott · 2024-07-02T10:44:26Z

Integration test passed

alexott · 2024-07-03T07:06:03Z

thank you for contribution @karolusz !

### Internal Changes * Add Release tag ([#3748](#3748)). * Improve Changelog by grouping changes ([#3747](#3747)). * Upgrade Go SDK to v0.43.2 ([#3750](#3750)). ### Other Changes * Add `databricks_schema` data source ([#3732](#3732)). * Add new APIErrorBody struct and update deps ([#3745](#3745)). * Added support for binding storage credentials and external locations to specific workspaces ([#3678](#3678)). * Adds `databricks_volume` as data source ([#3211](#3211)). * Change TF registry ownership ([#3736](#3736)). * Exporter: Emit directories during the listing only if they are explicitly configured in `-listing` ([#3673](#3673)). * Exporter: export libraries specified as `requirements.txt` ([#3649](#3649)). * Exporter: fix generation of `run_as` blocks in `databricks_job` ([#3724](#3724)). * Exporter: use Go SDK structs for `databricks_job` resource ([#3727](#3727)). * Fix invalid priviledges in grants.md ([#3716](#3716)). * Make the schedule.pause_status field read-only ([#3692](#3692)). * Refactored `databricks_cluster(s)` data sources to Go SDK ([#3685](#3685)). * Renamed `databricks_catalog_workspace_binding` to `databricks_workspace_binding` ([#3703](#3703)). * Run goreleaser action in snapshot mode from merge queue ([#3646](#3646)). * Update cluster.md: add data_security_mode parameters `NONE` and `NO_ISOLATION` ([#3740](#3740)). * Upgrade databricks-sdk-go ([#3743](#3743)). * remove references to basic auth ([#3720](#3720)).

## 1.49.0 ### New Features and Improvements * Added `databricks_dashboard` resource ([#3729](#3729)). * Added `databricks_schema` data source ([#3732](#3732)). * Added support for binding storage credentials and external locations to specific workspaces ([#3678](#3678)). * Added `databricks_volume` as data source ([#3211](#3211)). * Make the `schedule.pause_status` field read-only ([#3692](#3692)). * Renamed `databricks_catalog_workspace_binding` to `databricks_workspace_binding` ([#3703](#3703)). * Make `cluster_name_contains` optional in `databricks_clusters` data source ([#3760](#3760)). * Tolerate OAuth errors in databricks_mws_workspaces when managing tokens ([#3761](#3761)). * Permissions for `databricks_dashboard` resource ([#3762](#3762)). ### Exporter * Emit directories during the listing only if they are explicitly configured in `-listing` ([#3673](#3673)). * Export libraries specified as `requirements.txt` ([#3649](#3649)). * Fix generation of `run_as` blocks in `databricks_job` ([#3724](#3724)). * Use Go SDK structs for `databricks_job` resource ([#3727](#3727)). * Clarify use of `-listing` and `-services` options ([#3755](#3755)). * Improve code generation for SQL Endpoints ([#3764](#3764)) ### Documentation * Fix invalid priviledges in grants.md ([#3716](#3716)). * Update cluster.md: add data_security_mode parameters `NONE` and `NO_ISOLATION` ([#3740](#3740)). * Remove references to basic auth ([#3720](#3720)). * Update resources diagram ([#3765](#3765)). ### Internal Changes * Add Release tag ([#3748](#3748)). * Improve Changelog by grouping changes ([#3747](#3747)). * Change TF registry ownership ([#3736](#3736)). * Refactored `databricks_cluster(s)` data sources to Go SDK ([#3685](#3685)). * Upgrade databricks-sdk-go ([#3743](#3743)). * Run goreleaser action in snapshot mode from merge queue ([#3646](#3646)). * Make `dashboard_name` random in integration tests for `databricks_dashboard` resource ([#3763](#3763)). * Clear stale go.sum values ([#3768](#3768)). * Add "Owner" tag to test cluster in acceptance test ([#3771](#3771)). * Fix integration test for restrict workspace admins setting ([#3772](#3772)). * Add "Owner" tag to test SQL endpoint in acceptance test ([#3774](#3774)). * Move PR message validation to a separate workflow ([#3777](#3777)). * Trigger the validate workflow in the merge queue ([#3782](#3782)). * Update properties for managed SQL table on latest DBR ([#3784](#3784)). * Add "Owner" tag to test SQL endpoint in acceptance test ([#3785](#3785)).

data_volume

fdb99a6

karolusz requested review from a team as code owners February 2, 2024 21:38

karolusz requested review from hectorcast-db and removed request for a team February 2, 2024 21:38

alexott requested changes Feb 3, 2024

View reviewed changes

catalog/data_volume.go Outdated Show resolved Hide resolved

karolusz added 3 commits February 4, 2024 00:18

data_volume unit and acceptance tests

4423b42

docs

46c5f55

WorkspaceDataWithCustomParams test

a28ef66

karolusz requested a review from alexott February 4, 2024 23:40

fixed formatting

367300a

mgyucht mentioned this pull request Feb 5, 2024

Make IterFields take in aliases #3207

Merged

5 tasks

karolusz added 3 commits February 14, 2024 00:29

Removing unnecessary changes to resource.go

5433d72

Merge branch 'main' into feature/data-volume

6e55e9a

refactored data_volume

8f15bad

alexott reviewed Feb 20, 2024

View reviewed changes

common/resource.go Show resolved Hide resolved

karolusz requested a review from alexott March 5, 2024 12:52

karolusz added 2 commits April 4, 2024 23:02

Merge branch 'main' of github.com:karolusz/terraform-provider-databri…

de2e3b3

…cks into feature/data-volume

making change for consitency with GO SDK v0.35.0

356e33d

alexott requested a review from a team April 5, 2024 13:58

Merge branch 'main' into feature/data-volume

c3ab878

nkvuong reviewed May 14, 2024

View reviewed changes

catalog/data_volume.go Outdated Show resolved Hide resolved

edwardfeng-db reviewed May 14, 2024

View reviewed changes

catalog/data_volume.go Outdated Show resolved Hide resolved

nkvuong reviewed May 14, 2024

View reviewed changes

catalog/data_volume.go Outdated Show resolved Hide resolved

Update catalog/data_volume.go

859cb62

nkvuong reviewed May 14, 2024

View reviewed changes

catalog/data_volume.go Outdated Show resolved Hide resolved

Update catalog/data_volume.go

5d21bc8

alexott approved these changes May 14, 2024

View reviewed changes

edwardfeng-db approved these changes May 14, 2024

View reviewed changes

tanmay-db reviewed May 17, 2024

View reviewed changes

karolusz added 2 commits July 2, 2024 05:53

data source as nested strucutre

6decf39

merge conflict

759d387

karolusz requested review from tanmay-db and nkvuong July 2, 2024 04:10

alexott requested changes Jul 2, 2024

View reviewed changes

review comments addressed

714d0c2

alexott requested changes Jul 2, 2024

View reviewed changes

internal/acceptance/data_volume_test.go Outdated Show resolved Hide resolved

acceptance test

2b57c13

alexott enabled auto-merge July 3, 2024 07:05

alexott approved these changes Jul 3, 2024

View reviewed changes

alexott added this pull request to the merge queue Jul 3, 2024

Merged via the queue into databricks:main with commit ff837ab Jul 3, 2024
5 checks passed

karolusz deleted the feature/data-volume branch July 3, 2024 12:15

tanmay-db mentioned this pull request Jul 9, 2024

[Release] v1.49.0 #3752

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds `databricks_volume` as data source #3211

Adds `databricks_volume` as data source #3211

karolusz commented Feb 2, 2024 •

edited by alexott

Loading

alexott left a comment

codecov-commenter commented Feb 3, 2024 •

edited

Loading

karolusz commented Feb 4, 2024

karolusz commented Feb 5, 2024

karolusz commented Feb 14, 2024

karolusz commented Feb 19, 2024

alexott left a comment

karolusz commented Apr 4, 2024 •

edited

Loading

alexott left a comment

alexott commented May 15, 2024

tanmay-db left a comment

karolusz commented Jul 2, 2024

alexott left a comment

alexott left a comment

alexott commented Jul 2, 2024

alexott commented Jul 3, 2024

Adds databricks_volume as data source #3211

Adds databricks_volume as data source #3211

Conversation

karolusz commented Feb 2, 2024 • edited by alexott Loading

Changes

Tests

alexott left a comment

Choose a reason for hiding this comment

codecov-commenter commented Feb 3, 2024 • edited Loading

Codecov Report

karolusz commented Feb 4, 2024

karolusz commented Feb 5, 2024

karolusz commented Feb 14, 2024

karolusz commented Feb 19, 2024

alexott left a comment

Choose a reason for hiding this comment

karolusz commented Apr 4, 2024 • edited Loading

alexott left a comment

Choose a reason for hiding this comment

alexott commented May 15, 2024

tanmay-db left a comment

Choose a reason for hiding this comment

karolusz commented Jul 2, 2024

alexott left a comment

Choose a reason for hiding this comment

alexott left a comment

Choose a reason for hiding this comment

alexott commented Jul 2, 2024

alexott commented Jul 3, 2024

Adds `databricks_volume` as data source #3211

Adds `databricks_volume` as data source #3211

karolusz commented Feb 2, 2024 •

edited by alexott

Loading

codecov-commenter commented Feb 3, 2024 •

edited

Loading

karolusz commented Apr 4, 2024 •

edited

Loading