Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds IP Filter feature support to google_storage_bucket resource. #12945

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

kautikdk
Copy link
Member

@kautikdk kautikdk commented Feb 3, 2025

This PR enables users to specify IP Filtering configuration within google_storage_bucket resource.

Original PR: #12576
Authors: @translucens, @kautikdk

storage: added `ip_filter` to `google_storage_bucket` resource.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 1 file changed, 1 insertion(+))
google-beta provider: Diff ( 2 files changed, 357 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 118
Passed tests: 4
Skipped tests: 9
Affected tests: 105

Click here to see the affected service packages
  • storage

Action taken

Found 105 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataSourceGoogleStorageBucketObject_basic
  • TestAccDataSourceGoogleStorageBucketObjects_basic
  • TestAccDataSourceGoogleStorageBucket_avoidComputeAPI
  • TestAccDataSourceGoogleStorageBucket_basic
  • TestAccDataSourceGoogleStorageBuckets_basic
  • TestAccDataSourceStorageBucketObjectContent_Basic
  • TestAccDataSourceStorageBucketObjectContent_Issue15717
  • TestAccDataSourceStorageBucketObjectContent_Issue15717BackwardCompatibility
  • TestAccStorageBucketAccessControl_storageBucketAccessControlPublicBucketExample
  • TestAccStorageBucketAccessControl_update
  • TestAccStorageBucketAcl_RemoveOwner
  • TestAccStorageBucketAcl_basic
  • TestAccStorageBucketAcl_downgrade
  • TestAccStorageBucketAcl_predefined
  • TestAccStorageBucketAcl_unordered
  • TestAccStorageBucketAcl_upgrade
  • TestAccStorageBucketAcl_upgradeSingleUser
  • TestAccStorageBucketIamBindingGenerated
  • TestAccStorageBucketIamBindingGenerated_withCondition
  • TestAccStorageBucketIamMemberGenerated
  • TestAccStorageBucketIamMemberGenerated_withCondition
  • TestAccStorageBucketIamPolicy
  • TestAccStorageBucketIamPolicyGenerated
  • TestAccStorageBucketIamPolicyGenerated_withCondition
  • TestAccStorageBucket_AutoclassDiffSuppress
  • TestAccStorageBucket_IPFilter
  • TestAccStorageBucket_SoftDeletePolicy
  • TestAccStorageBucket_basic
  • TestAccStorageBucket_basicWithAutoclass
  • TestAccStorageBucket_basic_hns
  • TestAccStorageBucket_cors
  • TestAccStorageBucket_customAttributes
  • TestAccStorageBucket_defaultEventBasedHold
  • TestAccStorageBucket_dualLocation
  • TestAccStorageBucket_dualLocation_lowercase
  • TestAccStorageBucket_dualLocation_rpo
  • TestAccStorageBucket_enable_object_retention
  • TestAccStorageBucket_forceDestroy
  • TestAccStorageBucket_forceDestroyObjectDeleteError
  • TestAccStorageBucket_forceDestroyWithVersioning
  • TestAccStorageBucket_hns_force_destroy
  • TestAccStorageBucket_labels
  • TestAccStorageBucket_lifecycleRuleStateAny
  • TestAccStorageBucket_lifecycleRuleStateArchived
  • TestAccStorageBucket_lifecycleRuleStateLive
  • TestAccStorageBucket_lifecycleRulesVirtualFields
  • TestAccStorageBucket_logging
  • TestAccStorageBucket_lowercaseLocation
  • TestAccStorageBucket_multiLocation_rpo
  • TestAccStorageBucket_publicAccessPrevention
  • TestAccStorageBucket_requesterPays
  • TestAccStorageBucket_retentionPolicy
  • TestAccStorageBucket_retentionPolicyLocked
  • TestAccStorageBucket_storageClass
  • TestAccStorageBucket_uniformBucketAccessOnly
  • TestAccStorageBucket_update
  • TestAccStorageBucket_update_requesterPays
  • TestAccStorageBucket_versioning
  • TestAccStorageBucket_website
  • TestAccStorageDefaultObjectAccessControl_storageDefaultObjectAccessControlPublicExample
  • TestAccStorageDefaultObjectAccessControl_update
  • TestAccStorageDefaultObjectAcl_basic
  • TestAccStorageDefaultObjectAcl_downgrade
  • TestAccStorageDefaultObjectAcl_noRoleEntity
  • TestAccStorageDefaultObjectAcl_unordered
  • TestAccStorageDefaultObjectAcl_upgrade
  • TestAccStorageFolder_DeleteEmptyFolderWithForceDestroyDefault
  • TestAccStorageFolder_FailDeleteNonEmptyFolder
  • TestAccStorageFolder_FolderForceDestroy
  • TestAccStorageFolder_hnsDisabled
  • TestAccStorageFolder_storageFolderBasic
  • TestAccStorageFolder_storageFolderBasicExample
  • TestAccStorageManagedFolderIamBindingGenerated
  • TestAccStorageManagedFolderIamBindingGenerated_withCondition
  • TestAccStorageManagedFolderIamMemberGenerated
  • TestAccStorageManagedFolderIamMemberGenerated_withCondition
  • TestAccStorageManagedFolderIamPolicyGenerated
  • TestAccStorageManagedFolderIamPolicyGenerated_withCondition
  • TestAccStorageManagedFolder_storageManagedFolderBasicExample
  • TestAccStorageManagedFolder_storageManagedFolderUpdate
  • TestAccStorageNotification_basic
  • TestAccStorageNotification_withEventsAndAttributes
  • TestAccStorageObjectAccessControl_storageObjectAccessControlPublicObjectExample
  • TestAccStorageObjectAccessControl_update
  • TestAccStorageObjectAccessControl_updateWithSlashes
  • TestAccStorageObjectAcl_basic
  • TestAccStorageObjectAcl_downgrade
  • TestAccStorageObjectAcl_explicitToPredefined
  • TestAccStorageObjectAcl_predefined
  • TestAccStorageObjectAcl_predefinedToExplicit
  • TestAccStorageObjectAcl_unordered
  • TestAccStorageObjectAcl_upgrade
  • TestAccStorageObjectKms
  • TestAccStorageObject_basic
  • TestAccStorageObject_cacheControl
  • TestAccStorageObject_content
  • TestAccStorageObject_customerEncryption
  • TestAccStorageObject_dynamicContent
  • TestAccStorageObject_folder
  • TestAccStorageObject_holds
  • TestAccStorageObject_metadata
  • TestAccStorageObject_recreate
  • TestAccStorageObject_retention
  • TestAccStorageObject_storageClass
  • TestAccStorageObject_withContentCharacteristics

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccDataSourceGoogleStorageBucketObject_basic [Debug log]
TestAccDataSourceGoogleStorageBucketObjects_basic [Debug log]
TestAccDataSourceGoogleStorageBucket_avoidComputeAPI [Debug log]
TestAccDataSourceGoogleStorageBucket_basic [Debug log]
TestAccDataSourceGoogleStorageBuckets_basic [Debug log]
TestAccDataSourceStorageBucketObjectContent_Basic [Debug log]
TestAccDataSourceStorageBucketObjectContent_Issue15717 [Debug log]
TestAccDataSourceStorageBucketObjectContent_Issue15717BackwardCompatibility [Debug log]
TestAccStorageBucketAccessControl_storageBucketAccessControlPublicBucketExample [Debug log]
TestAccStorageBucketAccessControl_update [Debug log]
TestAccStorageBucketAcl_RemoveOwner [Debug log]
TestAccStorageBucketAcl_basic [Debug log]
TestAccStorageBucketAcl_downgrade [Debug log]
TestAccStorageBucketAcl_predefined [Debug log]
TestAccStorageBucketAcl_unordered [Debug log]
TestAccStorageBucketAcl_upgrade [Debug log]
TestAccStorageBucketAcl_upgradeSingleUser [Debug log]
TestAccStorageBucketIamBindingGenerated [Debug log]
TestAccStorageBucketIamBindingGenerated_withCondition [Debug log]
TestAccStorageBucketIamMemberGenerated [Debug log]
TestAccStorageBucketIamMemberGenerated_withCondition [Debug log]
TestAccStorageBucketIamPolicy [Debug log]
TestAccStorageBucketIamPolicyGenerated [Debug log]
TestAccStorageBucketIamPolicyGenerated_withCondition [Debug log]
TestAccStorageBucket_AutoclassDiffSuppress [Debug log]
TestAccStorageBucket_SoftDeletePolicy [Debug log]
TestAccStorageBucket_basic [Debug log]
TestAccStorageBucket_basicWithAutoclass [Debug log]
TestAccStorageBucket_basic_hns [Debug log]
TestAccStorageBucket_cors [Debug log]
TestAccStorageBucket_customAttributes [Debug log]
TestAccStorageBucket_defaultEventBasedHold [Debug log]
TestAccStorageBucket_dualLocation [Debug log]
TestAccStorageBucket_dualLocation_lowercase [Debug log]
TestAccStorageBucket_dualLocation_rpo [Debug log]
TestAccStorageBucket_enable_object_retention [Debug log]
TestAccStorageBucket_forceDestroy [Debug log]
TestAccStorageBucket_forceDestroyObjectDeleteError [Debug log]
TestAccStorageBucket_forceDestroyWithVersioning [Debug log]
TestAccStorageBucket_hns_force_destroy [Debug log]
TestAccStorageBucket_labels [Debug log]
TestAccStorageBucket_lifecycleRuleStateAny [Debug log]
TestAccStorageBucket_lifecycleRuleStateArchived [Debug log]
TestAccStorageBucket_lifecycleRuleStateLive [Debug log]
TestAccStorageBucket_lifecycleRulesVirtualFields [Debug log]
TestAccStorageBucket_logging [Debug log]
TestAccStorageBucket_lowercaseLocation [Debug log]
TestAccStorageBucket_multiLocation_rpo [Debug log]
TestAccStorageBucket_publicAccessPrevention [Debug log]
TestAccStorageBucket_requesterPays [Debug log]
TestAccStorageBucket_retentionPolicy [Debug log]
TestAccStorageBucket_retentionPolicyLocked [Debug log]
TestAccStorageBucket_uniformBucketAccessOnly [Debug log]
TestAccStorageBucket_update [Debug log]
TestAccStorageBucket_update_requesterPays [Debug log]
TestAccStorageBucket_versioning [Debug log]
TestAccStorageBucket_website [Debug log]
TestAccStorageDefaultObjectAccessControl_storageDefaultObjectAccessControlPublicExample [Debug log]
TestAccStorageDefaultObjectAccessControl_update [Debug log]
TestAccStorageDefaultObjectAcl_basic [Debug log]
TestAccStorageDefaultObjectAcl_downgrade [Debug log]
TestAccStorageDefaultObjectAcl_noRoleEntity [Debug log]
TestAccStorageDefaultObjectAcl_unordered [Debug log]
TestAccStorageDefaultObjectAcl_upgrade [Debug log]
TestAccStorageFolder_DeleteEmptyFolderWithForceDestroyDefault [Debug log]
TestAccStorageFolder_FailDeleteNonEmptyFolder [Debug log]
TestAccStorageFolder_FolderForceDestroy [Debug log]
TestAccStorageFolder_hnsDisabled [Debug log]
TestAccStorageFolder_storageFolderBasic [Debug log]
TestAccStorageFolder_storageFolderBasicExample [Debug log]
TestAccStorageManagedFolderIamBindingGenerated [Debug log]
TestAccStorageManagedFolderIamBindingGenerated_withCondition [Debug log]
TestAccStorageManagedFolderIamMemberGenerated [Debug log]
TestAccStorageManagedFolderIamMemberGenerated_withCondition [Debug log]
TestAccStorageManagedFolderIamPolicyGenerated [Debug log]
TestAccStorageManagedFolderIamPolicyGenerated_withCondition [Debug log]
TestAccStorageManagedFolder_storageManagedFolderBasicExample [Debug log]
TestAccStorageManagedFolder_storageManagedFolderUpdate [Debug log]
TestAccStorageNotification_basic [Debug log]
TestAccStorageNotification_withEventsAndAttributes [Debug log]
TestAccStorageObjectAccessControl_storageObjectAccessControlPublicObjectExample [Debug log]
TestAccStorageObjectAccessControl_update [Debug log]
TestAccStorageObjectAccessControl_updateWithSlashes [Debug log]
TestAccStorageObjectAcl_basic [Debug log]
TestAccStorageObjectAcl_downgrade [Debug log]
TestAccStorageObjectAcl_explicitToPredefined [Debug log]
TestAccStorageObjectAcl_predefined [Debug log]
TestAccStorageObjectAcl_predefinedToExplicit [Debug log]
TestAccStorageObjectAcl_unordered [Debug log]
TestAccStorageObjectAcl_upgrade [Debug log]
TestAccStorageObjectKms [Debug log]
TestAccStorageObject_basic [Debug log]
TestAccStorageObject_cacheControl [Debug log]
TestAccStorageObject_content [Debug log]
TestAccStorageObject_customerEncryption [Debug log]
TestAccStorageObject_dynamicContent [Debug log]
TestAccStorageObject_folder [Debug log]
TestAccStorageObject_holds [Debug log]
TestAccStorageObject_metadata [Debug log]
TestAccStorageObject_recreate [Debug log]
TestAccStorageObject_retention [Debug log]
TestAccStorageObject_storageClass [Debug log]
TestAccStorageObject_withContentCharacteristics [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🔴 Tests failed during RECORDING mode:
TestAccStorageBucket_IPFilter [Error message] [Debug log]
TestAccStorageBucket_storageClass [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 1 file changed, 1 insertion(+))
google-beta provider: Diff ( 2 files changed, 357 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 118
Passed tests: 107
Skipped tests: 9
Affected tests: 2

Click here to see the affected service packages
  • storage

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccStorageBucket_IPFilter
  • TestAccStorageBucket_storageClass

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccStorageBucket_IPFilter [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🔴 Tests failed during RECORDING mode:
TestAccStorageBucket_storageClass [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 1 file changed, 1 insertion(+))
google-beta provider: Diff ( 2 files changed, 357 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 118
Passed tests: 108
Skipped tests: 9
Affected tests: 1

Click here to see the affected service packages
  • storage

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccStorageBucket_storageClass

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccStorageBucket_storageClass [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 362 insertions(+), 1 deletion(-))
google-beta provider: Diff ( 2 files changed, 362 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 118
Passed tests: 108
Skipped tests: 9
Affected tests: 1

Click here to see the affected service packages
  • storage

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccStorageBucket_storageClass

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccStorageBucket_storageClass [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 394 insertions(+), 1 deletion(-))
google-beta provider: Diff ( 3 files changed, 394 insertions(+), 1 deletion(-))

Errors

google provider:

  • The diff processor failed to build. This is usually due to the downstream provider failing to compile.

google-beta provider:

  • The diff processor failed to build. This is usually due to the downstream provider failing to compile.

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 0
Passed tests: 0
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages
  • storage
#### Non-exercised tests

🔴 Tests were added that are skipped in VCR:

  • TestAccStorageBucket_IPFilter
    🔴 Errors occurred during REPLAYING mode. Please fix them to complete your PR.

View the build log

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 394 insertions(+), 1 deletion(-))
google-beta provider: Diff ( 3 files changed, 394 insertions(+), 1 deletion(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 118
Passed tests: 108
Skipped tests: 9
Affected tests: 1

Click here to see the affected service packages
  • storage

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccStorageBucket_IPFilter

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccStorageBucket_IPFilter [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

@kautikdk
Copy link
Member Author

kautikdk commented Feb 4, 2025

@roaks3 can you please continue your review on this PR?

@kautikdk kautikdk marked this pull request as ready for review February 4, 2025 19:09
@github-actions github-actions bot requested a review from trodge February 4, 2025 19:10
Copy link

github-actions bot commented Feb 4, 2025

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@trodge, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

Copy link
Contributor

@roaks3 roaks3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small comment, but as mentioned, this was reviewed before and only blocked on tests (which are now passing)

@@ -963,7 +1050,7 @@ func resourceStorageBucketRead(d *schema.ResourceData, meta interface{}) error {

// There seems to be some eventual consistency errors in some cases, so we want to check a few times
// to make sure it exists before moving on
res, err := config.NewStorageClient(userAgent).Buckets.Get(bucket).Do()
res, err := config.NewStorageClient(userAgent).Buckets.Get(bucket).Projection("full").Do()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't remember if this came up on the previous PR, but to confirm, this won't change the behavior for existing bucket users?

Copy link
Member Author

@kautikdk kautikdk Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So as per this documentation: https://cloud.google.com/storage/docs/json_api/v1/buckets/get, projection is an optional query parameter which can have two values: full and noAcl. The default is noAcl which also excludes ipFilter property from the API response. Now to include IP filter in the response we need to use full projection which also adds owner, acl and defaultObjectAcl fields. These extra fields are not being parsed by flatteners available in google_storage_bucket resource but they are available in google_storage_bucket_acl. Using full projection should not break existing user script as we are not going to parse additional fields coming from the API response. I will still check with Feature team on this if they have plans to add IP filter in noAcl projection. I will update on this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok yea, seems worth checking if we can just leave it as noAcl, but what you described does sound safe. I would consider this non-blocking, but let me know when you hear back.

@roaks3 roaks3 removed the request for review from trodge February 5, 2025 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants