v1.4.0

The v1.4.0 release introduces MR metrics, bug fixes, enhancements, and dependency updates.

We are excited to introduce a new set of managed resource (MR) metrics in this release. These metrics are pivotal for monitoring the state and performance of resources managed through Crossplane. This enhancement follows the implementation guidelines from the provider-kubernetes and incorporates changes from crossplane/crossplane-runtime#683. The newly available metrics include:

• crossplane_managed_resource_exists{"gvk"}
• crossplane_managed_resource_ready{"gvk"}
• crossplane_managed_resource_synced{"gvk"}
• crossplane_managed_resource_first_time_to_reconcile_seconds{"gvk"}
• crossplane_managed_resource_first_time_to_readiness_seconds{"gvk"}
• crossplane_managed_resource_deletion_seconds{"gvk"}
• crossplane_managed_resource_drift_seconds{"gvk"}

For a detailed overview of the implementation, please refer to this crossplane/crossplane-runtime#683.

Note that, for async resources, upjet_resource_ttr_bucket{"group","kind","version"} gives a more accurate measurement of when the external resource becomes available, compared to crossplane_managed_resource_first_time_to_readiness_seconds{"gvk"} .

Bug Fixes and Enhancements

• Bug Fix: Addresses a critical bug related to asynchronous operation failures (#1164). With this fix, the "Synced" status will immediately reflect as "False" upon any failure in asynchronous operations, ensuring compliance with the XRM contract and improving reliability in status reporting. Previously, the error handling only updated the LastAsyncOperation status without properly setting the Synced condition to False. For more information, please see the crossplane/upjet#391 (comment).
• Bug Fix: Lambda Functions keeps updating after few seconds with Diff Detected #1027
• Enhancement: Adds ClusterParameterGroup and ParameterGroup Selectors for the Cluster.rds resource

What's Changed

• Monitor, mark and close issues and PRs with github workflow by @turkenf in #1250
• pin aws-sdk-go-v2 dependency versions for ClusterAuth presign breaking change by @erhancagirici in #1251
• Change the value of meta.crossplane.io/source by @turkenf in #1246
• Update docker/setup-buildx-action digest to d70bba7 by @renovate in #1258
• Update module sigs.k8s.io/controller-runtime to v0.17.3 by @renovate in #1259
• Removes GitHub issue triage workflow by @jeanduplessis in #1263
• Update kubernetes patches to v0.29.4 by @renovate in #1267
• Select spec fields specific to Upbound OIDC by @jastang in #1265
• Remove ARN Parameter from Firehose DeliveryStream Resource by @blakeromano in #1233
• Update actions/upload-artifact digest to 1746f4a by @renovate in #1269
• Remove resource from externalnamenottested that is already implemented by @mbbush in #1270
• Update slack channel name in readme by @turkenf in #1275
• Update actions/checkout digest to 1d96c77 by @renovate in #1283
• Rename upbound provider ci repo by @mbbush in #1287
• [Cluster.RDS] Add ClusterParameterGroup and ParameterGroup Selectors by @ytsarev in #1288
• Fix update loop for Function.lambda resource by @turkenf in #1266
• Introduce MR metrics by @mergenci in #1281

Full Changelog: v1.3.1...v1.4.0

v1.3.1

The release v1.3.1 introduces an important bug fix: [Bug]: EKS ClusterAuth v1.3.0 kubeconfig results in Unauthorized#1248

What's Changed

• [Backport release-1.3] pin aws-sdk-go-v2 dependency versions for ClusterAuth presign breaking change by @github-actions in #1257

Full Changelog: v1.3.0...v1.3.1

v1.3.0

The v1.3.0 release introduces a new family provider provider-aws-kafkaconnect, new resources, bug fixes, and dependency updates.

This release also introduces a credential cache for IRSA authentication, which greatly reduces the number of AWS STS calls the provider makes. This cache is currently only employed for IRSA configurations. Please refer to the description here for the results of some experiments and the observed improvements in those experiments.

Also in this release, we add the API call counters for the provider. An example for the new metrics is as follows:

# HELP upjet_resource_external_api_calls_total The number of external API calls.
# TYPE upjet_resource_external_api_calls_total counter
upjet_resource_external_api_calls_total{operation="AssumeRole",service="STS"} 2
upjet_resource_external_api_calls_total{operation="AssumeRoleWithWebIdentity",service="STS"} 1
upjet_resource_external_api_calls_total{operation="CreateRole",service="IAM"} 1
upjet_resource_external_api_calls_total{operation="GetCallerIdentity",service="STS"} 1
upjet_resource_external_api_calls_total{operation="GetRole",service="IAM"} 61
upjet_resource_external_api_calls_total{operation="GetRolePolicy",service="IAM"} 60
upjet_resource_external_api_calls_total{operation="ListAttachedRolePolicies",service="IAM"} 60
upjet_resource_external_api_calls_total{operation="ListRolePolicies",service="IAM"} 60
upjet_resource_external_api_calls_total{operation="PutRolePolicy",service="IAM"} 1

Please refer to the description of this PR for the details.

Support for New Resources

• User.memorydb.aws.upbound.io/v1beta1
• Connector.kafkaconnect.aws.upbound.io/v1beta1
• CustomPlugin.kafkaconnect.aws.upbound.io/v1beta1
• WorkerConfiguration.kafkaconnect.aws.upbound.io/v1beta1

Bug Fixes

• [Bug]: IAM Role inlinePolicy conflicting with IAM RolePolicy #1207
• Excessive calls to AssumeRoleWithWebIdentity w/ IRSA #997

What's Changed

• Add Matt Bush (mbbush) as maintainer by @jeanduplessis in #1214
• Generate license headers for setup.go files by consuming the latest commit of upjet by @sergenyalcin in #1216
• fix(example): remove spec.forProvider.name for iam policy examples by @haarchri in #1219
• Add region parameter back to TF setup.Configuration by @erhancagirici in #1221
• Log a deprecation message when the monolithic provider is run by @sergenyalcin in #1230
• Update kubernetes patches by @renovate in #1201
• Update actions/checkout digest to b4ffde6 by @renovate in #1206
• Update actions/cache digest by @renovate in #1205
• Update docker/setup-buildx-action digest to 2b51285 by @renovate in #1236
• Update fkirc/skip-duplicate-actions action to v5.3.1 by @renovate in #1237
• Add memorydb _user by @stevendborrelli in #1170
• Update authentication document for WebIdentity by @turkenf in #1171
• Update actions/cache action to v4 by @renovate in #1238
• Update actions/setup-go action to v5 by @renovate in #1239
• Skip late initialization for several duplicate resource policy fields by @mbbush in #1213
• Count external API calls by @mergenci in #1241
• Cache AWS Config's CredentialsProvider to reduce STS calls by @erhancagirici in #1235
• Add support for msk connect resources by @mbbush in #1162
• Use Generation instead of ResourceVersion when computing the cache key by @ulucinar in #1244

Full Changelog: v1.2.1...v1.3.0

v1.2.1

The release v1.2.1 includes a bug fix: [Bug]: SNS Topics Not Creating #1220

What's Changed

• [Backport release-1.2] Add region parameter back to TF setup.Configuration by @github-actions in #1222

Full Changelog: v1.2.0...v1.2.1

v1.2.0

The release v1.2.0 includes bug fixes, and updating dependencies, with the most significant ones detailed below.

• Sets a default io.Discard logger for the controller-runtime if debug logging is not enabled which fixes #974 and #854. If debug logging is enabled, then the controller-runtime uses a debug mode zap logger as usual.
• Refactors AWS client configuration logic with a single path #1204. This refactoring also leads to a 50% reduction in the number of STS calls made per reconcile.
• Adds ability to auto-generate password in referenced secret for Cluster.rds resource #1169
• Fixes the update loop in Firewall.networkfirewall resource #1167

Breaking UX Changes

The release switches the order of the SYNCED and READY printer columns so that the SYCNED column is now printed before the READY column as follows:

❯ kubectl get managed

NAME                                SYNCED   READY   EXTERNAL-NAME           AGE
vpc.ec2.aws.upbound.io/sample-vpc   True     True    vpc-0c4b6ee9568fe7a16   30s

This used to be first the READY column and then the SYNCED column previously. Please see crossplane/upjet#360 for further details.

What's Changed

• Update kubernetes patches by @renovate in #1152
• Fix the update loop in Firewall.networkfirewall resource by @ulucinar in #1167
• Pin the UXP version in Makefile by @sergenyalcin in #1190
• Update docker/setup-buildx-action digest to 0d103c3 by @renovate in #1176
• Update actions/cache digest to ab5e6d0 by @renovate in #1183
• Consume StateFunc fix by @sergenyalcin in #1188
• rds.cluster: add ability to auto-generate password in referenced secret by @ravibagri5 in #1169
• Decrease linter's memory usage by @ulucinar in #1194
• Consume the reusable workflows from upbound/uptest@standard-runners by @ulucinar in #1180
• Run the "Cleanup Disk" step in the publish-service-artifacts CI job by @ulucinar in #1209
• Add a "Disk Cleanup" step as the first step in the lint job by @ulucinar in #1211
• Unify TF AWS provider configuration logic by @erhancagirici in #1204
• Set log.Default's output to io.Discard by @ulucinar in #1203
• Applying license changes by @sergenyalcin in #1212

New Contributors

• @ravibagri5 made their first contribution in #1169

Full Changelog: v1.1.1...v1.2.0

v0.47.4

The release 0.47.4 sets a default io.Discard logger for the controller-runtime if debug logging is not enabled which fixes #974 and #854. If debug logging is enabled, then the controller-runtime uses a debug mode zap logger as usual.

What's Changed

• Set log.Default's output to io.Discard by @ulucinar in #1215

Full Changelog: v0.47.3...v0.47.4

v1.1.1

The release v1.1.1 includes two important bug fixes detailed below:

• The first fix is related to sensitive fieldpath generation for slice-type fields. In the current behavior, we do not have any handling mechanism for slices while calculating the fieldpath for sensitive fields. With this PR we will have this and correctly generate the fieldpaths for sensitive slice typed fields. We started to generate the correct fieldpath for the passwordsSecretRef field of the elasticache.User resource. In this way, we will correctly resolve the sensitive data for this field. Related upstream fix: crossplane/upjet#355
• The second fix removes the applying of StateFuncs to parameters in the upstream with this PR. We noticed that the user_data field of the ec2.Instance resource does not work properly. Please see this issue. The related field started working correctly with the fix of a bug we found in Upjet.

What's Changed

• [release-1.1] Bump upjet to 1.2.2 by @sergenyalcin in #1196

Full Changelog: v1.1.0...v1.1.1

v0.47.3

The release v0.47.3 includes two important bug fixes detailed below:

• The first fix is related to sensitive fieldpath generation for slice-type fields. In the current behavior, we do not have any handling mechanism for slices while calculating the fieldpath for sensitive fields. With this PR we will have this and correctly generate the fieldpaths for sensitive slice typed fields. We started to generate the correct fieldpath for the passwordsSecretRef field of the elasticache.User resource. In this way, we will correctly resolve the sensitive data for this field. Related upstream fix: crossplane/upjet#355
• The second fix removes the applying of StateFuncs to parameters in the upstream with this PR. We noticed that the user_data field of the ec2.Instance resource does not work properly. Please see this issue. The related field started working correctly with the fix of a bug we found in Upjet.

What's Changed

• Reclaim disk space in an initial step while running the CI jobs by @ulucinar in #1197
• [release-0.47] Bump upjet to 1.1.3 by @sergenyalcin in #1195

Full Changelog: v0.47.2...v0.47.3

v0.47.2

The release v0.47.2 includes some important bug fixes & dependency bumps detailed below:

• Bump upjet to v1.1.1
• Bump crossplane-runtime to commit 7fcb8c5cad6f
• Bump uptest to v0.11.0
• Fixes an update loop issue for the Firewall.networkfirewall resource #1160
• AWS Secrets Manager Provider Constantly Adds and Deletes Regional Replication #1079
• [Bug]: Secret never Sync #1128
• v0.46.0 broke localstack config #1063

What's Changed

• [Backport release-0.47] fix Secret/secretmanagers.aws update loop for replica field and custom diff logic when replica config is empty by @turkenf in #1158
• [Backport release-0.47] Make endpoint provider configuration value single-element list by @turkenf in #1156
• Bump upjet to v1.1.1 by @ulucinar in #1160
• [Backport release-0.47] tests and testing changes by @mbbush in #1161

Full Changelog: v0.47.1...v0.47.2

v1.1.0

The v1.1.0 release introduces a new family provider, new resources, important bug fixes, dependency updates, and a new ProviderConfig API regarding the WebIdentity authentication.

Support for New Resources

• StackSetInstance.cloudformation.aws.upbound.io/v1beta1
• AccessPolicy.opensearchserverless.aws.upbound.io/v1beta1
• Collection.opensearchserverless.aws.upbound.io/v1beta1
• LifecyclePolicy.opensearchserverless.aws.upbound.io/v1beta1
• SecurityConfig.opensearchserverless.aws.upbound.io/v1beta1
• SecurityPolicy.opensearchserverless.aws.upbound.io/v1beta1
• VPCEndpoint.opensearchserverless.aws.upbound.io/v1beta1

Bug Fixes

We also have several bug fixes with this release:

• [Bug]: Secret never Sync #1128
• [Bug]: Can't install provider-aws 1.0.0 on crossplane versions before 1.14 #1147
• There’s also a fix in the implementation of the LateInitialize management policy in the context of the no-fork architecture. Previously, we were relying solely on the managed reconciler for preventing the late-initialization of the managed resources if the specified policies do not contain it. Now, we also explicitly skip the late-initialization in upjet to prevent some accidental updates to the spec.forProvider while, for example, updating the annotations.

New API for WebIdentity authentication support in ProviderConfig

It’s now possible to specify the WebIdentity tokens to be used in ProviderConfigs for WebIdentity authentication. Previously, it was only possible to do so via the environment variables.

ProviderConfig API specification is expanded with spec.credentials.webIdentity.tokenConfig, which allows consumers to configure the token to be used for WebIdentity authentication.
Consumers can reference a secret or filesystem location for the token to be used for WebIdentity authentication.

• Each ProviderConfig using WebIdentity authentication can now use different tokens per ProviderConfig object, allowing multiple WebIdentity configurations in a single cluster.

• ℹ️ The change is backward compatible for consumers relying on the old behavior where they set both of the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables. When spec.credentials.webIdentity.tokenConfig is not specified, the old behavior is assumed.

• ⚠️ Deprecation Notice: Configuring the WebIdentity authentication using the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables is now deprecated in favor of the new spec.credentials.webIdentity.tokenConfig API.

An example WebIdentity token configuration where the token is read from a Kubernetes secret is as follows:

apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: webidentity-example
spec:
  credentials:
    source: WebIdentity
    webIdentity:
      roleARN: arn:aws:iam::123456789012:role/providerexamplerole
      tokenConfig:
        source: Secret
        secretRef:
          key: token
          name: example-web-identity-token-secret
          namespace: upbound-system

Another example using a filesystem location is as follows:

apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: webidentity-example
spec:
  credentials:
    source: WebIdentity
    webIdentity:
      roleARN: arn:aws:iam::123456789012:role/providerexamplerole
      tokenConfig:
        source: Filesystem
        fs:
          path: /path/to/token/file

Please note that the Filesystem source option needs the token to be mounted as a file in the filesystem of the provider pod, e.g,. via a DeploymentRuntimeConfig.

The difference is that the new API effectively allows specifying the token per ProviderConfig.

Changes in conversion Webhook TLS Certificate & Key Configuration

This PR implements the following protocol for configuring the conversion Webhook TLS certificate & key for the provider:

If the --certs-dir command-line option is supplied, it's used.
If the --certs-dir command-line option is not supplied, the following environment variables are used in the given order: CERTS_DIR (for backwards-compatibility reasons), TLS_SERVER_CERTS_DIR (the new environment variable, which has replaced the WEBHOOK_TLS_CERT_DIR env. variable in Crossplane), and WEBHOOK_TLS_CERT_DIR (for backwards-compatibility).
This change in the configuration API is backwards-compatible.

What's Changed

• complete example for securitygrouprule with self=null by @mbbush in #1072
• Update kubernetes patches to v0.29.1 by @renovate in #1094
• Update actions/upload-artifact digest to 5d5d22a by @renovate in #1125
• Add aws_cloudformation_stack_set_instance resource by @sergenyalcin in #1129
• Update alpine Docker tag to v3.19.1 by @renovate in #1127
• Update dependency kubernetes-sigs/kind to v0.21.0 by @renovate in #1143
• Function type config.ConfigurationInjector now returns an error by @ulucinar in #1150
• fix Secret/secretmanager.aws custom diff logic when replica config is empty by @erhancagirici in #1144
• Honour the WEBHOOK_TLS_CERT_DIR env. variable for finding the Webhook TLS certificate by @ulucinar in #1157
• Add opensearch serverless group resources to v1beta1 by @turkenf in #1130
• add web identity token configuration to ProviderConfig spec by @erhancagirici in #1148
• Bump crossplane-runtime to commit 7fcb8c5cad6f by @ulucinar in #1159

Full Changelog: v1.0.0...v1.1.0