Reconcile Route53 ResourceRecordSet Dependencies When Applying Multiple Resources

[phantasm66]

It would be great if Crossplane could reconcile Route53 dependencies when applying multiple resources at the same time.

For example:

The following manifest defines multiple resources for Crossplane to provision an AWS Security Group, ElastiCache (redis) cluster and a vanity DNS CNAME Route53 record:

apiVersion: ec2.aws.crossplane.io/v1beta1
kind: SecurityGroup
metadata:
  name: redis-cache-kubernetes-lab
spec:
  providerConfigRef:
    name: default
  forProvider:
    description: "testing crossplane"
    region: us-east-1
    vpcId: vpc-abc123
    groupName: redis-cache-kubernetes-lab
    ingress:
      - fromPort: 6379
        toPort: 6379
        ipProtocol: tcp
        ipRanges:
          - cidrIp: 10.208.0.0/16

---

apiVersion: cache.aws.crossplane.io/v1alpha1
kind: CacheCluster
metadata:
  name: redis-sidekiq
spec:
  providerConfigRef:
    name: default
  forProvider:
    region: us-east-1
    engine: redis
    engineVersion: "5.0.6"
    cacheParameterGroupName: redis-50-staging-useast1
    cacheNodeType: cache.t2.small
    numCacheNodes: 2
    cacheSubnetGroupNameRef:
      name: kubernetes-cache-lab
    securityGroupIdRefs:
      - name: redis-cache-kubernetes-lab

---

apiVersion: route53.aws.crossplane.io/v1alpha1
kind: ResourceRecordSet
metadata:
  name: redis-sidekiq.my-domain.com
spec:
  providerConfigRef:
    name: default
  forProvider:
    type: CNAME
    ttl: 300
    resourceRecords:
      - value: ${the AWS generated endpoint FQDN of the CacheCluster resource}
    zoneIdRef:
      name: my-domain.com

The advantage of applying everything all at the same time is that we can template out all resources for an application and apply/provision everything in a single automated deployment pipeline.

The problem with the above manifest is that the spec.forProvider.resourceRecords.value is not actually known until after the CacheCluster resource is created and AWS has generated an endpoint FQDN we can then reference in this ResourceRecordSet resource.

Ideally, this feature should be generic enough that a ResourceRecordSet's spec.forProvider.resourceRecords.value reference to a dependency resource being applied at the same time will work for CacheCluster, RDSInstance, etc...

[muvaf]

There are two possible solutions to this:

• We have the usual reference&selector fields but they are strongly types, so you can't use different managed resource kinds for the same field. We can have an additional way in references where you can specify the type as well assuming provider-aws knows how to take the value from that type, too. Additionally, there could be a DSL-like way of saying go get this field from that type and format it that way.
• In Composition, we have that patch & format mechanism. However, it doesn't support inter-resource value propagation yet. But once Composite resources should support bidirectional patching crossplane/crossplane#1639 is implemented, users will be able to say RDSInstance.status.atProvider.fqdn should be propagated to composite.status.atProvider.rdsURL and ResoruceRecordSet.spec.forProvider.resourceRecords[0].value should be patched by the value in composite.status.atProvider.rdsURL after applying some format transforms.

I believe the second solution is cleaner and keeps the managed resources simple, however, it works only if you use composition. To cover your use case, I believe hard-coded sets (there would be a set of kinds you can refer, similar to today but you control the kind as well) could be better compared to a small DSL.

[chlunde]

It looks like this issue will be fixed once crossplane/crossplane#2180 is merged

[haarchri]

@phantasm66 is it still relevant since crossplane/crossplane#2180 and crossplane/crossplane#1639 are merged it is possible and we also use this with other resources like your example

[phantasm66]

I’m good. I’ll close this….

[jcputter]

@muvaf are there any docs or examples of how to do this? i'm trying to create an rds via composition and create a vanity dns for it

[ashishkurian]

@muvaf or @haarchri

I have created a NLB using the LoadBalancer CRD. But now I want to attach a specific DNS entry to it. How can I acheive that using the ResourceRecordSet CRD.

I could not find any documentation anywhere as to how to achieve that connection using the resource name or selectors.