Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support separate subnets for LBs and workers #827

Merged
merged 4 commits into from
Oct 28, 2024

Conversation

mikkeloscar
Copy link
Contributor

@mikkeloscar mikkeloscar commented Oct 25, 2024

Add support for choosing different subnets for running Load Balancers and nodes. Until now they all used the same subnets leading to challenges when wanting to use different ingress and egress rules.

The subnets are selected based on tags:

  • LB: kubernetes.io/role/elb
  • Nodes: kubernetes.io/role/node

For safe roll out, we will first add the kubernetes.io/role/node tag to existing subnets with the tag: kubernetes.io/role/elb such that there is no change for running clusters as a change will require a migration.

@mikkeloscar mikkeloscar added minor Minor changes, e.g. low risk config updates, changes that do not introduce a new API call. do-not-merge labels Oct 25, 2024
Signed-off-by: Mikkel Oscar Lyderik Larsen <[email protected]>
Signed-off-by: Mikkel Oscar Lyderik Larsen <[email protected]>
Signed-off-by: Mikkel Oscar Lyderik Larsen <[email protected]>
subnetsValueKey: azInfoWorkers.SubnetsByAZ(),
availabilityZonesValueKey: azInfoWorkers.AvailabilityZones(),
subnetIPV6CIDRsKey: strings.Join(azInfoWorkers.SubnetIPv6CIDRs(), ","),
"lb_subnets": azInfoLBs.SubnetsByAZ(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is new, do we need in a similar change in provisionNodePool ?

Copy link
Member

@AlexanderYastrebov AlexanderYastrebov Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is handled by azInfo: azInfoWorkers change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be used like this: zalando-incubator/kubernetes-on-aws#8440

@AlexanderYastrebov
Copy link
Member

👍

Signed-off-by: Mikkel Oscar Lyderik Larsen <[email protected]>
@AlexanderYastrebov
Copy link
Member

👍

@mikkeloscar
Copy link
Contributor Author

This change require that we set kubernetes.io/role/node tag on existing subnets to be backwards compatible.

I have tested what happens if we should have forgotten to set the tags in an existing cluster. The CF stack fails with:

image

And rolls back without disruption. So there is no impact if it should be the case.

@mikkeloscar
Copy link
Contributor Author

👍

@mikkeloscar mikkeloscar merged commit 8fa036f into master Oct 28, 2024
8 of 10 checks passed
@mikkeloscar mikkeloscar deleted the split-lb-worker-subnets branch October 28, 2024 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor Minor changes, e.g. low risk config updates, changes that do not introduce a new API call.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants