Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NodeGetInfo returns static max_volumes_per_node #6

Open
Mattes83 opened this issue Jun 5, 2024 · 1 comment
Open

NodeGetInfo returns static max_volumes_per_node #6

Mattes83 opened this issue Jun 5, 2024 · 1 comment
Labels
enhancement New feature or request kind/feature Categorizes issue or PR as related to a new feature.

Comments

@Mattes83
Copy link

Mattes83 commented Jun 5, 2024

Describe the solution you'd like:
It is the csi drivers task to inform the scheduler about the maximum number of volumes that can be attached to a node. Currently this seems to be a static value. It works as long as the nodes have only one network interface.
In CAPIC you can specify additional networks (https://github.com/ionos-cloud/cluster-api-provider-ionoscloud/blob/main/api/v1alpha1/ionoscloudmachine_types.go#L156).
In our case we are having multiple network interfaces. This leads to problems because pods are scheduled to nodes where no more volumes can be attached.

To fix this you could either take into account the number of nics attach when responding to NodeGetInfo.max_volumes_per_node or simply make this value configurable.

Anything else you would like to add:
This is a flaw in the underlying csi driver. As the source code of the csi driver is not public yet I am posting this here. It would be great if you can forward this to the appropriate people.

additional context:
https://kubernetes.io/docs/concepts/storage/storage-limits/#dynamic-volume-limits
https://github.com/container-storage-interface/spec/blob/master/spec.md#nodegetinfo

@Mattes83 Mattes83 added enhancement New feature or request kind/feature Categorizes issue or PR as related to a new feature. labels Jun 5, 2024
@avorima
Copy link
Collaborator

avorima commented Jun 6, 2024

NodeGetInfo returns the upper bound of possible volumes that can be attached, but limits are checked for before each volume creation to prevent the situation you're talking about.

If possible, provide your cluster template as well as your helm values to see whether it's a misconfiguration. I might've missed something that we configure internally for the open source helm chart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

2 participants