Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit one session per user per pool #669

Open
olevski opened this issue Feb 24, 2025 · 7 comments
Open

Limit one session per user per pool #669

olevski opened this issue Feb 24, 2025 · 7 comments

Comments

@olevski
Copy link
Member

olevski commented Feb 24, 2025

This requires some design or a pitch.

But a crude flag or something that will limit so that the same user can only launch 1 session in the same pool would be useful. Especially where large number of students in a class start using renku and launch many different sessions.

Potentially we can apply this or a similar protection in the default resource pool.

@olevski
Copy link
Member Author

olevski commented Feb 24, 2025

Ah thanks @leafty I was looking for an issue for this. Because I vaguely remember talking about it.

@lokijuhy
Copy link
Member

lokijuhy commented Mar 4, 2025

I think we should consider putting the restriction at the resource class level, not the resource pool level. Consider a use case where in a course resource pool there is a CPU-only resource class and a GPU resource class. We want to be able to put a restriction on GPU sessions, but without handicapping the user in terms of cpu-only sessions. Similarly, we have the same situation for the SDSC resource pool, where the critical resource class to restrict is the GPU resource class, but we want people to be able to have more than one session using the CPU-only resource classes there.
Thoughts?

@leafty
Copy link
Member

leafty commented Mar 4, 2025

If we do this at the class level, it will be very tedious for admins to set the limit for each and every class. I think it is preferable to add a similar mechanism but for GPUs: set a limit of GPUs in use per pool per user. Doing things at the class level would not prevent someone from starting two sessions with two different classes which both have a GPU attached. But if we add a maxGPUsPerUser limit at the resource pool level then it is easy to control usage.
Control at the pool level is what makes sense to me: administrators provision pools of resources, so it is a lot easier for them to think in terms of controlling usage per pool.

@leafty
Copy link
Member

leafty commented Mar 4, 2025

I will also add that we do have in production cases where a resource pool has multiple classes with GPUs to cater to different uses. If the limit is set at the class level, then a single user can hold on to X GPUs where X is the number of classes with GPUs attached. It becomes impossible to limit that number to only one GPU per user in that resource pool.

@lokijuhy
Copy link
Member

lokijuhy commented Mar 6, 2025

That makes sense to me. So it sounds like what you're recommending is not a broad-strokes session-per-user-per-pool limit, but a resource-quota-per-user-per-pool limit - is that right? If so, I am in agreement.

@leafty
Copy link
Member

leafty commented Mar 6, 2025

Yes, the original idea of using a session as a unit for limits was to have something simple to implement even if it is coarse. If we need more control, we can update the idea to allow limiting resource usage per user per resource (CPU, RAM, disk, GPU).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants