New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KernelCI mini-CDN (storage solution) #442

Open

nuclearcat opened this issue Sep 4, 2024 · 0 comments

Labels

Member

nuclearcat commented Sep 4, 2024

Right now we are using Azure files storage, which is not very good for storing artifacts by multiple reasons:

Very slow API to upload, list, delete many files
We have no control over storage on downloads, we cannot check if someone have improper caching, or if someone is downloading too much
Generated URL is "ugly" and not very user friendly
Generally cloud storage is very expensive

We need to investigate better storage solution, that is likely will implement function of "mini-CDN".

Keep "hot" artifacts in fast storage (block device)
Keep "cold" artifacts that might be used from Azure in Azure files mounted by NFS to "CDN" VM
Keep "cold" artifacts that might be used by other users on some cheap storages - Hetzner, etc.

This subject need to be investigated, and probably idea is improved.
Another prior research: #310

nuclearcat added the techdebt label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment