Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: federated learning use case implementation #70

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

ferrarimarco
Copy link
Member

This commit introduces the Federated Learning core platform use case.

@ferrarimarco ferrarimarco changed the title feat: federated learning use case implementation (#49) feat: federated learning use case implementation Dec 12, 2024
@ferrarimarco
Copy link
Member Author

@arueth Can you please rebase int-federated-learning on top of main? That would be helpful for this branch as well. Thanks!

@ferrarimarco
Copy link
Member Author

PS: the only conflict is about a typo I fixed in platforms/gke/base/_shared_config/scripts/set_environment_variables.sh. It's no longer applicable.

@ferrarimarco ferrarimarco self-assigned this Dec 20, 2024
@arueth arueth force-pushed the int-federated-learning branch from 9123bf4 to e5385e4 Compare December 20, 2024 15:51
@arueth arueth force-pushed the int-federated-learning branch 2 times, most recently from 2bac60b to 9588c0b Compare January 14, 2025 16:42
This commit introduces the Federated Learning core platform use case.

As a first step to get a feeling of how it is to integrate and existing
use case with the platform, we provision only a simple resource (an
Artifact Registry repository).
Configure Private Google Access for the federated learning use case
* chore: simplify fl scripts

* chore: remove initialize terraservice in fl

Remove the 'initialize' terraservice in the Federated Learning use case
because the core platform 'initialize' terraservice takes care of
configuring backends for use cases after #71 is merged.

The only task implemented in the use case 'initialize' terraservice was
to initialize backend configuration, so we don't need it anymore.

Also, simplify provisioning and teardown scripts because we don't need
two different terraform init commands anymore because now all the
terraservices in the use case work with a remote backend.
Configure the GKE cluster for the federated learning use case
- configure firewall for federated learning
- configure iam roles and service accounts
- configure dedicated node pools
- configure policy controller and policies
- configure dedicated Kubernetes namespaces
@arueth arueth force-pushed the int-federated-learning branch from 15a8eb2 to a39221f Compare January 17, 2025 19:16
@ferrarimarco
Copy link
Member Author

ferrarimarco commented Jan 24, 2025

@arueth The build is failing because despite the KMS API being enabled, it likely needs time to set resources up.

Did we already solve this issue in the core platform? It seems like that other APIs might have this problem too, but we didn't notice it until we used the same project for CI.

If we didn't solve it yet, I can look into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant