Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement cmpd's PolicyRules #8328

Merged
merged 40 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
17e60fc
wip
cjc7373 Oct 22, 2024
8a37f6b
cleanup
cjc7373 Oct 22, 2024
8dfd349
wip
cjc7373 Oct 23, 2024
a80d8f0
cleanup kubeblocks-kb-agent-pod-role
cjc7373 Oct 23, 2024
092e869
refactor
cjc7373 Oct 23, 2024
7169d42
unit test
cjc7373 Oct 24, 2024
1aea7fe
fix tests
cjc7373 Oct 24, 2024
6282bd3
fix tests
cjc7373 Oct 24, 2024
b262349
cleanup
cjc7373 Oct 25, 2024
ad2e09c
api doc
cjc7373 Oct 25, 2024
a57f03b
static check
cjc7373 Oct 25, 2024
618a559
doc
cjc7373 Oct 25, 2024
b3e4c6f
cleanup
cjc7373 Oct 25, 2024
208cf73
fix: wrong kind
cjc7373 Oct 28, 2024
155b67a
fix
cjc7373 Oct 31, 2024
f472d6f
clean old resources
cjc7373 Nov 12, 2024
ea9c0a7
review comments
cjc7373 Nov 14, 2024
8416e61
fix test
cjc7373 Nov 14, 2024
7581dc6
typo
cjc7373 Nov 14, 2024
e5587b3
Merge remote-tracking branch 'origin/main' into feature/rbac-for-comp…
cjc7373 Nov 22, 2024
1b7bc7c
clean code
cjc7373 Nov 22, 2024
7235c5e
Merge remote-tracking branch 'origin/main' into feature/rbac-for-comp…
cjc7373 Nov 25, 2024
5e68101
Merge remote-tracking branch 'origin/main' into feature/rbac-for-comp…
cjc7373 Nov 26, 2024
287bb06
refine description
cjc7373 Nov 28, 2024
70deaa9
change sa naming rule, share rbac resources among components
cjc7373 Nov 28, 2024
8a5a122
fix unit test
cjc7373 Nov 29, 2024
f55de76
typo
cjc7373 Nov 29, 2024
3bceb13
fix test
cjc7373 Nov 29, 2024
8fc1b2f
Merge remote-tracking branch 'origin/main' into feature/rbac-for-comp…
cjc7373 Dec 3, 2024
e04467b
clean up
cjc7373 Dec 3, 2024
9e56c89
clean up
cjc7373 Dec 3, 2024
52354b6
Merge remote-tracking branch 'origin/main' into feature/rbac-for-comp…
cjc7373 Jan 7, 2025
d32f3cd
review comments
cjc7373 Jan 9, 2025
5d97cb9
skip deleting component
cjc7373 Jan 9, 2025
8875d00
remove transitioning code
cjc7373 Jan 9, 2025
e54b7cc
remove old test
cjc7373 Jan 9, 2025
8283c96
Merge remote-tracking branch 'origin/main' into feature/rbac-for-comp…
cjc7373 Jan 14, 2025
bcab487
fix lint error
cjc7373 Jan 14, 2025
3851918
review comments
cjc7373 Jan 20, 2025
d1262f4
exclude comp labels when updating labels
cjc7373 Jan 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions apis/apps/v1/cluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -404,17 +404,15 @@ type ClusterComponentSpec struct {
// This ServiceAccount is used to grant necessary permissions for the Component's Pods to interact
// with other Kubernetes resources, such as modifying Pod labels or sending events.
//
// Defaults:
// To perform certain operational tasks, agent sidecars running in Pods require specific RBAC permissions.
// The service account will be bound to a default role named "kubeblocks-cluster-pod-role" which is installed together with KubeBlocks.
// If not specified, KubeBlocks automatically assigns a default ServiceAccount named "kb-{cluster.name}"
//
// Future Changes:
// Future versions might change the default ServiceAccount creation strategy to one per Component,
// potentially revising the naming to "kb-{cluster.name}-{component.name}".
//
// Users can override the automatic ServiceAccount assignment by explicitly setting the name of
// an existed ServiceAccount in this field.
// If not specified, KubeBlocks automatically creates a default ServiceAccount named
// "kb-{componentdefinition.name}", bound to a role with rules defined in ComponentDefinition's
// `policyRules` field. If needed (currently this means if any lifecycleAction is enabled),
// it will also be bound to a default role named
// "kubeblocks-cluster-pod-role", which is installed together with KubeBlocks.
// If multiple components use the same ComponentDefinition, they will share one ServiceAccount.
//
// If the field is not empty, the specified ServiceAccount will be used, and KubeBlocks will not
// create a ServiceAccount. But KubeBlocks does create RoleBindings for the specified ServiceAccount.
//
// +optional
ServiceAccountName string `json:"serviceAccountName,omitempty"`
Expand Down
19 changes: 9 additions & 10 deletions apis/apps/v1/component_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,16 +175,15 @@ type ComponentSpec struct {
// This ServiceAccount is used to grant necessary permissions for the Component's Pods to interact
// with other Kubernetes resources, such as modifying Pod labels or sending events.
//
// Defaults:
// If not specified, KubeBlocks automatically assigns a default ServiceAccount named "kb-{cluster.name}",
// bound to a default role defined during KubeBlocks installation.
//
// Future Changes:
// Future versions might change the default ServiceAccount creation strategy to one per Component,
// potentially revising the naming to "kb-{cluster.name}-{component.name}".
//
// Users can override the automatic ServiceAccount assignment by explicitly setting the name of
// an existed ServiceAccount in this field.
// If not specified, KubeBlocks automatically creates a default ServiceAccount named
// "kb-{componentdefinition.name}", bound to a role with rules defined in ComponentDefinition's
// `policyRules` field. If needed (currently this means if any lifecycleAction is enabled),
// it will also be bound to a default role named
// "kubeblocks-cluster-pod-role", which is installed together with KubeBlocks.
// If multiple components use the same ComponentDefinition, they will share one ServiceAccount.
//
// If the field is not empty, the specified ServiceAccount will be used, and KubeBlocks will not
// create a ServiceAccount. But KubeBlocks does create RoleBindings for the specified ServiceAccount.
//
// +optional
ServiceAccountName string `json:"serviceAccountName,omitempty"`
Expand Down
2 changes: 0 additions & 2 deletions apis/apps/v1/componentdefinition_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -483,8 +483,6 @@ type ComponentDefinitionSpec struct {
// for the Component based on the specified policy rules.
// This ensures that the Pods in the Component has appropriate permissions to function.
//
// Note: This field is currently non-functional and is reserved for future implementation.
//
// This field is immutable.
//
// +optional
Expand Down
38 changes: 16 additions & 22 deletions config/crd/bases/apps.kubeblocks.io_clusters.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5077,19 +5077,16 @@ spec:
with other Kubernetes resources, such as modifying Pod labels or sending events.


Defaults:
To perform certain operational tasks, agent sidecars running in Pods require specific RBAC permissions.
The service account will be bound to a default role named "kubeblocks-cluster-pod-role" which is installed together with KubeBlocks.
If not specified, KubeBlocks automatically assigns a default ServiceAccount named "kb-{cluster.name}"
If not specified, KubeBlocks automatically creates a default ServiceAccount named
"kb-{componentdefinition.name}", bound to a role with rules defined in ComponentDefinition's
`policyRules` field. If needed (currently this means if any lifecycleAction is enabled),
it will also be bound to a default role named
"kubeblocks-cluster-pod-role", which is installed together with KubeBlocks.
If multiple components use the same ComponentDefinition, they will share one ServiceAccount.


Future Changes:
Future versions might change the default ServiceAccount creation strategy to one per Component,
potentially revising the naming to "kb-{cluster.name}-{component.name}".


Users can override the automatic ServiceAccount assignment by explicitly setting the name of
an existed ServiceAccount in this field.
If the field is not empty, the specified ServiceAccount will be used, and KubeBlocks will not
create a ServiceAccount. But KubeBlocks does create RoleBindings for the specified ServiceAccount.
type: string
serviceRefs:
description: |-
Expand Down Expand Up @@ -13827,19 +13824,16 @@ spec:
with other Kubernetes resources, such as modifying Pod labels or sending events.


Defaults:
To perform certain operational tasks, agent sidecars running in Pods require specific RBAC permissions.
The service account will be bound to a default role named "kubeblocks-cluster-pod-role" which is installed together with KubeBlocks.
If not specified, KubeBlocks automatically assigns a default ServiceAccount named "kb-{cluster.name}"
If not specified, KubeBlocks automatically creates a default ServiceAccount named
"kb-{componentdefinition.name}", bound to a role with rules defined in ComponentDefinition's
`policyRules` field. If needed (currently this means if any lifecycleAction is enabled),
it will also be bound to a default role named
"kubeblocks-cluster-pod-role", which is installed together with KubeBlocks.
If multiple components use the same ComponentDefinition, they will share one ServiceAccount.


Future Changes:
Future versions might change the default ServiceAccount creation strategy to one per Component,
potentially revising the naming to "kb-{cluster.name}-{component.name}".


Users can override the automatic ServiceAccount assignment by explicitly setting the name of
an existed ServiceAccount in this field.
If the field is not empty, the specified ServiceAccount will be used, and KubeBlocks will not
create a ServiceAccount. But KubeBlocks does create RoleBindings for the specified ServiceAccount.
type: string
serviceRefs:
description: |-
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8331,9 +8331,6 @@ spec:
This ensures that the Pods in the Component has appropriate permissions to function.


Note: This field is currently non-functional and is reserved for future implementation.


This field is immutable.
items:
description: |-
Expand Down
18 changes: 8 additions & 10 deletions config/crd/bases/apps.kubeblocks.io_components.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4841,18 +4841,16 @@ spec:
with other Kubernetes resources, such as modifying Pod labels or sending events.


Defaults:
If not specified, KubeBlocks automatically assigns a default ServiceAccount named "kb-{cluster.name}",
bound to a default role defined during KubeBlocks installation.


Future Changes:
Future versions might change the default ServiceAccount creation strategy to one per Component,
potentially revising the naming to "kb-{cluster.name}-{component.name}".
If not specified, KubeBlocks automatically creates a default ServiceAccount named
"kb-{componentdefinition.name}", bound to a role with rules defined in ComponentDefinition's
`policyRules` field. If needed (currently this means if any lifecycleAction is enabled),
it will also be bound to a default role named
"kubeblocks-cluster-pod-role", which is installed together with KubeBlocks.
If multiple components use the same ComponentDefinition, they will share one ServiceAccount.


Users can override the automatic ServiceAccount assignment by explicitly setting the name of
an existed ServiceAccount in this field.
If the field is not empty, the specified ServiceAccount will be used, and KubeBlocks will not
create a ServiceAccount. But KubeBlocks does create RoleBindings for the specified ServiceAccount.
type: string
serviceRefs:
description: |-
Expand Down
37 changes: 3 additions & 34 deletions controllers/apps/cluster/transformer_cluster_deletion.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,13 @@ package cluster

import (
"fmt"
"reflect"
"strings"
"time"

"golang.org/x/exp/maps"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
"k8s.io/apimachinery/pkg/util/sets"
"k8s.io/klog/v2"
"sigs.k8s.io/controller-runtime/pkg/client"

appsv1 "github.com/apecloud/kubeblocks/apis/apps/v1"
Expand Down Expand Up @@ -129,7 +128,7 @@ func (t *clusterDeletionTransformer) Transform(ctx graph.TransformContext, dag *
delKindMap := map[string]sets.Empty{}
for _, o := range delObjs {
// skip the objects owned by the component and InstanceSet controller
if shouldSkipObjOwnedByComp(o, *cluster) || isOwnedByInstanceSet(o) {
if isOwnedByComp(o) || isOwnedByInstanceSet(o) {
continue
}
graphCli.Delete(dag, o, inUniversalContext4G())
Expand All @@ -138,6 +137,7 @@ func (t *clusterDeletionTransformer) Transform(ctx graph.TransformContext, dag *

// set cluster action to status until all the sub-resources deleted
if len(delObjs) == 0 {
transCtx.Logger.Info(fmt.Sprintf("deleting cluster %v", klog.KObj(cluster)))
graphCli.Delete(dag, cluster)
} else {
transCtx.Logger.Info(fmt.Sprintf("deleting the sub-resource kinds: %v", maps.Keys(delKindMap)))
Expand Down Expand Up @@ -174,37 +174,6 @@ func kindsForWipeOut() ([]client.ObjectList, []client.ObjectList) {
return append(namespacedKinds, namespacedKindsPlus...), nonNamespacedKinds
}

// shouldSkipObjOwnedByComp is used to judge whether the object owned by component should be skipped when deleting the cluster
func shouldSkipObjOwnedByComp(obj client.Object, cluster appsv1.Cluster) bool {
ownByComp := isOwnedByComp(obj)
if !ownByComp {
// if the object is not owned by component, it should not be skipped
return false
}

// Due to compatibility reasons, the component controller creates cluster-scoped RoleBinding and ServiceAccount objects in the following two scenarios:
// 1. When the user does not specify a ServiceAccount, KubeBlocks automatically creates a ServiceAccount and a RoleBinding with named pattern kb-{cluster.Name}.
// 2. When the user specifies a ServiceAccount that does not exist, KubeBlocks will automatically create a ServiceAccount and a RoleBinding with the same name.
// In both cases, the lifecycle of the RoleBinding and ServiceAccount should not be tied to the component. They should be deleted when the cluster is deleted.
doNotSkipTypes := []interface{}{
&rbacv1.RoleBinding{},
&corev1.ServiceAccount{},
}
for _, t := range doNotSkipTypes {
if objType, ok := obj.(interface{ GetName() string }); ok && reflect.TypeOf(obj) == reflect.TypeOf(t) {
if strings.EqualFold(objType.GetName(), constant.GenerateDefaultServiceAccountName(cluster.GetName())) {
return false
}
labels := obj.GetLabels()
value, ok := labels[constant.AppManagedByLabelKey]
if ok && value == constant.AppName {
return false
}
}
}
return true
}

func deleteCompNShardingInOrder4Terminate(transCtx *clusterTransformContext, dag *graph.DAG) (sets.Set[string], error) {
nameSet, err := clusterRunningCompNShardingSet(transCtx.Context, transCtx.Client, transCtx.Cluster)
if err != nil {
Expand Down
4 changes: 1 addition & 3 deletions controllers/apps/component/component_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -212,10 +212,8 @@ func (r *ComponentReconciler) setupWithManager(mgr ctrl.Manager) error {

if viper.GetBool(constant.EnableRBACManager) {
b.Owns(&rbacv1.RoleBinding{}).
Owns(&rbacv1.Role{}).
Owns(&corev1.ServiceAccount{})
} else {
b.Watches(&rbacv1.RoleBinding{}, handler.EnqueueRequestsFromMapFunc(r.filterComponentResources)).
Watches(&corev1.ServiceAccount{}, handler.EnqueueRequestsFromMapFunc(r.filterComponentResources))
}

return b.Complete(r)
Expand Down
Loading
Loading