Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor cmpd's role definition #8416

Merged
merged 41 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
c3962bf
api redefine
cjc7373 Oct 31, 2024
9981d3f
wip
cjc7373 Oct 31, 2024
3ddc10e
migrations (mostly tests)
cjc7373 Nov 4, 2024
caf994f
cmpd controller migration
cjc7373 Nov 6, 2024
cf64001
migrate ops
cjc7373 Nov 6, 2024
f94842c
fix unit test
cjc7373 Nov 6, 2024
16567e0
use type alias
cjc7373 Nov 6, 2024
2a2966d
typo
cjc7373 Nov 6, 2024
4285d88
wip
cjc7373 Nov 6, 2024
589adca
remove unused function
cjc7373 Nov 6, 2024
027c6d9
fix
cjc7373 Nov 7, 2024
40a77d5
remove required field, move replicarole to base package
cjc7373 Nov 8, 2024
bca059b
delete AccessModeLabelKey
cjc7373 Nov 8, 2024
7ad342b
fix unit test
cjc7373 Nov 8, 2024
90b8f22
make
cjc7373 Nov 8, 2024
6fc8514
comments
cjc7373 Nov 12, 2024
6439d9d
add a validation rule
cjc7373 Nov 12, 2024
d8e34de
comments
cjc7373 Nov 12, 2024
60456ae
fix cel
cjc7373 Nov 12, 2024
9dadfa8
fix cel again
cjc7373 Nov 12, 2024
2e630a9
delete cel
cjc7373 Nov 12, 2024
88cec5a
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Nov 15, 2024
b0e9c70
make leaveMemberForPod cleaner
cjc7373 Nov 15, 2024
12ffec8
fix test
cjc7373 Nov 15, 2024
9a6c1b9
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Nov 22, 2024
aa3010a
remove ReadyWithoutPrimary
cjc7373 Nov 22, 2024
198ac42
remove legacy code
cjc7373 Nov 22, 2024
2fe6d1f
make manifests/api-doc
cjc7373 Nov 22, 2024
eef3b70
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Dec 10, 2024
f1a42dc
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Dec 11, 2024
571e3b5
make generate && make api-doc
cjc7373 Dec 11, 2024
0e5569b
remove swtichoverBeforeUpdate
cjc7373 Dec 13, 2024
6fc0fda
switchover refactor
cjc7373 Dec 13, 2024
96b02ed
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Dec 23, 2024
b492aa8
comments
cjc7373 Dec 23, 2024
bc08373
go fmt
cjc7373 Dec 23, 2024
5adf450
remove accessMode
cjc7373 Dec 24, 2024
f032589
fix ut
cjc7373 Dec 24, 2024
7fb886c
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Jan 9, 2025
253a236
remove basev1
cjc7373 Jan 9, 2025
adbf63c
Merge remote-tracking branch 'origin/main' into feature/refactor-role
cjc7373 Jan 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 39 additions & 21 deletions apis/apps/v1/componentdefinition_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -415,11 +415,16 @@ type ComponentDefinitionSpec struct {
// +optional
Available *ComponentAvailable `json:"available,omitempty"`

// FIXME: there's a bug in CEL's cost estimation when chaining .filter() and .map().
// It was fixed in k8s 1.30, see: https://github.com/kubernetes/kubernetes/pull/123562.
// Maybe we can add this back later.
// TODO +kubebuilder:validation:XValidation:rule="self.filter(x, x.participatesInQuorum == true).map(x, x.updatePriority).min() > self.filter(x, x.participatesInQuorum == false).map(x, x.updatePriority).max()",message="Roles participate in quorum should have higher update priority than roles do not participate in quorum."

// Enumerate all possible roles assigned to each replica of the Component, influencing its behavior.
//
// A replica can have zero to multiple roles.
// KubeBlocks operator determines the roles of each replica by invoking the `lifecycleActions.roleProbe` method.
// This action returns a list of roles for each replica, and the returned roles must be predefined in the `roles` field.
// A replica can have zero or one role.
// KubeBlocks operator determines the role of each replica by invoking the `lifecycleActions.roleProbe` method.
// This action returns the role for each replica, and the returned role must be predefined here.
//
// The roles assigned to a replica can influence various aspects of the Component's behavior, such as:
//
Expand All @@ -430,6 +435,7 @@ type ComponentDefinitionSpec struct {
//
// This field is immutable.
//
// +kubebuilder:validation:MaxItems=128
// +optional
Roles []ReplicaRole `json:"roles,omitempty"`

Expand Down Expand Up @@ -1419,10 +1425,15 @@ type ComponentAvailableProbeAssertion struct {
Strict *bool `json:"strict,omitempty"`
}

// ReplicaRole represents a role that can be assumed by a component instance.
// ReplicaRole represents a role that can be assigned to a component instance, defining its behavior and responsibilities.
type ReplicaRole struct {
// Defines the role's identifier. It is used to set the "apps.kubeblocks.io/role" label value
// on the corresponding object.
// Name defines the role's unique identifier. This value is used to set the "apps.kubeblocks.io/role" label
// on the corresponding object to identify its role.
//
// For example, common role names include:
// - "leader": The primary/master instance that handles write operations
// - "follower": Secondary/replica instances that replicate data from the leader
// - "learner": Read-only instances that don't participate in elections
//
// This field is immutable once set.
//
Expand All @@ -1431,32 +1442,39 @@ type ReplicaRole struct {
// +kubebuilder:validation:Pattern=`^.*[^\s]+.*$`
Name string `json:"name"`

// Indicates whether a replica assigned this role is capable of providing services.
// UpdatePriority determines the order in which pods with different roles are updated.
// Pods are sorted by this priority (higher numbers = higher priority) and updated accordingly.
// Roles with the highest priority will be updated last.
// The default priority is 0.
//
// This field is immutable once set.
//
// +kubebuilder:default=false
// +optional
Serviceable bool `json:"serviceable,omitempty"`

// Determines if a replica in this role has the authority to perform write operations.
// A writable replica can modify data, handle update operations.
// For example:
// - Leader role may have priority 2 (updated last)
// - Follower role may have priority 1 (updated before leader)
// - Learner role may have priority 0 (updated first)
//
// This field is immutable once set.
//
// +kubebuilder:default=false
// +kubebuilder:default=0
// +optional
Writable bool `json:"writable,omitempty"`
UpdatePriority int `json:"updatePriority"`

// Specifies whether a replica with this role has voting rights.
// In distributed systems, this typically means the replica can participate in consensus decisions,
// configuration changes, or other processes that require a quorum.
// ParticipatesInQuorum indicates if pods with this role are counted when determining quorum.
// This affects update strategies that need to maintain quorum for availability. Roles participate
// in quorum should have higher update priority than roles do not participate in quorum.
// The default value is false.
//
// For example, in a 5-pod component where:
// - 2 learner pods (participatesInQuorum=false)
// - 2 follower pods (participatesInQuorum=true)
// - 1 leader pod (participatesInQuorum=true)
// The quorum size would be 3 (based on the 3 participating pods), allowing parallel updates
// of 2 learners and 1 follower while maintaining quorum.
//
// This field is immutable once set.
//
// +kubebuilder:default=false
// +optional
Votable bool `json:"votable,omitempty"`
ParticipatesInQuorum bool `json:"participatesInQuorum"`
}

// UpdateStrategy defines the update strategy for cluster components. This strategy determines how updates are applied
Expand Down
50 changes: 4 additions & 46 deletions apis/workloads/v1/instanceset_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -291,11 +291,6 @@ type InstanceSetStatus struct {
// +optional
MembersStatus []MemberStatus `json:"membersStatus,omitempty"`

// Indicates whether it is required for the InstanceSet to have at least one primary instance ready.
//
// +optional
ReadyWithoutPrimary bool `json:"readyWithoutPrimary,omitempty"`

// currentRevisions, if not empty, indicates the old version of the InstanceSet used to generate the underlying workload.
// key is the pod name, value is the revision.
//
Expand Down Expand Up @@ -330,33 +325,9 @@ const (
PreferInPlacePodUpdatePolicyType PodUpdatePolicyType = "PreferInPlace"
)

type ReplicaRole struct {

// Defines the role name of the replica.
//
// +kubebuilder:validation:Required
// +kubebuilder:default=leader
Name string `json:"name"`

// Specifies the service capabilities of this member.
//
// +kubebuilder:validation:Required
// +kubebuilder:default=ReadWrite
// +kubebuilder:validation:Enum={None, Readonly, ReadWrite}
AccessMode AccessMode `json:"accessMode"`

// Indicates if this member has voting rights.
//
// +kubebuilder:default=true
// +optional
CanVote bool `json:"canVote"`

// Determines if this member is the leader.
//
// +kubebuilder:default=false
// +optional
IsLeader bool `json:"isLeader"`
}
// ReplicaRole represents a role that can be assigned to a component instance, defining its behavior and responsibilities.
// +kubebuilder:object:generate=false
type ReplicaRole = kbappsv1.ReplicaRole

// AccessMode defines SVC access mode enums.
// +enum
Expand Down Expand Up @@ -604,18 +575,5 @@ func (r *InstanceSet) IsInstanceSetReady() bool {
return true
}
membersStatus := r.Status.MembersStatus
if len(membersStatus) != int(*r.Spec.Replicas) {
return false
}
if r.Status.ReadyWithoutPrimary {
return true
}
hasLeader := false
for _, status := range membersStatus {
if status.ReplicaRole != nil && status.ReplicaRole.IsLeader {
hasLeader = true
break
}
}
return hasLeader
return len(membersStatus) == int(*r.Spec.Replicas)
}
19 changes: 2 additions & 17 deletions apis/workloads/v1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

60 changes: 38 additions & 22 deletions config/crd/bases/apps.kubeblocks.io_componentdefinitions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8427,9 +8427,9 @@ spec:
Enumerate all possible roles assigned to each replica of the Component, influencing its behavior.


A replica can have zero to multiple roles.
KubeBlocks operator determines the roles of each replica by invoking the `lifecycleActions.roleProbe` method.
This action returns a list of roles for each replica, and the returned roles must be predefined in the `roles` field.
A replica can have zero or one role.
KubeBlocks operator determines the role of each replica by invoking the `lifecycleActions.roleProbe` method.
This action returns the role for each replica, and the returned role must be predefined here.


The roles assigned to a replica can influence various aspects of the Component's behavior, such as:
Expand All @@ -8443,49 +8443,65 @@ spec:

This field is immutable.
items:
description: ReplicaRole represents a role that can be assumed by
a component instance.
description: ReplicaRole represents a role that can be assigned
to a component instance, defining its behavior and responsibilities.
properties:
name:
description: |-
Defines the role's identifier. It is used to set the "apps.kubeblocks.io/role" label value
on the corresponding object.
Name defines the role's unique identifier. This value is used to set the "apps.kubeblocks.io/role" label
on the corresponding object to identify its role.


For example, common role names include:
- "leader": The primary/master instance that handles write operations
- "follower": Secondary/replica instances that replicate data from the leader
- "learner": Read-only instances that don't participate in elections


This field is immutable once set.
maxLength: 32
pattern: ^.*[^\s]+.*$
type: string
serviceable:
participatesInQuorum:
default: false
description: |-
Indicates whether a replica assigned this role is capable of providing services.
ParticipatesInQuorum indicates if pods with this role are counted when determining quorum.
This affects update strategies that need to maintain quorum for availability. Roles participate
in quorum should have higher update priority than roles do not participate in quorum.
The default value is false.


This field is immutable once set.
type: boolean
votable:
default: false
description: |-
Specifies whether a replica with this role has voting rights.
In distributed systems, this typically means the replica can participate in consensus decisions,
configuration changes, or other processes that require a quorum.
For example, in a 5-pod component where:
- 2 learner pods (participatesInQuorum=false)
- 2 follower pods (participatesInQuorum=true)
- 1 leader pod (participatesInQuorum=true)
The quorum size would be 3 (based on the 3 participating pods), allowing parallel updates
of 2 learners and 1 follower while maintaining quorum.


This field is immutable once set.
type: boolean
writable:
default: false
updatePriority:
default: 0
description: |-
Determines if a replica in this role has the authority to perform write operations.
A writable replica can modify data, handle update operations.
UpdatePriority determines the order in which pods with different roles are updated.
Pods are sorted by this priority (higher numbers = higher priority) and updated accordingly.
Roles with the highest priority will be updated last.
The default priority is 0.


For example:
- Leader role may have priority 2 (updated last)
- Follower role may have priority 1 (updated before leader)
- Learner role may have priority 0 (updated first)


This field is immutable once set.
type: boolean
type: integer
required:
- name
type: object
maxItems: 128
type: array
runtime:
description: |-
Expand Down
Loading
Loading