Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CONNECTOR] BACKUP 預計完成事項 #1520

Open
5 tasks
chaohengstudent opened this issue Feb 25, 2023 · 6 comments
Open
5 tasks

[CONNECTOR] BACKUP 預計完成事項 #1520

chaohengstudent opened this issue Feb 25, 2023 · 6 comments
Assignees
Milestone

Comments

@chaohengstudent
Copy link
Collaborator

chaohengstudent commented Feb 25, 2023

BACKUP 下一階段目標為快照整個叢集(能指定範圍)以及根據快照還原叢集(資料、分佈及參數)
因此有以下幾個項目需要完成:

@chaohengstudent
Copy link
Collaborator Author

這邊更新Metadata備份

ClusterInfo 序列化後存入目前支援的 file system (Implement Serializer)

  • ClusterInfo 序列化格式
    [NodeInfo] [Replicas]

從 file system 讀入反序列化還原ClusterInfo (implement Deferializer)

@chia7712 想確認這邊是否有需要修改的部份
如果可以的話,我們會再討論細節由 @Haser0305 進行序列化 @chaohengstudent 處理反序列化

@chia7712
Copy link
Contributor

chia7712 commented Mar 8, 2023

ClusterInfo 序列化格式

這部分可否條列一下預期包含哪些資訊?如果可以所有資訊都保存當然最好,否則可以只挑選要用來建構叢集的資訊就好

@chaohengstudent
Copy link
Collaborator Author

預期包含的資訊:
nodeInfo [ id ]
replicas [ topic, partition, nodeinfo-id, isLeader ]

@chia7712
Copy link
Contributor

chia7712 commented Mar 8, 2023

nodeInfo [ id ]
replicas [ topic, partition, nodeinfo-id, isLeader ]

能否也包含 topic configs ?

另外這些資訊只是 cluster info 的“部分資訊”,該些項目能夠重建 cluster info 回來嗎?如果不行的話,是否要用別的資料結構呢?

@chaohengstudent
Copy link
Collaborator Author

chaohengstudent commented Mar 9, 2023

能夠重建 cluster info 回來嗎?如果不行的話,是否要用別的資料結構呢?

如果我們想取所有資訊來做,包含的 [cluster id] [nodes] [replicas] [topics] ,這樣是否可以?

@chia7712
Copy link
Contributor

chia7712 commented Mar 9, 2023

如果我們想取所有資訊來做,包含的 [cluster id] [nodes] [replicas] [topics] ,這樣是否可以?

能完整保存 cluster info 當然很好,我只是擔心內部的資料結構太複雜會不好存放而已

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants