Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upgrade to use google protobuf #9190

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

upgrade to use google protobuf #9190

wants to merge 2 commits into from

Conversation

mangalaman93
Copy link
Contributor

@mangalaman93 mangalaman93 commented Oct 7, 2024 •
edited by shivaji-kharse
Loading

inspired from #7313

@github-actions github-actions bot added area/testing Testing related issues area/schema Issues related to the schema language and capabilities. area/graphql Issues related to GraphQL support on Dgraph. area/bulk-loader Issues related to bulk loading. area/core internal mechanisms go Pull requests that update Go code labels Oct 7, 2024
@github-actions github-actions bot added area/enterprise Related to proprietary features area/enterprise/backup Related to binary backups labels Oct 8, 2024
@shivaji-kharse shivaji-kharse marked this pull request as ready for review October 8, 2024 10:35
@shivaji-kharse shivaji-kharse requested a review from a team as a code owner October 8, 2024 10:35
mangalaman93
mangalaman93 commented Oct 17, 2024
View reviewed changes
conn/raft_server.go
@@ -128,6 +128,7 @@ func (p *proposals) Done(key uint64, err error) {
type RaftServer struct {
m sync.RWMutex
node *Node
pb.RaftServer
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to ensure that we implement the pb.RaftServer interface instead embedding it like this

dgraph/cmd/zero/zero.go
@@ -79,6 +79,7 @@ type Server struct {
blockCommitsOn *sync.Map

checkpointPerGroup map[uint32]uint64
pb.UnimplementedZeroServer
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably the same issue here

edgraph/server.go
m := jsonpb.Marshaler{EmitDefaults: true}
var jsonState bytes.Buffer
if err := m.Marshal(&jsonState, ms); err != nil {
m := protojson.MarshalOptions{EmitUnpopulated: true}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we get some understanding of EmitDefaults vs EmitUnpopulated?

edgraph/server.go
@@ -102,7 +101,9 @@ var (
)

// Server implements protos.DgraphServer
type Server struct{}
type Server struct {
api.DgraphServer
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same problem here too

x/minioclient.go
@@ -31,8 +29,8 @@ const (
// If these credentials are missing the default credentials will be used.
type MinioCredentials struct {
AccessKey string
SecretKey pb.Sensitive
SessionToken pb.Sensitive
SecretKey string
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will start printing these sensitive data into logs, we need to figure out a fix for this.

tlstest/zero_https/all_routes_tls/all_routes_tls_test.go
@@ -101,7 +101,7 @@ func TestZeroWithAllRoutesTLSWithTLSClient(t *testing.T) {
}

body := readResponseBody(t, do)
if !strings.Contains(string(body), test.response) {
if !strings.Contains(strings.ReplaceAll(string(body), " ", ""), test.response) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we calling ReplaceAll here?

worker/backup_ee.go
size := proto.Size(list)

// Ensure the size is non-negative and fits within uint64 bounds
if size < 0 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be an assert

worker/worker.go
@@ -78,6 +78,7 @@ func Init(ps *badger.DB) {
// grpcWorker struct implements the gRPC server interface.
type grpcWorker struct {
sync.Mutex
pb.WorkerServer
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

x/keys.go
@@ -419,7 +419,7 @@ func (p ParsedKey) ToBackupKey() *pb.BackupKey {
key.Attr = attr
key.Uid = p.Uid
key.StartUid = p.StartUid
key.Term = p.Term
key.Term = []byte(p.Term)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may affect performance

x/keys.go
@@ -455,7 +455,7 @@ func FromBackupKey(backupKey *pb.BackupKey) []byte {
case pb.BackupKey_DATA:
key = DataKey(attr, backupKey.Uid)
case pb.BackupKey_INDEX:
key = IndexKey(attr, backupKey.Term)
key = IndexKey(attr, string(backupKey.Term))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@shivaji-kharse shivaji-kharse

Labels
area/bulk-loader Issues related to bulk loading. area/core internal mechanisms area/enterprise/backup Related to binary backups area/enterprise Related to proprietary features area/graphql Issues related to GraphQL support on Dgraph. area/schema Issues related to the schema language and capabilities. area/testing Testing related issues go Pull requests that update Go code
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mangalaman93 @shivaji-kharse