Skip to content

Commit

Permalink
internal_gengo: avoid allocations in rawDescGZIP() accessors
Browse files Browse the repository at this point in the history
Use unsafeBytes in rawDescGZIP() as well, which is safe because our
CompressGZIP() does not write to the byte slice.

Store the result of CompressGZIP as a byte slice so that subsequent
calls do not cause any more allocations.

In http://go.dev/cl/638135, I asserted that rawDescGZIP() is rarely-used,
but after rolling out the change Google-internally, affected users made
me aware of a few programs that heavily access these descriptors.

Change-Id: Ieb5010ddc7b9ac6be88970321ff01a3d29e846bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/643276
Reviewed-by: Chressie Himpel <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Auto-Submit: Nicolas Hillegeer <[email protected]>
Reviewed-by: Nicolas Hillegeer <[email protected]>
Commit-Queue: Nicolas Hillegeer <[email protected]>
  • Loading branch information
stapelberg authored and gopherbot committed Jan 17, 2025
1 parent 2005adb commit 5f93d99
Show file tree
Hide file tree
Showing 98 changed files with 308 additions and 300 deletions.
26 changes: 17 additions & 9 deletions cmd/protoc-gen-go/internal_gengo/reflect.go
Original file line number Diff line number Diff line change
Expand Up @@ -187,12 +187,10 @@ func genReflectFileDescriptor(gen *protogen.Plugin, g *protogen.GeneratedFile, f
g.P("out := ", protoimplPackage.Ident("TypeBuilder"), "{")
g.P("File: ", protoimplPackage.Ident("DescBuilder"), "{")
g.P("GoPackagePath: ", reflectPackage.Ident("TypeOf"), "(x{}).PkgPath(),")
// Avoid a copy of the descriptor by using an inlined version of
// [strs.UnsafeBytes] (gencode cannot depend on internal/strs).
// This means modification of the RawDescriptor byte slice
// will crash the program. But generated RawDescriptors
// are never supposed to be modified anyway.
g.P("RawDescriptor: ", unsafePackage.Ident("Slice"), "(", unsafePackage.Ident("StringData"), "(", rawDescVarName(f), "), len(", rawDescVarName(f), ")),")
// Avoid a copy of the descriptor. This means modification of the
// RawDescriptor byte slice will crash the program. But generated
// RawDescriptors are never supposed to be modified anyway.
g.P("RawDescriptor: ", unsafeBytesRawDesc(g, f), ",")
g.P("NumEnums: ", len(f.allEnums), ",")
g.P("NumMessages: ", len(f.allMessages), ",")
g.P("NumExtensions: ", len(f.allExtensions), ",")
Expand Down Expand Up @@ -270,20 +268,30 @@ func genFileDescriptor(gen *protogen.Plugin, g *protogen.GeneratedFile, f *fileI
dataVar := rawDescVarName(f) + "Data"
g.P("var (")
g.P(onceVar, " ", syncPackage.Ident("Once"))
g.P(dataVar, " = ", rawDescVarName(f))
g.P(dataVar, " []byte")
g.P(")")
g.P()

g.P("func ", rawDescVarName(f), "GZIP() []byte {")
g.P(onceVar, ".Do(func() {")
g.P(dataVar, " = string(", protoimplPackage.Ident("X"), ".CompressGZIP([]byte(", dataVar, ")))")
g.P(dataVar, " = ", protoimplPackage.Ident("X"), ".CompressGZIP(", unsafeBytesRawDesc(g, f), ")")
g.P("})")
g.P("return []byte(", dataVar, ")")
g.P("return ", dataVar)
g.P("}")
g.P()
}
}

// unsafeBytesRawDesc returns an inlined version of [strs.UnsafeBytes]
// (gencode cannot depend on internal/strs). Modification of this byte
// slice will crash the program.
func unsafeBytesRawDesc(g *protogen.GeneratedFile, f *fileInfo) string {
return fmt.Sprintf("%s(%s(%[3]s), len(%[3]s))",
g.QualifiedGoIdent(unsafePackage.Ident("Slice")),
g.QualifiedGoIdent(unsafePackage.Ident("StringData")),
rawDescVarName(f))
}

func genEnumReflectMethods(g *protogen.GeneratedFile, f *fileInfo, e *enumInfo) {
idx := f.allEnumsByPtr[e]
typesVar := enumTypesVarName(f)
Expand Down
6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/annotations/annotations.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/comments/comments.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/comments/deprecated.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/extensions/base/base.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/extensions/ext/ext.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/extensions/extra/extra.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/extensions/proto3/ext3.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/fieldnames/fieldnames.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/import_public/a.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/import_public/b.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/import_public/c.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/import_public/sub/a.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions cmd/protoc-gen-go/testdata/import_public/sub/b.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 5f93d99

Please sign in to comment.