Allow using together with protoc-gen-go

[saschagrunert]

Hey, right now using both protoc-gen-go and protoc-gen-go-plugin together will result in:

> protoc --go-plugin_out=. --go-plugin_opt=paths=source_relative --go_out=. --go_opt=paths=source_relative pkg/api/api.proto
pkg/api/api.pb.go: Tried to write the same file twice.

Would it be possible to run both plugins together so that they can share the types?

Somehow related to cri-o/cri-o#8715, containerd/nri#120

[knqyf263]

As written in README, TinyGo doesn't work with the code generated by protoc-gen-go. go-plugin ported the code from github.com/protocolbuffers/protobuf-go and added some tweaks so that it can work with TinyGo, which means go-plugin also generates code in xxx.pb.go.

[knqyf263]

However, that was two years ago. The situation might have changed. tinygo-org/tinygo#2667
If you confirm the latest TinyGo works with protoc-gen-go, we can remove the ported code and let protoc-gen-go generate xxx.pb.go.

[saschagrunert]

If you confirm the latest TinyGo works with protoc-gen-go

Did a test using:

tinygo version 0.34.0 linux/amd64 (using go version go1.23.2 and LLVM version 18.1.8)

And: https://github.com/containerd/nri/blob/main/pkg/api/api.proto

mkdir tmp && protoc --go_out=tmp --go_opt=paths=source_relative ./pkg/api/api.proto

Then using a main.go like:

package main

import (
	"github.com/containerd/nri/tmp/pkg/api"
)

func main() {
	_ = &api.RegisterPluginRequest{}
}

Which builds successfully using tinygo:

tinygo build -scheduler=none -target=wasi -no-debug main.go

[knqyf263]

If I remember correctly, some errors occur at runtime. Did you successfully run the code?

[inliquid]

I'am still getting errors when compiling using more or less complex proto definitions:

$ tinygo build -o wasm/plugin.wasm -scheduler=none -target=wasip1 --no-debug ./wasm
../../../../pkg/mod/golang.org/x/[email protected]/http2/transport.go:26:2: package net/http/httptrace is not in std (/home/***/.cache/tinygo/goroot-8cbcb1d9555609a60075264735fce03217a624835ebf583eb96aa0c5bf08e840/src/net/http/httptrace)
../../../../pkg/mod/google.golang.org/[email protected]/internal/transport/proxy.go:29:2: package net/http/httputil is not in std (/home/***/.cache/tinygo/goroot-8cbcb1d9555609a60075264735fce03217a624835ebf583eb96aa0c5bf08e840/src/net/http/httputil)

Tracking issues: tinygo-org/tinygo#2704 and tinygo-org/tinygo#4580

[saschagrunert]

@inliquid this is probably because of the gRPC generated code

[inliquid]

this is probably because of the gRPC generated code

Most likely it is. The package referred to by the wasm module is quite complex, I would like to just import it, but can't, so have to re-generate these definitions locally as well using go-plugin.

[knqyf263]

We may be able to add a new option to disable generating pb.go files as below. Then, we can run protoc --go-plugin_out=. --go-plugin_opt=paths=source_relative,disable_pb_gen=true --go_out=. --go_opt=paths=source_relative pkg/api/api.proto . However, I've never tested, so please don't trust this code.

diff --git a/cmd/protoc-gen-go-plugin/main.go b/cmd/protoc-gen-go-plugin/main.go
index 0e64222..6945787 100644
--- a/cmd/protoc-gen-go-plugin/main.go
+++ b/cmd/protoc-gen-go-plugin/main.go

@@ -10,6 +11,7 @@ import (
 
 func main() {
 	var flags flag.FlagSet
+	disablePbGen := flags.Bool("disable_pb_gen", false, "disable .pb.go generation")
 	protogen.Options{ParamFunc: flags.Set}.Run(func(plugin *protogen.Plugin) error {
 		g, err := gen.NewGenerator(plugin)
 		if err != nil {
@@ -20,7 +22,7 @@ func main() {
 			if !f.Generate {
 				continue
 			}
-			g.GenerateFiles(f)
+			g.GenerateFiles(f, gen.Options{DisablePBGen: *disablePbGen})
 		}
 
 		plugin.SupportedFeatures = uint64(pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL)
diff --git a/gen/main.go b/gen/main.go
index 3d96403..3d29c83 100644
--- a/gen/main.go
+++ b/gen/main.go
@@ -89,6 +89,10 @@ type Generator struct {
 	vtgen  *vtgenerator.Generator
 }
 
+type Options struct {
+	DisablePBGen bool
+}
+
 func NewGenerator(plugin *protogen.Plugin) (*Generator, error) {
 	ext := &vtgenerator.Extensions{}
 	featureNames := []string{"marshal", "unmarshal", "size"}
@@ -138,9 +142,11 @@ func replaceImport(m *protogen.Message) {
 }
 
 // GenerateFiles generates the contents of a .pb.go file.
-func (gg *Generator) GenerateFiles(file *protogen.File) *protogen.GeneratedFile {
+func (gg *Generator) GenerateFiles(file *protogen.File, opts Options) *protogen.GeneratedFile {
 	f := gg.newFileInfo(file)
-	gg.generatePBFile(f)
+	if !opts.DisablePBGen {
+		gg.generatePBFile(f)
+	}
 	gg.generateHostFile(f)
 	gg.generatePluginFile(f)
 	gg.generateVTFile(f)
@@ -252,7 +258,7 @@ func (gg *Generator) genImport(g *protogen.GeneratedFile, f *fileInfo, imp proto
 
 	// Generate public imports by generating the imported file, parsing it,
 	// and extracting every symbol that should receive a forwarding declaration.
-	impGen := gg.GenerateFiles(impFile)
+	impGen := gg.GenerateFiles(impFile, Options{})
 	impGen.Skip()
 	b, err := impGen.Content()
 	if err != nil {