I encounter an issue with Go Module management and a generation of protobuffers (using go1.16, protoc-gen-go@latest).
I have this project structure:
subproj
├── go.mod (module company.tld/proj/subproj)
├── subproj.go (entry point : package main)
├── proto (folder containing .proto files)
├── packageFolder
| └── file1.go (package packageFolder)
└── Makefile (used to generate *.pb.go and build subproj binary)
The proto folder is used by other projects (obviously...) (via git submodule).
Protos are like the following:
syntax = "proto3"
option csharp_namespace = "Proj.Proto";
option go_package = "company.tld/proj/projpb";
package entity.proj
...
because of different version of messages, few protobuffer files need to be in another "namespace":
option go_package = "company.tld/proj/projpb/other";
package entity.proj.other
In my Makefile, I tried to generate the right *.pb.go at the right place:
# Proto sources
PROTO= $(wildcard ${PROTODIR}/*.proto)
PBGO= $(PROTO:.proto=.pb.go)
MODULE_NAME=company.tld/proj
GO_OPT_FLAG= --go_opt=module=${MODULE_NAME}
GRPC_OPT_FLAG= --go-grpc_opt=module=${MODULE_NAME}
#GO_OPT_FLAG= --go_opt=paths=import
#GRPC_OPT_FLAG= --go-grpc_opt=paths=import
.PHONY: clean install proto
## Builds the project
build: proto
go build ${LDFLAGS} -o ${BINARY}
$(PROTOBUF_GO_PLUGIN):
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
$(GRPC_GO_PLUGIN):
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
%.pb.go: %.proto | $(PROTOBUF_GO_PLUGIN) $(GRPC_GO_PLUGIN)
protoc --proto_path=${PROTODIR} --go_out=. ${GO_OPT_FLAG} --go-grpc_out=. ${GRPC_OPT_FLAG} $<
proto: $(PBGO)
So, depending on the option used with the protoc compiler:
→ With --go_opt=paths=import
A folder tree company.tld/proj/projpb is created by protoc at the project's root. Each object is in a package called projpb or other, in the subpackage other.
Generated Proto objects, that include the other namespace-d objects, have the import path import other "company.tld/proj/projpb/other" (which is brought by the go_package option, but which is wrong because it is not an existing module - go mod tidy/vendor is complaining that it cannot find it).
Normal project files need the following import path to reach the Generated Proto objects :
import pb "company.tld/proj/subproj/company.tld/proj/projpb"
which seems odd and not the proper way to do.
→ With --go_opt=module=company.tld/proj
A folder projpb is created by protoc at the project's root and each generated .pb.go has the package projpb or other, in the subpackage other.
Generated Proto objects, that include the other namespace-d objects, still have the import path import other "company.tld/proj/projpb/other" (which is still brought by the go_package option and is still wrong because this is still a non-existing module - these are generated files... why would I want to create a module of these ?).
The cool thing is that with this go_opt, accessing generated types looks much more normal with
import pb "company.tld/proj/subproj/projpb".
Finally, I tried
- using local import path on the go_package option in the .proto files (that is refused on build time, because there would be an
import other "./projpb/other"in generated protobuffer object) - to use the replace instruction in the go.mod file like this :
replace (
company.tld/proj/projpb => ./projpb
company.tld/proj/projpb/other => ./projpb/other
)
(but go mod tidy/vendor is complaining that it cannot find the go.mod file inside the generated folder ./projpb)
Has someone encountered a similar problem? Or am I missing a command option to tell to Go, «I generate protobuffer objects in a package, or package in a package, and I simply want to use them. They are not a module, so please, provide the right import paths to the generated object and let me use them in my code».
[Update 01]
I gave a try to the go_opt=paths=source_relative (inspired by this ticket).
I created the folder in the Makefile, protoc generates files inside.
Notes:
- generated protos use the full path, specified with the
go_packageoption, to relate to one another. - As long as
go_packageoption needs a full path, Go (go mod tidy/vendor) will want to search for a go.mod file inside the created folder, containing generated protos.
What is the correct way to tell Go that I am not looking for a Module, yet still satisfy the go_package option's full path constraint in the protobuffer file ?
