How to import a private Go library (as module) within another private Go project (as module)

Question

I am moving a few private Go projects to GitLab while getting rid of Godeps, Go dep with vendor directory and all of that because I would like to use just Go modules.

I am using Go version: go1.12.6 linux/amd64.

• I have a private "library" project / git repository at gitlab.com/my-company/my-team/my-library. This works as a Go module with go.mod and go.sum.
• I would like to use my-library as a dependency for another project, this: gitlab.com/my-company/my-team/my-project.

The structure of the URL is the same, the only thing that changes is the name of the repository.

I am facing all sorts of errors while importing my-library in my-project.

• How can I import Go modules as private repositories inside other Go modules as private repositories?
• What's wrong with the command outputs below?

I know the GitLab tokens work because the go get command figures out itself the commit ID of my-library. Anyway I've done the following (see https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html):

git config \
  --global \
  url."https://token_name:token_val@gitlab.com".insteadOf \
  "https://gitlab.com" 

These are the error messages:

$ go get -u gitlab.com/my-company/my-team/my-library.git
go: finding gitlab.com/my-company/my-team/my-library.git latest
go: gitlab.com/my-company/my-team/my-library.git@v0.0.0-20190802120216-712a10fb5fac: parsing go.mod: unexpected module path "gitlab.com/my-company/my-team/my-library"
go get: error loading module requirements
$ 
$ go get -u gitlab.com/my-company/my-team/my-library
go get gitlab.com/my-company/my-team/my-library: git ls-remote -q https://gitlab.com/my-company/my-team.git in /home/foo/Development/go-workspace/pkg/mod/cache/vcs/9be637426eac43b329899d57d9375d12246f2cc0f6ddd098446bc42ed1ca534d: exit status 128:
    remote: The project you were looking for could not be found.
    fatal: repository 'https://token_name:token_val@gitlab.com/my-company/my-team.git/' not found
$ 
$ GO111MODULE=off go get -u gitlab.com/my-company/my-team/my-library.git
package gitlab.com/my-company/my-team/my-library.git: no Go files in /home/foo/Development/go-workspace/src/gitlab.com/my-company/my-team/my-library.git
$

• The first attempt is the one I think I should use, but it does not work and I don't know why.
  • I thought this was happening because somehow there was already a dependency like this in the project. But when executing go mod graph | grep gitlab I find an empty output, i.e. there is no conflicting dependency.
• The second attempt is probably to be avoided because the URL does not end with .git and (for some reason that I don't understand) the URL is cut at the level of my-team.
• GO111MODULE=off seems to force go get to check in $GOPATH which I think I should avoid at all, but I was just trying to see if that way I was able to fetch that dependency.

Answer 1

I would definitely suggest trying with Go 1.13 beta:

$ go get golang.org/dl/go1.13beta1
$ go1.13beta1 download
$ go1.13beta1 get foo

or even better, try the latest Go on tip / master (given the Go 1.13 beta1 release is over a month old at this point):

$ go get golang.org/dl/gotip
$ gotip download
$ gotip get foo

Go 1.13 improved several aspects of working with private repositories (including CL 170879 among other improvements), and also it generally has better error messages compared to Go 1.12.

For your first error message:

$ go get -u gitlab.com/my-company/my-team/my-library.git
go: finding gitlab.com/my-company/my-team/my-library.git latest
go: gitlab.com/my-company/my-team/my-library.git@v0.0.0-20190802120216-712a10fb5fac:
 parsing go.mod: unexpected module path "gitlab.com/my-company/my-team/my-library"
go get: error loading module requirements

That is the go command complaining about a mismatch between how a module is imported/required, vs. how it declares its own identity in the module line of its go.mod. If module foo is importing module bar, then foo needs to refer to bar the same way that bar declares its identity on the the module line of bar's go.mod file.

Said another way, the import path used to import a module (or go get a module) needs to start with the exact module path declared on the module line of the imported module's go.mod. You might need to change the importer to match the form declared on the module line in the go.mod file, or you might need to change the module line in the go.mod to match the form used by the importer, but they can't disagree. If they disagree, you get the first error you reported.

In general, you have a few choices about how to use private repos with Go modules. These two blog posts outline several of the issues and cover a few approaches, and are well worth a read if you haven't read them:

• "Fetching Private Dependencies with Go Modules", by Tim Raymond
• "Go Modules with Private Git Repositories", by Tim Jones

Finally, if it is still not clear what is going on, you probably should try go get -v foo or go get -v -x foo:

• The -v flag to go get asks to print more verbose details, including the HTTPS requests, though be mindful that certain "errors" such as 404 errors might be expected based on how a remote repository was configured.

• If the nature of the problem is still not clear, you can also try the more verbose go get -v -x foo, which also shows the git or other VCS commands being issued. If warranted, you can often execute the same git commands outside of the context of the go tool for troubleshooting purposes.

Answer 2

1. If the building server has no access to the internet, we need a private "nexus3 goproxy".

export GOPROXY=http://nexus.my-company.com/nexus/repository/goproxy/,direct
export GONOPROXY=gitlab.my-company.com
export GOSUMDB=off
export GO111MODULE=on

2. Forbid trying to download through http(s) protocol

even if "gitlab.my-company.com" does not support https, just config like this:

git config --global url."git@gitlab.my-company.com:".insteadOf "https://gitlab.my-company.com/"

3. Because of gitlab's flaw, we need to replace the module download path in go.mod config

require (
    gitlab.my-company.com/my-team/my-library v0.0.0-00010101000000-000000000000
)

replace (
    #use "go get -insecure gitlab.my-company.com/my-team/my-library" to get latest commit hash
    gitlab.my-company.com/my-team/my-library => git.midea.com/my-team/my-library.git v0.0.0-XXXXXXXXXXXXXX-YYYYYYYYYYYY
)