5

For my project I am trying to get all the dependencies and sub dependencies of my project. I need to specific version of each of these dependencies. Not only do I need the dependencies of my project, but the dependencies of the dependencies and so on until the root.

For my project, go list -m all works for everything except indirect dependencies that have not opted into using go.mod files. Right now my workflow is taking an initial batch of repositories, downloading them from git then using “GO111MODULE=on go build ./…”. and “GO111MODULE=on go list -m -json all” to get the list of dependencies. I do not check for go.mod as all of the repositories I am scanning are using go.mod files.

For the list of dependencies that come out of this initial list I have some questions, for files without go.mod files, I used this as a reference: “https://blog.golang.org/using-go-modules”

-Path = Received from go list -m all, it can be GitHub, gopkg, or whatever is used to dl the go package.

Without go.mod

-“GO111MODULE=on go mod init <PATH from parent go.mod>”

-“GO111MODULE=on go build ./…”

-“GO111MODULE=on go mod tidy”

-“GO111MODULE=on go list -m -json all”

-From there I get a list of the dependencies of this module.

With go.mod 

-“GO111MODULE=on go build ./…”

-“GO111MODULE=on go mod tidy”

-“GO111MODULE=on go list -m -json all”

Should I be running go build on each dependency that has a go.mod file? For ones without a go.mod file, I understand this should be done, as how else will we populate the go.mod file with the dependencies. But for files with a go.mod file, will I be pulling extra stuff that is not necessarily being used by my project with go build, like test files and other files that might not be used when I am simply importing that project? I understand that its better to get more unused dependencies rather than missing some, but it is getting a bit overwhelming with how massive the amount of dependencies is.

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
deniv
  • 61
  • 1
  • 3
  • 3
    go list works no matter if a package is module or not just do not use the -m flag. For the rest of the questions: There is nothing to worry about, all works fine. Unused dependencies are not a problem and stuff inside a package but unused during your build also is not a problem to worry about. – Volker Sep 12 '19 at 03:39
  • 1
    Having some difficulty understanding the question. I think the answer is "no, I don't think you should be running `go build` on each dep that uses go modules". Perhaps you can understand why better from this: https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod – Dean Coakley Sep 12 '19 at 13:42
  • The issue with using go list is that it does not show the version of the dependency used, which is something I need and is why I am using the -m tag – deniv Sep 12 '19 at 14:45
  • It has no notion of non-module dependencies' versions. It only understands versions for modules. – Adrian Sep 12 '19 at 15:10
  • 2
    does it help: https://stackoverflow.com/a/55866702/819651 ? – Luís Soares Apr 03 '20 at 23:43

2 Answers2

1

I can try to analyze go.sum file (when you execute go list -u, go.sum was created)

The go command uses the go.sum file to ensure that future downloads of these modules retrieve the same bits as the first download, to ensure the modules your project depends on do not change unexpectedly, whether for malicious, accidental, or other reasons. Both go.mod and go.sum should be checked into version control. (Using Go Modules - Adding a dependency)

go.sum file lists down the checksum (and version tag) of direct and indirect dependency required by the module.

% cat go.sum
...
github.com/bmizerany/perks v0.0.0-20141205001514-d9a9656a3a4b/go.mod h1:ac9efd0D1fsDb3EJvhqgXRbFx7bs2wqZ10HQPeU8U/Q=
github.com/c2h5oh/datasize v0.0.0-20171227191756-4eba002a5eae/go.mod h1:S/7n9copUssQ56c7aAgHqftWO4LTf4xY6CGWt8Bc+3M=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cockroachdb/apd v1.1.0/go.mod h1:8Sl8LxpKi29FqWXR16WEFZRNSz3SoPzUzeMeY4+DwBQ=
...
kozmo
  • 4,024
  • 3
  • 30
  • 48
0

The phrase “all the dependencies” is unfortunately ambiguous.

go list -m lists all modules whose selected versions are determined by your go.mod file. That is one possible definition of "all the dependencies", but it is a much broader set of modules than I think most people intend when they talk about the “dependencies” of a module.

In practice, go list -test all (without the -m flag) is the broadest set of dependencies I would generally care about: it includes all of the packages listed in import statements within your module (i.e. everything you need in order to run go test ./... within your module), plus all of the packages transitively needed to run go test on those packages.

(In particular, go list -test all is also the set of packages that will be resolved when you run go mod tidy.)

bcmills
  • 4,391
  • 24
  • 34