0

I am using multiple packages that I import into different projects, these range from custom adapters for my business logic that are shared by lambda and google cloud functions and other public packages. The way I do this right now is that I vendor them and include them for cloud functions. For applications that can be compiled and deployed on a VM, I compile them separately. This works fine for me, however, its a pain developing these modules.

If I update the method signature and names in the package, I have to push my changes to github / gitlab (my package path is something like gitlab.com/groupName/projectName/pkg/packageName) and then do a go get -u <pacakgeName> to update the package.

This also, does not really update it, sometimes am stuck with an older version with no idea on how to update it. Is there an easier way of working with this I wonder.

For sake of clarity:

Exported package 1 Path: gitlab.com/some/name/group/pkg/clients/psql

psql-client
    |
    |_ pkg
        |
        |_psql.go

Application 1 uses psql-client Path: gitlab.com/some/name/app1

Application 2 uses psql-client Path: gitlab.com/some/name/app2

automaticAllDramatic
  • 2,025
  • 1
  • 21
  • 25
  • What error do you get when doing go get -u? There's no concept of versions when using go get so what does it mean to be stuck with an older version? – Charlie Aug 08 '19 at 17:46

2 Answers2

1

My understanding is that (a) you are using the new Go modules system, and that (b) part of the problem is that you don't want to keep pushing changes to github or gitlab across different repositories when you are doing local development.

In other words, if you have your changes locally, it sounds like you don't want to round-trip those changes through github/gitlab in order for those changes to be visible across your related repositories that you are working on locally.

Most important advice

It is greatly complicating your workflow to have > 1 module in a single repository.

As is illustrated by your example, in general it is almost always more work on an on-going basis to have > 1 module in a single repository. It is also very hard to get right. For most people, the cost is almost always not worth it. Also, often the benefit is not what people expect, or in some cases, there is no practical benefit to having > 1 module in a repo.

I would definitely recommend you follow the commonly followed rule of "1 repo == 1 module", at least for now. This answer has more details about why.

Working with multiple repos

Given you are using Go modules, one approach is you can add a replace directive to a module's go.mod file that informs that Go module about the on-disk location of other Go modules.

Example structure

For example, if you had three repos repo1, repo2, repo3, you could clone them so that they all sit next to each other on your local disk:

myproject/
├── repo1
├── repo2
└── repo3

Then, if repo1 depends on repo2 and repo3, you could set the go.mod file for repo1 to know the relative on-disk location of the other two modules:

repo1 go.mod:

replace github.com/me/repo2 => ../repo2
replace github.com/me/repo3 => ../repo3

When you are inside the repo1 directory or any of its child directories, a go command like go build or go test ./.... will use the on-disk versions of repo2 and repo3.

repo2 go.mod:

If repo2 depends on repo3, you could also set:

replace github.com/me/repo3 => ../repo3

repo3 go.mod:

If for example repo3 does not depend on either of repo1 or repo2, then you would not need to add a replace to its go.mod.

Additional details

The replace directive is covered in more detail in the replace FAQ on the Modules wiki.

Finally, it depends on your exact use case, but a common solution at this point is to use gohack, which automates some of this process. In particular, it creates a mutable copy of a dependency (by default in $HOME/gohack, but the location is controlled by $GOHACK variable). gohackalso sets your current go.mod file to have a replace directive to point to that mutable copy.

thepudds
  • 4,787
  • 3
  • 20
  • 37
  • I already use replace and use an on disk location, however, this is how my go.mod file looks ``` replace github.com/me/parent-repo v0.0.0-20190808145620-8465703829e1 => ../../ ``` I guess instead of using replace for child package, I was using replace for parent, which somehow made sense to me back then – automaticAllDramatic Aug 08 '19 at 20:47
  • am marking this as the accepted answer cause it answers the query in a way. I still have the problem cause i have a lot of internal dependencies that I guess I need to solve for by structuring my package better – automaticAllDramatic Aug 08 '19 at 20:48
  • One question is how many repositories do you have, and how many modules do you have? Do you have one module per repository, or do you have some repositories with more than one module/go.mod file? – thepudds Aug 08 '19 at 21:19
  • The reason I ask about how many modules you have in a single repo is that sometimes people put > 1 module in a repo, and that usually greatly complicates workflows. It is almost always easier to follow the general rule of "1 repo == 1 module". – thepudds Aug 09 '19 at 00:12
  • yeah, i have a parent module that defines my types and two "child modules" in them (if i can call them that) then there are other repositories that import the types and the modules from this repository – automaticAllDramatic Aug 09 '19 at 07:03
  • 1
    It is almost always more work on an on-going basis to have > 1 module in a single repository. It is also very hard to get right. For most people, the cost is almost always not worth it. Also, often the benefit is not what people expect, or in some cases, there is no practical benefit to having > 1 module in a repo. I would definitely recommend you follow the commonly followed rule of "1 repo == 1 module", at least for now. [This answer](https://stackoverflow.com/a/57314494/11210494) has more details about why. – thepudds Aug 09 '19 at 13:43
0

go get is transitive, so you can just add it to your build process. A typical Go project build is basically:

go get -u ./... && go test ./... && go build ./cmd/myapp

Which gets & updates dependencies, runs all project tests, then builds the binary.

Adrian
  • 42,911
  • 6
  • 107
  • 99