1

Let's say I have several small standalone projects, all using their own git repository.

This could be in any language I guess, but in practice, I'm interested mostly in C, C++, and Java.

Alongside and supporting these projects, I have a variety of small utility functions and classes, a "personal standard library" if you will (hereafter I'll call this "the lib" - but keep in mind it is not a monolithic thing: you can pick and choose pieces from it). In most of my projects, I use one or more of the files from this library.

I am looking for some way to manage this situation. Each git repository should be standalone in the sense that if you clone the project alone it comes with all the files needed to compile it. That is, I don't want to add the entire lib as a submodule (indeed, some projects may be a couple of source files, one of which comes from the lib and I don't want to import 100 other files).

Each project should be able to pick and choose which files from lib it consumes. If I update lib, ideally there should be a simple way for projects to pick up the updates.

I considered symlinks: each project symlinks to the files it wants from lib, but git doesn't follow symlinks. I considered hardlinks - maybe this works? It doesn't seem very portable to file systems that don't support hard links.

How can I accomplish my goals?

BeeOnRope
  • 60,350
  • 16
  • 207
  • 386
  • what do you mean that 'git does not allow symlinks'? it does. – Serge May 05 '19 at 17:44
  • @Serge - you missed an 'f': it says "follow" not "allow". You can put symlinks to files outside of the project, but the symlink itself is committed, e.g., a file with contents `/home/me/personal_lib/file.hpp`, not the actual contents of `file.hpp`, so the repository doens't work after cloning for anyone but me. – BeeOnRope May 05 '19 at 17:45
  • oops.my apologies. git commits the symlink content. It does not care if it points to a real file at commit or checkout. This is the responsibility of the user to make sure that the targets exists at check-out. Git only cares about what is in its worktree. Unless you use one of git integration techniques (i.e., submodules, subtres, subrepos) you wold need to take additional step to get other files right. – Serge May 06 '19 at 01:22
  • @Serge - yes, that's what this question is about. The symlink behavior isn't useful for out-of-tree files. – BeeOnRope May 06 '19 at 01:27
  • then you have no choice. Either put them in the tree (using one of the available methods) or make sure that they are present at checkout by other means. in the first place you can use branches to build different configurations. – Serge May 06 '19 at 02:09

1 Answers1

1

This is better address by submodules

git submodule add /url/repo amodule will add a subfolder with amodule with, as its content, the cloned repo.

The URL can actually be a relative path:

<repository> is the URL of the new submodule’s origin repository. This may be either an absolute URL, or (if it begins with ./ or ../), the location relative to the superproject’s default remote repository.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • submodule is a disaster: it doesn't come along automatically when you clone, it doesn't update when you pull changes after updating the submodule, etc. We're tallking about small projects which pull in usually 1-5 files from the personal library and so introducing all the problems of submodule is a non-starter unfortunately. – BeeOnRope May 06 '19 at 15:34
  • @BeeOnRope It can come along automatically when you clone (https://stackoverflow.com/a/4438292/6309), a pull will update them (https://stackoverflow.com/a/49427199/6309), but yes, I understand they do add complexity. – VonC May 06 '19 at 16:02
  • Yes, I know about `--recursive`, but how many people use that by default? Again `pull` does not update them by default - but yes there is a non-default option to do something different. This isn't about my personal use: I can figure those out and be careful about everything: it's about users who do a plain `git clone/git pull` and `make` workflow: submodules just dosn't work well at all. It's pushing the management of my shared into a place where the end user sees it. They could care less if `util.hpp` is shared with another project. For the end user, the repo _must_ look like a single repo. – BeeOnRope May 06 '19 at 16:39
  • BTW, for the places where I do currently use submodules I'm switching to `subtree`. Submodules just not worth it... – BeeOnRope May 06 '19 at 16:39
  • @BeeOnRope Yes, for your case, git subtree (that I presented here: https://stackoverflow.com/a/24709789/6309) is probably more appropriate. I am used to differentiate the two (https://stackoverflow.com/a/31770147/6309) – VonC May 06 '19 at 16:42