25

I have two development branches in git and I frequently need to change between the two. However, the really frustrating thing is that every time I change branches in git, the entire project gets rebuilt because the file-system timestamps for some files will change.

Ofc, the makefiles are configured to build the project into two different build directories .

Is there any way around this? Compilation is a very long and time-consuming process...

Edit:- This is a slightly more detailed explanation of the question... Say I have a header files Basic.h which is included in a number of other files. Basic.h is different between branch 1 and branch 2.

Now let's say I have compiled branch 1 into build_branch1 and branch 2 into build_branch2. Say I have branch 2 currently checked out. Now I checkout branch 1 and change File1.cpp and recompile. Ideally, since only File1.cpp has changed since I compiled it the last time, this is the only file that should be recompiled.

However, since Basic.h has it's timestamp changed due to the checkout, all files that are including Basic.h will get recompiled. I want to avoid this.

Sergey K.
  • 24,894
  • 13
  • 106
  • 174
owagh
  • 3,428
  • 2
  • 31
  • 53
  • 2
    I thought git only changes the timestamps on files that have indeed changed. In that case, you have to rebuild anyway to get correct results. What behavior are you looking for? – sinelaw Apr 27 '12 at 18:50
  • 1
    See [this](http://kerneltrap.org/mailarchive/git/2007/3/1/240167) related thread. Basically, doing something like this is not recommended. – mtvec Apr 27 '12 at 18:52
  • @sinelaw :- a.cpp includes Basic.h. I have two copies of a.o in branch1_build/a.o and branch2_build/a.o . Now when I switch branches from branch1 to branch2, it will recompile branch2_build/a.o even though it is up to date – owagh Sep 22 '16 at 16:38
  • Related: https://stackoverflow.com/questions/4913360/can-i-rebase-a-git-branch-without-modifying-my-working-copy – jozxyqk May 24 '18 at 23:29

5 Answers5

16

Git changes only the files that are updated between branches. But if your compiler does a full rebuild even if any single file was changed you can always clone and checkout your different branches into different directories. That's like:

/your-repo-name.branch1
/your-repo-name.branch2

This takes extra disk space but is much more convenient than switching divergent branches in a huge repo.

Michael Mior
  • 28,107
  • 9
  • 89
  • 113
Sergey K.
  • 24,894
  • 13
  • 106
  • 174
  • 7
    You can also use `--shared` to save most of that disk space: pick one of those clones as your "main" storage, and make the rest with `git clone --shared` referring back to the main-storage-clone. Just don't delete any branches in the "main" repo (or only delete ones that are not being shared) and you'll be safe there. – torek Apr 27 '12 at 19:24
  • 1
    Hey torek. Can you expand your answer about shared clones? Won't that create a shared repository that is group writable? – owagh Apr 27 '12 at 19:36
  • 1
    No, `--shared` only affects the new clone. In fact you don't even need `--shared` most of the time as git is (if not too ancient) smart enough to realize that it's a local clone and use hard links. (I imagine `--shared` is required on Windows though.) – torek Apr 27 '12 at 20:01
  • 1
    Well this is not exactly what I'd hoped to achieve but I guess this is the closest workable solution... – owagh Apr 27 '12 at 23:13
  • 3
    Also, you can use `git worktree` instead of full clones. That way you'll share the .git at least. – autra Oct 24 '18 at 07:06
12

Another partial answer: compiler cache.

When you switch back to the original branch and rebuild, although the dependencies say that numerous files dependent on Basic.h have to be rebuilt, the object files can be pulled from a compiler cache.

ccache ( http://ccache.samba.org/ ) still has to do some fairly expensive work (processing the pre-processed translation unit, since an entire translation unit is used a hash key) but it's a lot cheaper than compiling.

In some cases ccache can eliminate a compile in the face of a change that does not affect it, like some whitespace changes. For instance if you change a comment in a dependent file (header or source), that does not invalidate the cached object file.

So it can help even if you do a git pull and pick up a new change to Basic.h you haven't seen before.

Kaz
  • 55,781
  • 9
  • 100
  • 149
4

If you know which files actually need compilation, and do those manually, GNU Make (at least, I don't know about other implementations) has a flag for you: -t, which basically runs over your Makefile and changes timestamps instead of running commands.

You'll still need to update the files that need updating before using this flag, or you'll end up with object files that are legitimately out-of-date but look updated. See the linked doc for details.

The -o option might also interest you, depending on how much changes when you switch branches.

3

If Basic.h actually differs between the branches, then the only fix is to break it up into smaller files with more fine-grained dependencies, so that less stuff is rebuilt when it changes.

But suppose that Basic.h is actually exactly the same between branches, but git is changing its timestamp. This is a false trigger, like doing touch Basic.h, which reveals a limitation of timestamp-based build systems. Ideally we want a build system to rebuild when contents change, not when timestamps change. Timestamps are used because they are a cheap substitute for detecting a content change. The superior method is for the build system to keep hashes of all files and detect actual modifications, without regard for the time stamp. This also fixes problems like "clock skew detected; your build may be incomplete".

You're not going to get this kind of build system out of Make; but you may be able to work some aspects of it with regard to certain files. For instance, you can write your Make rules such that your object files do not depend directly on Basic.h but on Basic.h.sha, which is a stamp file. Now, what is in Basic.h.sha? This file contains a SHA256 hash of Basic.h.

Every time you run Make, logic in the Makefile computes the hash over Basic.h and compares it to the hash stored in Basic.h.sha. If they differ, then Basic.h.sha is overwritten with the new hash. Thereby, its timestamp is bumped, triggering a rebuild.

BASIC_H_SHA := $(shell sha256sum Basic.h)
BASIC_H_SHA_OLD := $(shell cat Basic.h.sha)

ifneq ($(BASIC_H_SHA),$(BASIC_H_SHA_OLD))
$(info Basic.h has changed!)
DUMMY := $(shell echo "$(BASIC_H_SHA)" > Basic.h.sha)
endif

I have implemented logic along these lines to make modules dependent on changes to their respective CFLAGS. Basically, we can turn a change in anything into a touch of a timestamp file, which controls what is built.

Kaz
  • 55,781
  • 9
  • 100
  • 149
  • 3
    I hope you do realize the ridiculousness of your answer. What I have given is the smallest subset that appropriately illustrates the problem at hand. – owagh Apr 27 '12 at 23:05
  • Obviously, you believe that although numerous files are recompiled because `Basic.h` changed, they do not actually have to be. This means that you believe that the files have some *false dependencies* on `Basic.h` (in addition to real ones). The only way to change that is to break up `Basic.h`, or else to introduce a completely different kind of build system which can manage fine grained dependencies such as "argument 3 of function `foo` changed; recompile everything that calls foo". – Kaz Apr 27 '12 at 23:25
  • 2
    The second part of your answer is a completely valid point about general programming techniques which I agree with but you have completely missed the whole point of my question and answered something I didn't even ask for. Thanks anyways... – owagh Apr 30 '12 at 14:32
2

There is no good way to preserve timestamps and not get into trouble in your build environment. Use git clone / git checkout branchB to create a second working directory for building and working with branch B. Don't have your Makefile build both, have each one build into its own build directory.

stark
  • 12,615
  • 3
  • 33
  • 50