7

Context

I'm the main author of Next Right Now which is an open source "boilerplate" containing several "presets" of building a web app using the Next.js framework. Each preset comes with built-in features and is meant to be forked so other can build their app based on it. Each preset lives in its own git branch, such as:

I'm working on NRN and making it evolve regularly. But, I also have forked one of the available NRN presets and made my own -proprietary- app from it.

Definitions

Here are some definitions in order to avoid terminology misunderstandings.

Problem

The problem with this way of doing things is that I'm not sure how to keep the "Fork" in sync with the NRN boilerplate preset. Both evolve in their own way. Also, NRN is not a framework but a boilerplate, which is meant to be overridden to customize the base code, and this eventually leads to lots of conflicts between a Fork and the Source.

What I've been doing so far

In order to keep my Fork synced with the latest changes on the Source, I basically rebase my own work on top of the Source git history. (e.g: git rebase NRN-v2-mst-aptd-at-lcz-sty)

This has following advantages (pros):

  • It keeps the history clean and simple to understand/compare. I can rather know easily which was the latest commit I synced from the Source by comparing their history. All the work done in the Fork is done on top of what's been done in the Source.
  • The git tree is separated in two distinct parts, the Source commits tree and the Fork commit tree.
  • I can sync the new changes done from the Source into the Fork by using git rebase to get my Fork up-to-date and then push --force to override the remote.

But also a few disadvantages (cons):

  • It's not so complicated to deal with syncing between both branches when there is only one branch in the Fork, but it gets very messy as soon as there are several, because it rewrites the git history of all branches, it gets quite complicated when there is ongoing work on other "feature" branches in the Fork. First, I need to rebase the Fork:master and then rebase every branch with the Fork:master. If I do it the wrong way around it messes up the whole tree (I did the mistake once, and it was 2 painful hours of rebasing around with --force everywhere)
  • It uses --force on the Fork:master branch, which is not so great IMHO and can lead to quite a few troubles if not handled correctly. I'm a bit familiar with what I'm doing, but this wouldn't be viable if there were more people in the team.
  • Overall, I'm not confident in my own ability not to mess it up someday.
  • It doesn't feel adapted to a team, it works only because I'm solo working on this, IMHO.
  • When it conflicts, it can be painful to resolve and I happened to make mistakes on several occasions.
  • The git history is untrustworthy, my Fork working branches get their commit history rewritten upon sync with the Source, and all GitHub comments lose their usefulness because they don't match with any commit anymore.

Using rebase, I eventually had to wipe my whole working branch and recreate it from the Source by cherry-picking all commits I had done in the Fork, because the history didn't match anymore and I needed a clean start. This happened after I made a few mistakes by rebasing the wrong way around.

What I'm looking for

My current way works fine, as long as I'm solo, as long as I know my git branches well, as long as I don't mess it up by rebasing and pushing with --force the wrong way around. It doesn't satisfy me, though.

I'm looking for a better way, which would be usable for a team, and which I could use as the "officially recommended" way to keep a Fork in sync with its Source for NRN.

Alternatives

I've thought about cherry-pick-ing commits from the Source to my Fork, but I'm not sure if it's a better alternative, because it'd mix both Source and Fork commits together (no separation between both anymore). This would eventually lead to difficulties while comparing both trees and figuring out which commits have been cherry-picked and which haven't. Also, it doesn't protect me against forgetting to cherry-pick one commit and run into trouble weeks after that, which might lead to using --force to rewrite the history to include the missing commits at the right place.

I haven't considered any other alternative, as I don't know any.

So, I'm looking for "best practices" for my particular use-case. I'm pretty sure Git has some awesome ways of dealing with that, which are unknown to me.

Vadorequest
  • 16,593
  • 24
  • 118
  • 215
  • 2
    Could you also elaborate on why you didn't use `merge` for getting the updates from the source to the fork? That seems the most natural solution to your problem. – František Hartman Nov 02 '20 at 15:39
  • 1
    Perhaps `git merge --no-ff branch` to keep the merge history in one commit (see also `--no-commit`). – Christoph Nov 02 '20 at 15:42
  • But, if I use merge then won't it be worse? I'll have a real hard time to know which commits have been synched between both repo, and clutter the tree with merge commits too. – Vadorequest Nov 02 '20 at 20:51

3 Answers3

2

I see several options:

Merge

As some suggested the simplest option to "update" the fork with new commits on the original(root) repository is to merge. This will make sure:

  • you get the latest fixes from the root repository library/framework/ easily
  • your commits in the fork and the ones in the root are cleanly separated

I would discourage rebase for this particular problem. As you mention, the history of your forked repository will be effectively modified, and that could affect other developers working there / feature branches (even on a mono-developer repo) etc...

If you have to merge patches in the opposite direction, fork -> root, you would then git cherry-pick

git submodule

Another option is to have the base library/framework as a git submodule in the fork. In the standard form, a git submodule is just a pointer to another repository+commit. Histories are separated, as they are indeed two different repositories.

To integrate new changes on the base, you just need to repoint the git submodule to this new commit.

One important note; this would only work well if your forked repository doesn't touch the files of the root repository.

git subtree

I am not familiar enough with git subtree to be able to judge. But you should probably have a look too, as it sounds like another viable solution

msune
  • 231
  • 1
  • 3
  • Merging within a PR that gets squashed is appealing, as it would keep the history rather clean, and would allow testing the merge in a distinct branch on the Fork. The `submodule` strategy won't do, because the Fork might (very likely) update the same files as the Source. I'll learn more about the `git subtree` command, thanks for the hint. – Vadorequest Nov 07 '20 at 20:06
  • @Vadorequest Yes, I thought so, just seeing that you keep two separate branches for essentially two variants of the same library/framework. (early enter) In sort of related environments, I've seen the following; using submodules (and merging both variants into the same branch, in different repositories), and then on the fork(s) create a small tool to "instantiate" the template. The same tool should allow to "apply" or "merge" (not git merge) the new commits of the root to the instantiated template. Just some food for thought – msune Nov 07 '20 at 20:43
  • 1
    Erratas: a small typo here "in different repositories", I meant in the same repo. And the instantiation tool would typically be in the "root" repository, but used by forks. – msune Nov 07 '20 at 20:51
  • Yeah, I see what you mean. Not sure I'll go this way considering the complexity behind such tooling, though. – Vadorequest Nov 08 '20 at 10:43
  • 1
    I'm awarding you the bounty as your answer solves part of my overall issue, and answers the particular topic my bounty was about. Thank you! (still gotta try the merge strategy for real, though) :) – Vadorequest Nov 08 '20 at 10:44
1

git --force-with-lease is a safer option that will not overwrite any work on the remote branch if more commits were added to the remote branch (by another team-member or coworker or what have you). It ensures you do not overwrite someone else's work by force pushing.

Rebase alone can still be a good option if every feature goes through a pull request, and no changes are directly made on the master branch

Simon
  • 369
  • 1
  • 9
  • 2
    Even better (soon): `git push --force-with-lease --force-if-includes`, [starting with Git 2.30](https://stackoverflow.com/a/64627761/6309). – VonC Nov 03 '20 at 11:18
  • This seems better than `git push --force`, but seems to have some downsides if `git fetch` isn't performed before as mentioned in https://stackoverflow.com/a/52823955/2391795, but it doesn't really help me with an actionable answer, it only treats part of the problem. – Vadorequest Nov 05 '20 at 23:14
0

After experimenting for a while, here are my findings:

  1. Rebase is not a good solution, as I first thought. It's to be dismissed in such case, as it'll rewrite the history and severely complicate collaboration.
  2. Merge is a good alternative. But, in order not to have "merge" commits - which personally, I dislike very much -, the trick is to create a branch in the Fork, and then merge the Source within that branch. Then, squashing the branch when merging will keep the history neat.

The merge alternative also provides the benefit of testing the code in a dedicated branch, which is essential when synching such projects, because changes are coming both ways, and even though there might not be obvious "conflicting code", the behaviour might be affected. (and only automated testing will be able to detect it automatically, whether unit, integration or E2E tests)

While the "merge" strategy is the most common (because it's the default and simplest), I'm really not familiar with it anymore because I always use rebase to keep the tree history clean. But merging within a branch is perfectly fine, and that's what I also do when I sync a branch with master, to avoid having to resolve too many conflicts. Thanks for your help!

Vadorequest
  • 16,593
  • 24
  • 118
  • 215