-2

I like Mercurial a lot better for its consistent command line, but git has a feature which I very much miss:

Can Mercurial simply ignore an .hg folder within a subdirectory, in order to track all files below it 'normally'?

With git it is possible and it does not even require a special switch!

Edit: Somewhat sad, that one gets downvoted for simply asking if the feature is present in merc as well.


Details:

Current Behaviour

As soon an .hg directory is created within the tree, Mercurial stops tracking all content from its containers directory, treating it as another independently managed repo by default:

/tmp$ mkdir -p a/b/c/ && touch a/b/c/foo && hg init a && cd a && hg status
? b/c/foo
/tmp/a$ mkdir b/.hg && hg status
/tmp/a$ 

Merc. thinks now there is another repo tracking all within ./b, by presence of .hg folder. But can I switch this off? Even if I explicitly ignore b/.hg:

/tmp/a$ echo -e "syntax: glob\nb/.hg*" > .hgignore
/tmp/a$ hg status
? .hgignore

the main repo does not track below ./b anymore.

I would like to have ALL files below a tracked within a/.hg, no matter if someone created a .hg folder within.


git:

/tmp$ mkdir -p a/b/c/ && touch a/b/c/foo && \
        git init a && cd a && git add -A &&  git status
Initialized empty Git repository in /tmp/a/.git/
#   new file:   b/c/foo
/tmp/a$ git init b
Initialized empty Git repository in /tmp/a/b/.git/
/tmp/a$ git status
#   new file:   b/c/foo   # <- still tracked.

```


Two Potential Use Cases

Disclaimer: The comments show that the information given in these use cases alone is NOT sufficient to justify the requested feature of my question: For case A as well as B, subrepos COULD be the right answer (definitely not in my specific case though). I still leave the use cases in, in order to demonstrate the directory structure but I pls. mind the main question of this post.

A) Different commit purposes, intervals, periods, policies, skills within one large project:

Juniors/Partners                      Seniors, Admins
 contribute to subdir(s)              overlook the whole project
  |                                   |
  |                                   |
  |             LargeProject/         |
  |             ├── .hg <------------ 'official' commits
  |             ├── conf              high quality, customer tracks,
  |             ├── lib               show up in release notes.
  |             ├── (...)
  |             └── frontend,contrib,...
  └-freq. commits-> └── .hg
    of frontend     └── js
    stuff           └── css
                    └── (...)
  • The project is mature and stable in general but at certain periods work in not critical part(s) happens, in subdirectories, which only after contributers give a go, should show up as official commits.
  • Which parts are handled like this is changing over time and not known in advance.
  • For topological reasons all the work happens on one machine in the DEV or staging zones of the customer.
  • From time to time a tagged push to the central repo to transfer to (pre)production.

If another VCS would be used for the partner stuff (with its meta files ignored in the global repo), it would all work like expected - except that I would like to use (and explain & train) Mercurial only for all contributions.


B) Using hg solely as Filesystem change tracking tool only - within a project repo.

Within my hg project directory I have a subdirectory, containing dumped files of an object database as integral part of the project.

Now I would love to use hg within that one directory, for some of its tracking features (basically for tracking transactions within the OODB and being able to push back changes on the filesystem into it, maybe after merges). I need frequent commits here - but don't want to explode the history of the official project repo with those.

Problem is just: As soon I run an hg init there, the main repo outside is stopping to track the directory.

I DON'T want subrepos since I'm really not pushing or pulling anything there, as said it is just for local OODB changes tracking. Also I don't want the caveats of subrepos, just because I require those tracking features within the subdir, incl. automatic commits.

My current work around is to use git like shown, that way the main repo does display any changes of the OODB just like for any other file of the project:

my_project/
├── .hg   #  Main project repo, 'offical commits'
├── conf  # 'Normal' project repo folders
├── lib
└── zodb  # Containing OODB dumps
    ├── .git        # Repo to auto track transactions of the OODB,
    └── zodb_root   # frequently dumped within this directory, and
        └── tenant1 # often committed, for diffing / merging reasons.  

But I would really love to use .hg as well for the inner 'tracking only' directory as well.

The most simple solution would be if I could configure mercurial to ignore the .hg in the OODB directory. Tried with unshared bindmounts but alone the empty mountpoint directory is enough to make hg refuse to look into the directory.

Any Ideas out there? A simple solution would be if Mercurial would respect *.hg within the global .hgignore, like it does for *.git - but it doesn't.

Red Pill
  • 511
  • 6
  • 15
  • possible duplicate of [Mercurial repo inside a repo](http://stackoverflow.com/questions/3028777/mercurial-repo-inside-a-repo) – dimo414 Jun 26 '15 at 22:13
  • Or [Mercurial Subrepos - How do you create them and how do they work?](http://stackoverflow.com/q/2083393/113632). – dimo414 Jun 26 '15 at 22:13
  • Hi did see both - and both are not the answer of what I need: The outer repo should treat ALL content as part of it and neither ignore it (as would be the case with simply running hg init inside) - or pull it only recursively (as would be the case with subrepos). – Red Pill Jun 26 '15 at 22:35
  • Tracking the high frequent mostly automatic commits within a nested git repo is giving me exactly what I require - except that I would prefer to work also on the inner one with Mercurial and question is how. – Red Pill Jun 26 '15 at 22:37
  • Don't version databases with VCS. It offends the database gods (and will not give erroneous results. Database intrinsic tools won't screw up the database by making a backup. – msw Jun 26 '15 at 22:52
  • thanks but I'm talking about an _Object_ database with purely business logic inside (Zodb). We get notified about each of its transactions (which happen far less frequent than those for the actual data, basically only when business logic changes) and dumped cleartext to the FS is giving us the best of both worlds. Independent of that UC: I do see other use cases for more frequent commit cycles within a subdir of a big repo, without wanting to induce an 'official' commit each time. – Red Pill Jun 26 '15 at 23:01

3 Answers3

0

I found how it is done:

For the subrepo you create a repo anywhere and symlink its .hg file over to the subdirectory where you want to have the independent tracking:

/tmp$ mkdir -p a/b/c/ && touch a/b/c/foo && hg init a && cd a && hg status
? b/c/foo
/tmp/a$ mkdir b/sub && hg init b/sub && ln -s /tmp/a/b/sub/.hg /tmp/a/b/.hg
/tmp/a$ cd b && hg add * && hg commit -m subcommit && hg status
adding c/foo
committing files:
c/foo (...)
/tmp/a/b$ hg status
/tmp/a/b$ cd ..
/tmp/a$ hg status
? b/.hg
? b/c/foo

-> Can now independently track changes within a/b while still everything is 'under the hood' of main repo in a :-)

Red Pill
  • 511
  • 6
  • 15
0

Independent whether it's a good idea to track databases: The structure you show in example A) but also the OODB one calls for the proper use of sub-repositories instead of the need to commit the whole .hg dir. The parent repository records the state of the sub-repository, thus a commit at the parent can also be to update to a new(er) version of the sub-repository.

Tracking the .hg dir by whatever means in the parent dir is definitely a very bad idea.

planetmaker
  • 5,884
  • 3
  • 28
  • 37
  • I seem to be not able to transfer that – Red Pill Jun 30 '15 at 20:34
  • ... there IS just one repo. There is no justification whatsoever to split those directories off this repo. Are you aware of the caveats of subrepos? Pls. read, they mention those very prominently. Care to explain why the need to track is ALWAYS EQUAL with the need to push and manage independenty? I *my* use case it is irrelevant that they built convenience features around subrepos or not, like the commit. For me it is *very* relevant that I don't want to push what I track in those subdirectories - except the main repo has a state which is worth pushing. – Red Pill Jun 30 '15 at 20:43
  • I could also just say: Hey, git offers this feature - and I was asking how I get it in merc. That I tried to give examples why one could want it - I begin to regret und I understand that they might be misleading regarding what I asked for actually => sorry. – Red Pill Jun 30 '15 at 20:48
  • I know sub-repositories from my own usage and experience - and I recommend to avoid them when possible. However I maintain that you with your use case are actually asking for them and that they are the best solution to what you ask. You commit changes to the sub-repo locally when you need them. And commit at the parent repo when you see a 'stable' state or whatever. Then you push both repos where the parent repo records the state of the sub-repo. – planetmaker Jun 30 '15 at 23:06
  • So, we are both clear about the benefits but also the overhead of subrepos. Not so about the reason for .hg dirs, sorry, symlinks within the directory structure: In B I'm using merc. as a (fast) *filesystem state tracker* - to be able to diff current state of the OODB in cleartext against that of last week, Monday, 10:am. Plus to push back edits on the FS back to Zodb in a transactional way. Not more not less. A daemon is running a commit in the helper repo at every commit into the OODB - fully automated. – Red Pill Jun 30 '15 at 23:43
  • The customer is pretty interested in what we do in the whole project and monitors commits which are referring to ticket numbers he creates - and which are typically affecting both OODB AND FS (App server interplays with worker daemons). He wants to see what was done for ticket xyz - and is zero interested in what was autocommited by the OODB transaction monitor. – Red Pill Jun 30 '15 at 23:48
  • In A: I maybe should have mentioned that we distribute work for our partners case by case. This month on the frontend, then another time maybe for westbound systems communication (...) The sub directories they work at are changing over time. We can't rip out subdirs and history of the whole project, just because a partner wants to track / rollback changes while working at a certain use case on the DEV cluster. Is it clearer now? – Red Pill Jun 30 '15 at 23:53
  • It seems to me that you haven't quite understood how sub-repos work. Give your customer the parent repo to check and maintain the sub-repo with your OODB commits without updating the parent repo each time you change something there. But as you're already sold on "I want to track the entire .hg dir" instead of solutions, I'll give up at this stage. However if you need so many comments to explain your question you likely are better of editing your question. – planetmaker Jul 01 '15 at 07:20
  • Thanks I made the intent of the question clearer now hopefully. ---- PS: I'm using subrepos as well as submodules with git since quite some years now, so I do think I understand the up and downsides of independent tracking of dependent repos. And still I claim that DVCS with their effiency regarding FS change tracking is sometimes pretty useful also w/o the 'D' - and then subrepos are at least questionable for each and any case, to say the least. – Red Pill Jul 01 '15 at 15:06
0

The solution with fooling mercurial with an .hg symlink, pointing to the actual merc directory has a downside: People accidentially checked the .hg symlink into the main repo and then multidimensional mess occurs. A nested .hg should never ever make it into a changeset!

Still I personally could not accept to have trained everybody on mercurial (overruling the git lovers, which gives you this specific feature by default) and then switch to git myself because nested local dir state mgmt does not work.

So I decided to patch the pathutil of mercurial itself. For everybody in the same situation, who can live with the obvious downsides of a patch, here is a clumsy but working overwrite of the default when the environment variable $HG_OVERWRITE_SKIP is set.

Note: This will most likely not allow to have subrepos anymore, when the environ var is set, which is not relevant for me.

HG version: 3.4.1.

7.oodblogd.DEV.NB01:/agent/lib/python2.7/site-packages/mercurial# hg diff .
diff -r 0548c0ff6c76 lib/python2.7/site-packages/mercurial/dirstate.py
--- a/lib/python2.7/site-packages/mercurial/dirstate.py Mon Jul 27 00:05:50 2015 +0000
+++ b/lib/python2.7/site-packages/mercurial/dirstate.py Thu Jul 30 22:16:39 2015 +0000
@@ -16,6 +16,12 @@

 dirstatetuple = parsers.dirstatetuple

+# AX: making hg NOT ignore dirs with a nested .hg via this environ flag,
+# which you set when you want to track nested dirs:
+nested_hg_skip_match = '.hg'
+if os.environ.get('HG_OVERWRITE_SKIP'):
+    nested_hg_skip_match = '.no_match_hg'
+
 class repocache(filecache):
     """filecache for files in .hg/"""
     def join(self, obj, fname):
@@ -772,7 +778,7 @@
                 if nd == '.':
                     nd = ''
                 else:
-                    skip = '.hg'
+                    skip = nested_hg_skip_match
                 try:
                     entries = listdir(join(nd), stat=True, skip=skip)
                 except OSError, inst:
diff -r 0548c0ff6c76 lib/python2.7/site-packages/mercurial/pathutil.py
--- a/lib/python2.7/site-packages/mercurial/pathutil.py Mon Jul 27 00:05:50 2015 +0000
+++ b/lib/python2.7/site-packages/mercurial/pathutil.py Thu Jul 30 22:16:39 2015 +0000
@@ -4,6 +4,13 @@
 import util
 from i18n import _

+# AX: making hg NOT ignore dirs with a nested .hg via this environ flag,
+# which you set when you want to track nested dirs:
+nested_hg_skip_match = '.hg'
+if os.environ.get('HG_OVERWRITE_SKIP'):
+    nested_hg_skip_match = '.no_match_hg'
+
+
 def _lowerclean(s):
     return encoding.hfsignoreclean(s.lower())

@@ -43,7 +50,7 @@
             raise util.Abort(_("path ends in directory separator: %s") % path)
         parts = util.splitpath(path)
         if (os.path.splitdrive(path)[0]
-            or _lowerclean(parts[0]) in ('.hg', '.hg.', '')
+            or _lowerclean(parts[0]) in (nested_hg_skip_match, '.hg.', '')
             or os.pardir in parts):
             raise util.Abort(_("path contains illegal component: %s") % path)
         # Windows shortname aliases
@@ -55,7 +62,7 @@
                                      % path)
         if '.hg' in _lowerclean(path):
             lparts = [_lowerclean(p.lower()) for p in parts]
-            for p in '.hg', '.hg.':
+            for p in nested_hg_skip_match, '.hg.':
                 if p in lparts[1:]:
                     pos = lparts.index(p)
                     base = os.path.join(*parts[:pos])
@@ -87,7 +94,7 @@
                         _('path %r traverses symbolic link %r')
                         % (path, prefix))
                 elif (stat.S_ISDIR(st.st_mode) and
-                      os.path.isdir(os.path.join(curpath, '.hg'))):
+                      os.path.isdir(os.path.join(curpath, nested_hg_skip_match))):
                     if not self.callback or not self.callback(curpath):
                         raise util.Abort(_("path '%s' is inside nested "
                                            "repo %r")
(END)
Red Pill
  • 511
  • 6
  • 15