76

For one and a half years, I have been keeping my eyes on the git community in hopes of making the switch away from SVN. One particular issue holding me back is the inability to lock binary files. Throughout the past year I have yet to see developments on this issue. I understand that locking files goes against the fundamental principles of distributed source control, but I don't see how a web development company can take advantage of git to track source code and image file changes when there is the potential for binary file conflicts.

To achieve the effects of locking, a "central" repository must be identified. Regardless of the distributed nature of git, most companies will have a "central" repository for a software project. We should be able to mark a file as requiring a lock from the governing git repository at a specified address. Perhaps this is made difficult because git tracks file contents not files?

Do any of you have experience in dealing with git and binary files that should be locked before modification?

NOTE: It looks like Source Gear's new open source distributed version control project, Veracity, has locking as one of its goals.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Mario
  • 2,229
  • 2
  • 21
  • 21

17 Answers17

75

Subversion has locks, and they aren't just advisory. They can be enforced using the svn:needs-lock attribute (but can also be deliberately broken if necessary). It's the right solution for managing non-mergeable files. The company I work for stores just about everything in Subversion, and uses svn:needs-lock for all non-mergeable files.

I disagree with "locks are just a communication method". They are a much more effective method than push-notifications such as phone or e-mail. Subversion locks are self-documenting (who has the lock). On the other hand, if you have to communicate by other traditional push-notification channels, such as e-mail, who do you send the notification to? You don't know in advance who might want to edit the file, especially on open-source projects, unless you have a complete list of your entire development team. So those traditional communication methods aren't nearly as effective.

A central lock server, while against the principles of DVCS, is the only feasible method for non-mergeable files. As long as DVCS don't have a central lock feature, I think it will keep the company I work for using Subversion.

The better solution would be to make a merge tool for all your binary file formats, but that's a longer-term and ongoing goal that will never be "finished".

Here's an interesting read on the topic.

CharlesB
  • 86,532
  • 28
  • 194
  • 218
Craig McQueen
  • 41,871
  • 30
  • 130
  • 181
  • 9
    Exactly right. A DVCS isn't designed to be centrally controlled. However, it might be feasible to build a centrally controlled system on top of a DVCS, which gets you the power that most DVCSes can provide along with the central control needed in some situations. – Michael Johnson Jun 09 '10 at 15:34
  • I realise this question is getting on a bit, but vote-up as locking fundamentally doesn't make sense under a DVCS. Instead you should look at something like the 'Dictator and Lieutenants' workflow http://git-scm.com/book/en/Distributed-Git-Distributed-Workflows – Aaron Newton Jan 15 '13 at 01:25
10

I agree that locking binary files is a necessary feature for some environments. I just had a thought about how to implement this, though:

  • Have a way of marking a file as "needs-lock" (like the "svn:needs-lock" property).
  • On checkout, git would mark such a file as read-only.
  • A new command git-lock would contact a central lock server running somewhere to ask permission to lock.
  • If the lock server grants permission, mark the file read-write.
  • git-add would inform the lock server of the content hash of the locked file.
  • The lock server would watch for that content hash to appear in a commit on the master repository.
  • When the hash appears, release the lock.

This is very much a half-baked idea and there are potential holes everywhere. It also goes against the spirit of git, yet it can certainly be useful in some contexts.

Within a particular organisation, this sort of thing could perhaps be built using a suitable combination of script wrappers and commit hooks.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • 7
    The biggest problem I see is git is wholly intended to work offline. Although, as you say, you can use build custom scripts to implement this. Beyond that, I'd be tempted to have a 'lock' branch which gets pushed and pulled from a remote. All it has is the lock table, replacing the lock server. – Michael Johnson Sep 26 '08 at 17:53
  • 1
    @MichaelJohnson: You could also just have .lock- files in your main branch. That way, you could edit and unlock with one commit. – thejh Feb 09 '13 at 17:04
10

In response to Mario's additional concern with changes happening in multiple places on the binaries. So the scenario is Alice and Bob are both making changes to the same binary resource at the same time. They each have their own local repo, cloned from one central remote.

This is indeed a potential problem. So Alice finishes first and pushes to the central alice/update branch. Normally when this happens, Alice would make an announcement that it should be reviewed. Bob sees that and reviews it. He can either (1) incorporate those changes himself into his version (branching from alice/update and making his changes to that) or (2) publish his own changes to bob/update. Again, he makes an announcement.

Now, if Alice pushes to master instead, Bob has a dilemma when he pulls master and tries to merge into his local branch. His conflicts with Alice's. But again, the same procedure can apply, just on different branches. And even if Bob ignores all the warnings and commits over Alice's, it's always possible to pull out Alice's commit to fix things. This becomes simply a communication issue.

Since (AFAIK) the Subversion locks are just advisory, an e-mail or instant message could serve the same purpose. But even if you don't do that, Git lets you fix it.

No, there's no locking mechanism per se. But a locking mechanism tends to just be a substitute for good communication. I believe that's why the Git developers haven't added a locking mechanism.

Michael Johnson
  • 2,287
  • 16
  • 21
  • 104
    Any source control system is a better way to communicate between developers, because it's structured. Email, chat or phone is worse because it's not structured. So when people say that they will resort to communication by email, chat or phone instead of using scm, it is wrong. Keeping source code and organizing communication between developers are 2 parts of any SCM and git solves only one part when svn solves both. – alpav Jan 06 '10 at 17:18
  • 7
    The important point in my mind is that a locked file is read-only on disk, and an unlocked file is RW. This means when someone tries to edit a locked file, their editor will at least warn them the file is RO. At this point they are prompted to communicate with whoever has locked the file, to find out if their changes are redundant, complementary, or incompatible. Without the VCS changing file permissions, the user isn't automatically prompted to communicate, and it's left up to their fallible memory and procedures. – KeyserSoze May 04 '10 at 21:19
  • @keysersoze: But the point of a DVCS (such as git) is that everyone is free to make changes as desired. So any sort of lock would have to be advisory only. What you're looking for is a centrally managed VCS (Perforce, TFS). I don't think DVCSes are designed to do what you're looking for. – Michael Johnson Jun 09 '10 at 15:32
  • 53
    The typical git response of "it's a communication problem, so it has nothing to do with git" does not realise that "locking" *is effective* communication of one's intent to be only person working on a file at a given time - most likely because it's a complex binary file which is very difficult (impossible) to merge. This is a perfectly valid and reasonable requirement in a large team working on binary assets. At the very least being able to lock a file on a named branch would be very useful. This message could be propogated up to origin, origin's origin, etc... – Matt Connolly Dec 09 '10 at 04:21
  • 1
    Totally agree with alpav & Matt Connolly - expecting developers/designers etc to communicate about version control OUTSIDE of the version control system, in order to make the version control process work, is ridiculous. Alice seeing that Bob has locked the file and then emailing him asking why/when it will be unlocked is a much more effective form of communication. – Max Williams Sep 16 '11 at 16:11
  • This idea works when you are managing a heard of sheeps who will follow you and obey you nicely but not when you are handling a group of wild cats . you need some sort of controll over your code at some point of time. – Krishna Prasad Varma Mar 28 '12 at 07:42
  • Even with good communication I have no way to remind myself (within the tool) that "hey, maybe you should avoid superficial changes to this file, until Alice has made her major changes." I'd love to be able to add a meaningless flag to files (that I can interpret anyway i wish). – bshirley May 30 '12 at 14:58
  • 21
    -1 This does not answer the question. The (implicit) idea in the question is to lock files to make other aware you are working on a file *before they edit it*. What you describe is the standard git conflict resolution, which - while very helpful - only works *after* the conflict has occured. – sleske Sep 21 '12 at 10:31
  • 6
    So...on a DVCS project with 100 users, most of whom I don't necessarily "work" with, who do I send an email to when I want exclusive access to a binary file? – iheanyi Mar 12 '14 at 15:01
  • 1
    -1: it's not always possible or practical to reach all potential modifiers by other forms of communication than locking. Sorry git, there is no excuse :-) – gatinueta Mar 26 '14 at 14:27
10

Git LFS 2.0 has added support for file locking.

With Git LFS 2.0.0 you can now lock files that you're actively working on, preventing others from pushing to the Git LFS server until you unlock the files again.

This will prevent merge conflicts as well as lost work on non-mergeable files at the filesystem level. While it may seem to contradict the distributed and parallel nature of Git, file locking is an important part of many software development workflows—particularly for larger teams working with binary assets.

Community
  • 1
  • 1
osowskit
  • 5,904
  • 2
  • 29
  • 38
8

We've just recently started using Git (used Subversion previously) and I have found a change to workflow that might help with your problem, without the need for locks. It takes advantage of how git is designed and how easy branches are.

Basically, it boils down to pushing to a non-master branch, doing a review of that branch, and then merging into the master branch (or whichever the target branch is).

The way git is "intended" to be used, each developer publishes their own public repository, which they request others to pull from. I've found that Subversion users have trouble with that. So, instead, we push to branch trees in the central repository, with each user having their own branch tree. For instance, a hierarchy like this might work:

users/a/feature1
users/a/feature2
users/b/feature3
teams/d/featurey

Feel free to use your own structure. Note I'm also showing topic branches, another common git idiom.

Then in a local repo for user a:

feature1
feature2

And to get it to central server (origin):

git push origin feature1:users/a/feature1

(this can probably be simplified with configuration changes)

Anyway, once feature1 is reviewed, whomever is responsible (in our case, it's the developer of the feature, you could have a single user responsible for merges to master), does the following:

git checkout master
git pull
git merge users/name/feature1
git push

The pull does a fetch (pulling any new master changes and the feature branch) and the updates master to what the central repository has. If user a did their job and tracked master properly, there should be no problems with the merge.

All this means that, even if a user or remote team makes a change to a binary resource, it gets reviewed before it gets incorporated into the master branch. And there is a clear delineation (based on process) as to when something goes into the master branch.

You can also programmatically enforce aspects of this using git hooks, but again, I've not worked with these yet, so can't speak on them.

Michael Johnson
  • 2,287
  • 16
  • 21
  • 2
    The technology in a microwave was not intended to be used for heating food. Are you telling me, since git was not originally designed for my workflow (and a lot of people's workflow), I should not use git as a DVCS? You do realize the "pull request" was to create tiers of developers who had different levels of authority/trust on a project. For many of us, we work on projects where most engineers have the same authority, there are relatively few engineers so the work that each person does is critical to the whole and can't be left pending indefinitely. – iheanyi Mar 12 '14 at 15:09
  • @iheanyi A pull request workflow works well on the type of team you describe (usually any developer is allowed to merge someone else’s pull request). – Marnen Laibow-Koser Jun 15 '18 at 15:57
  • @MarnenLaibow-Koser not at all. What you've described inverts the workflow. Now I have merge someone else's changes as opposed to everyone being responsible for their own merging. – iheanyi Jun 15 '18 at 18:57
  • @iheanyi That’s a benefit. The idea is that that no one merges their own changes into master, in order to make sure that someone else is aware of them and approves them. And it doesn’t invert the workflow: you’re still merging the pull requests into master, not in your own branch. • But anyway, it’s also not necessary to do that in order to work with Git. You absolutely could do a feature branch workflow in Git where everyone merges their own changes and so there are no pull requests. I wouldn’t advise it, but Git supports it just fine. – Marnen Laibow-Koser Jun 15 '18 at 19:23
  • 1
    @MarnenLaibow-Koser benefit for some, not others. I'm starting to repeat myself. – iheanyi Jun 19 '18 at 14:01
  • @iheanyi If you don’t think that having more eyeballs on the code (within reason) is generally a benefit, I don’t know what to say either. :) – Marnen Laibow-Koser Jun 22 '18 at 17:08
  • @MarnenLaibow-Koser If I spent all my time eyeballing code, I'd never write code. . . – iheanyi Jun 22 '18 at 17:33
  • @iheanyi Reviewing code is a fundamental part of development, not a distraction from it. Writing code is not the only, and maybe not even the most important, thing that we should be doing as developers. But I think we’re losing the main topic here. :) – Marnen Laibow-Koser Jun 22 '18 at 19:36
  • @MarnenLaibow-Koser indeed, my original point was that we shouldn't let ourselves be constrained to using git based on how it was ***intended*** when originally designed and using it in a different manner is not necessarily suboptimal - that really depends on your workflow/team. – iheanyi Jun 22 '18 at 22:55
5

It's worth examining your current workflow to see if locking images is really necessary. It's relatively unusual for two people to independently edit an image, and a bit of communication can go a long way.

Khoth
  • 13,068
  • 3
  • 28
  • 27
5

When I was using Subversion, I religiously set the svn:needs-lock property on all binary and even the hard-to-edit text files. I never actually experienced any conflicts.

Now, in Git, I don't worry about such things. Remember: locks in Subversion aren't actually mandatory locks, they are merely communication tools. And guess what: I don't need Subversion to communicate, I can manage just fine with E-Mail, Phone and IM.

Another thing I did, is to replace many binary formats with plain text formats. I use reStructuredText or LaΤΕΧ instead of Word, CSV instead of Excel, ASCII-Art instead of Visio, YAML instead of databases, SVG instead of OO Draw, abc instead of MIDI, and so on.

Jörg W Mittag
  • 363,080
  • 75
  • 446
  • 653
  • 6
    I was thinking you were serious until I read "ASCII-Art for Visio" :/ (Maybe you were. What's the tool that you use to replace Visio other than good ol' Vi?) – kizzx2 Nov 30 '10 at 13:55
  • 9
    @kizzx2: the main tool I use is a good programming language that is readable enough that I don't need elaborate diagrams to understand WTF is going on. More imporantly, I try to write readable code. A good IDE which can infer diagrams from the code, instead of me having to maintain them separately by hand. For simple UML diagrams, I can use something like [yUML](http://yUML.Me/) which supports Class, Activity and Use Case Diagrams. For simple graphs, I use [Diagrammr](http://Diagrammr.Com/) which builds graphs from simple sentences and [GraphViz](http://GraphViz.Org/) for complex graphs. – Jörg W Mittag Nov 30 '10 at 14:09
  • 1
    Diagrammr seems really interesting! Thanks! – kizzx2 Dec 01 '10 at 02:21
  • 2
    Actually replacing to text format doesn't solve the problem. Some binary files (such as pure bitmap) can be merged smoothly. The point is internal structure and dependency. If you have any XML file which depends on link for the other internal nodes and needs integrity on that linkage, it can't be merged. Usually, most complex data formats uses this kind of internal linking like a graph-database. – eonil Nov 28 '11 at 05:07
  • 1
    The open source equivalent of yUML is [Plant UML](http://plantuml.sourceforge.net/index.html) – Mystic Feb 17 '12 at 17:20
  • @JörgWMittag So, you switched to "text" files. Good for you. What about the rest of us who work on things where that is not an option? What about those of us who actually experienced conflicts with binary files when svn:needs-lock was not set? – iheanyi Mar 12 '14 at 15:04
5

I have discussed this issue on git discussion groups and have concluded that at this time, there is no agreed upon method of centralized file locking for git.

Mario
  • 2,229
  • 2
  • 21
  • 21
2

I would not expect file-locking to ever make it as a feature in git. What kind of binary files are you primarily interested in? Are you actually interested in locking the files, or just preventing conflicts caused by not being able to merge them.

I seem to remember someone talking (or even implementing) support for merging OpenOffice-documents in git.

JesperE
  • 63,317
  • 21
  • 138
  • 197
2

This is not a solution but rather a comment on why locking mechanisms are needed. There are some tools used in some fields which use binary only formats which are flat out mission critical and the "use better/different tools" is just not an option. There are no viable alternate tools. The ones I'm familiar with really wouldn't be candidates for merging even if you stored the same information in an ascii format. One objection I've heard is that you want to be able to work offline. The particular tool I'm thinking of really doesn't work offline anyway because of needing to pull licenses so if I have data on a laptop it isn't like I can run the tool while on a train anyway. That said, what git does provide if I have a slow connection, I can get licenses and also pull down changes but have the fast local copy for looking at different versions. That is a nice thing that the DVCS gives you even in this case.

One view point is that git is simply not the tool to use but it is nice for all the text files which are also managed with it and it is annoying to need different version control tools for different files.

The sort-of-advisory-locking-via-mail approach really stinks. I've seen that and have been tired of an endless stream of emails of "I'm editing it" "I'm done editing" and seen changes lost because of it. The particular case I'm thinking of was one where a collection of smaller ascii files would have been much nicer but that is an aside.

Dan
  • 29
  • 1
  • Downvoting. Use of binary files does not require locking. Git even supports custom merge drivers for binary files (well, for any file); if nothing else, there’s https://github.com/synapsepd/bump-merge. – Marnen Laibow-Koser Jan 29 '21 at 15:03
1

What about cad files? If the files aren't locked, to be kept read-only as well, most cad programms would just open them an change arbitrary bits, seen as a new file by any vcs. So in my view, locking is an ideal means for communicating your intend to change some particalur file. As well, it prevents some Software to gain write access in the first place. This allows updates of the local files, without the need to close the software or at least all files entirely.

1

TortoiseGit supports full git workflow for Office documents delegating diff to Office itself. It works also delegating to OpenOffice for OpenDocument formats.

Antonio Bardazzi
  • 2,996
  • 1
  • 23
  • 20
0

Im not suggesting to use git at my company for the same problem. We use EA for all our designs and microsoft word for documentation, we don't know in advance who may edit a particular file so exclusive locking is our only option.

Hernan Rajchert
  • 1
  • 1
  • 3
    I think the better longterm solution would be to use better tools. A good wiki would go a long way, or just using something that isn't storing binaries (HTML, TeX, etc). Locking is nice, but it seems most people are just wanting to use locking because binary diffs are hard to handle, but for most of these uses there is no reason to store binaries. You keep source code in git, not dlls/sos and executables, so why store compiled versions of documents? – semi Nov 25 '10 at 19:13
0

git will work very well in a non-team environment where each developer is solely responsible for a piece of code or file, because in that case communication about locks is not needed.

If your organization requires team environment (usually to strip developers from job security), then use svn, git is not for you. Svn provides both - source control and communication between developers about locks.

alpav
  • 2,972
  • 3
  • 37
  • 47
  • 2
    A lot of git is specifically designed for teams, that's one area (among many others) where git is miles ahead of SVN. Locking is not as easy as SVN for this type of situation however there are features that would help such as merge drivers. – shmish111 Jul 09 '14 at 10:02
  • @shmish111: Communication of locks between developers is essential part of a team environment, why do you think "this type of situation" does not need to be covered ? Svn allows developers to communicate locks/unlocks, Git does not. Git should've made it optional, but available feature. – alpav Dec 14 '14 at 20:44
  • 1
    As I said, git is weaker than SVN when it comes to locking. I have only come across this requirement once and it turned out that we didn't need to do it in the end. I would suggest that often (not always) when a file needs to be locked, that is a good indication that you could be organizing your project better. Git is designed specifically for teamwork so to say that it's not for a team environment is just crazy. To say that team environments are made to strip developers of job security is unbelievably crazy! – shmish111 Dec 15 '14 at 12:37
  • @alpav “Communication of locks between developers is essential part of a team environment” Only if locks are necessary in the first place. In general, they’re not. (I’ve worked without locking for 20 years quite happily. I don’t see why I’d want it.) – Marnen Laibow-Koser Sep 23 '18 at 04:18
0

It might be true, that reorganising a project can help avoiding locks, but:

  • Teams are also organised by other priorities (location, customers, ...)
  • Tools are also selected by other targets (compatibility, price, ease of use by most of the employees)
  • Some tools (and therefore there binary files) cannot be avoided, as there is simply no replacement that can do the same job, fitting the same to the companys needs for the same price.

To request, that a whole company might reorganise their workflow and replace all their tools that produce binaries, only to be able to work with git, because of the lack of locks, sounds quite inefficient.

Locks do not fit into the git philosophy (which was never made for binaries), but there are non-neglectable situations, where locks are the most efficient way to solve such a problem.

Stefan
  • 9
  • 1
0

Just put a text file in cc with the file that you want to lock and then have the update hook reject it.

Remote Shell
  • 9
  • 1
-1

Git is not providing any command to lock files but I've fund a way to achieve that function using git hooks. An auxiliary server is needed to store the lock informations. We can use a pre-commit hook to check if any of the committed files is locked. And if anyone locks a file, a program should tell the auxiliary server the information of the locker and the locked file.

Cherler Ton
  • 1