121

I want to version control my web server as described in Version control for my web server, by creating a git repo out of my /var/www directory. My hope was that I would then be able to push web content from our dev server to github, pull it to our production server, and spend the rest of the day at the pool.

Apparently a kink in my plan is that Git won't respect file permissions (I haven't tried it, only reading about it now.) I guess this makes sense in that different boxes are liable to have different user/group setups. But if I wanted to force permissions to propagate, knowing my servers are configured the same, do I have any options? Or is there an easier way to approach what I'm trying to do?

Community
  • 1
  • 1
Yarin
  • 173,523
  • 149
  • 402
  • 512
  • 1
    possible duplicate of [git - how to recover the file permissions git thinks the file should be?](http://stackoverflow.com/questions/2517339/git-how-to-recover-the-file-permissions-git-thinks-the-file-should-be) – kennytm Jul 08 '10 at 20:34
  • 1
    Yeah guess so, though the solution they point to I'm frankly not sure what to do with. Was hoping for a more straightforward approach. – Yarin Jul 08 '10 at 20:47
  • What about the situation where the source code is coming from Dev environment (e.g. Windows - XAMPP etc) which doesn't have file ownership info? Files at end of git process need to match ownership & permissions for the target location. Can git-cache-meta deal with this? Agree with Yarin ... surely this is a pretty mainstream use case, that should have a pretty straightforward solution? – user3600150 Feb 01 '16 at 12:50

10 Answers10

68

Git is Version Control System, created for software development, so from the whole set of modes and permissions it stores only executable bit (for ordinary files) and symlink bit. If you want to store full permissions, you need third party tool, like git-cache-meta (mentioned by VonC), or Metastore (used by etckeeper). Or you can use IsiSetup, which IIRC uses git as backend.

See Interfaces, frontends, and tools page on Git Wiki.

Community
  • 1
  • 1
Jakub Narębski
  • 309,089
  • 65
  • 217
  • 230
  • 2
    Thanks Jakub- can you explain to my why Git cares about the executable bit and only that? – Yarin Jul 09 '10 at 14:58
  • 6
    @Yarin: executable bit only? When you clone a all set of files from one system to another, the notion of "read only" or "read-write" is not exactly relevant (as you said in your question: different users/groups). But the notion of "executable" doesn't depend on users and groups and can be reused from system to (remote) system. – VonC Jul 09 '10 at 15:11
  • 1
    Jakub, in that case it shouldn't *change* the permissions. I mean, it should leave perms alone or manage them, but not mess with them if it's not going to manage them. – CommaToast Sep 15 '14 at 22:38
  • 3
    Additionally, I've found `/usr/share/git-core/contrib/hooks/setgitperms.perl` in my `git-contrib` package-- a script for a similar purpose. ("This script can be used to save/restore full permissions and ownership data within a git working tree.") – imz -- Ivan Zakharyaschev Feb 09 '15 at 11:41
  • Is this still accurate or does github somehow do something on top of git? I just changed a file to executable and committed it, and the changelog for the commit shows up as 0 lines changed for the file, but has 100644 → 100755 beside the filename. This really looks like the full permissions are stored with the file. – Cruncher Jan 09 '20 at 16:00
  • @Cruncher - as I wrote Git itself stores the executable permission (but not, for example, owner, group, and other more detailed permissions). In other words Git stores *partial* permissions, not full permissions. – Jakub Narębski Jan 20 '20 at 12:13
  • @JakubNarębski Yeah, I did more searching after posting that comment. It appears that github only shows two possible file modes. 100755 which corresponds to executable bit set, and 100644 which corresponds to executable bit not set. Weird that they display it with a full unix-like scheme (with a random 100 prepended). But that appears to just be a UI decision by github. I think just changing the colour of the filename would be most intuitive and not as easy to miss – Cruncher Jan 20 '20 at 14:31
47

The git-cache-meta mentioned in SO question "git - how to recover the file permissions git thinks the file should be?" (and the git FAQ) is the more staightforward approach.

The idea is to store in a .git_cache_meta file the permissions of the files and directories.
It is a separate file not versioned directly in the Git repo.

That is why the usage for it is:

$ git bundle create mybundle.bdl master; git-cache-meta --store
$ scp mybundle.bdl .git_cache_meta machine2: 
#then on machine2:
$ git init; git pull mybundle.bdl master; git-cache-meta --apply

So you:

  • bundle your repo and save the associated file permissions.
  • copy those two files on the remote server
  • restore the repo there, and apply the permission
Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • 2
    VonC- Thanks for this, I'll try it out- but is the bundling necessary? Couldn't I retain my workflow (dev -> github -> production) and just checkin/checkout the metafile? – Yarin Jul 09 '10 at 15:06
  • @Yarin: no, the bundle is not mandatory. It is a neat way to transfer repo when no other transfer protocol are available though. – VonC Jul 09 '10 at 15:09
  • 5
    The use of bundle here was a major distraction for me. It put me off the answer entirely, actually. (I don't have difficulty pulling repo from server.) @omid-ariyan's answer below with pre/post commit hooks was much more understandable. Later I realized that those hook scripts are doing exact same work as git-cache-meta. Go see what I mean: https://gist.github.com/andris9/1978266. They are parsing and storing return from `git ls-files`. – pauljohn32 Aug 30 '16 at 23:02
  • 1
    The link to git-cache-meta is dead - can someone who knows about this locate it and edit the post? – rosuav Nov 20 '16 at 08:46
  • 1
    @rosuav Sure: I have edited the answer and restored the link. Thank you for letting me know of this dead link. – VonC Nov 20 '16 at 08:51
  • Suppose somebody injects malicious code in `.git_cache_meta` file and commits it to your repo? You will execute it with `git-cache-meta --apply` call. Even deadlier when automated using git hooks. IMHO Omid's solution is better than `git-cache-meta`. – Simon Rozman Nov 06 '22 at 14:20
  • @SimonRozman 12 years later, I agree. I did upvote Omid's answer back in 2016. – VonC Nov 06 '22 at 14:57
30

This is quite late but might help some others. I do what you want to do by adding two git hooks to my repository.

.git/hooks/pre-commit:

#!/bin/bash
#
# A hook script called by "git commit" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message if it wants
# to stop the commit.

SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions

# Clear the permissions database file
> $DATABASE

echo -n "Backing-up permissions..."

IFS_OLD=$IFS; IFS=$'\n'
for FILE in `git ls-files --full-name`
do
   # Save the permissions of all the files in the index
   echo $FILE";"`stat -c "%a;%U;%G" $FILE` >> $DATABASE
done

for DIRECTORY in `git ls-files --full-name | xargs -n 1 dirname | uniq`
do
   # Save the permissions of all the directories in the index
   echo $DIRECTORY";"`stat -c "%a;%U;%G" $DIRECTORY` >> $DATABASE
done
IFS=$IFS_OLD

# Add the permissions database file to the index
git add $DATABASE -f

echo "OK"

.git/hooks/post-checkout:

#!/bin/bash

SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions

echo -n "Restoring permissions..."

IFS_OLD=$IFS; IFS=$'\n'
while read -r LINE || [[ -n "$LINE" ]];
do
   ITEM=`echo $LINE | cut -d ";" -f 1`
   PERMISSIONS=`echo $LINE | cut -d ";" -f 2`
   USER=`echo $LINE | cut -d ";" -f 3`
   GROUP=`echo $LINE | cut -d ";" -f 4`

   # Set the file/directory permissions
   chmod $PERMISSIONS $ITEM

   # Set the file/directory owner and groups
   chown $USER:$GROUP $ITEM

done < $DATABASE
IFS=$IFS_OLD

echo "OK"

exit 0

The first hook is called when you "commit" and will read the ownership and permissions for all the files in the repository and store them in a file in the root of the repository called .permissions and then add the .permissions file to the commit.

The second hook is called when you "checkout" and will go through the list of files in the .permissions file and restore the ownership and permissions of those files.

  • You might need to do the commit and checkout using sudo.
  • Make sure the pre-commit and post-checkout scripts have execution permission.
Omid Ariyan
  • 1,164
  • 13
  • 19
  • 1
    Omid ... thank you! I found your code to be a perfect solution for me. – Ricalsin Jul 04 '16 at 15:11
  • @Ricalsin You're very welcome! I'm glad to have helped :) – Omid Ariyan Jul 05 '16 at 16:48
  • 1
    `$SELF_DIR/../../` is not necessarily the root of the repository... but `git rev-parse --show-toplevel` is. (Not sure why you wouldn't just use `pwd` for the current directory, but that's moot anyway.) – PJSCopeland Aug 15 '16 at 21:50
  • As it stands, the above will split file names with spaces in them. As per [this answer](http://askubuntu.com/a/344418/288221), you can set `IFS=$'\n'` before the `for` loop to stop that (and `unset IFS` afterwards to be safe). – PJSCopeland Aug 16 '16 at 00:20
  • This doesn't readily allow you to carry the permissions to a different system, with a different OS, where you have a different user name. I asked myself "what do I really *need?"* and cut the entire solution down to `chmod 0600 .pgpass` in `post-checkout`. Yes, I will have to update it manually whenever I have a file that needs specific permissions, but them's the breaks. – PJSCopeland Aug 29 '16 at 04:07
  • I second @PJSCopeland's comment. This existing code fails if files or directories have spaces, the error messages look like this: "cannot stat '04.StartingValuesInLISREL/Open': No such file or directory". – pauljohn32 Aug 30 '16 at 19:13
  • @PJSCopeland, thanks for your comments. I updated the answer accordingly. Although I can't think of any situation where $SELF_DIR/../../ would not equal the root of the repository when $SELF_DIR is sure to be the location of the hook script. By the way, 'pwd' could refer to a different directory if a script is run from a different directory than the one it exists in. – Omid Ariyan Sep 01 '16 at 20:36
  • @PJSCopeland Regarding the issue with carrying the permissions to a different system, you're right. Unfortunately one has to keep in mind to make sure the user and group names exist on the machine where the repository is cloned on. I don't see a way around it. – Omid Ariyan Sep 01 '16 at 20:36
  • @pauljohn32, thank you for your comment and your clarifications! I did use your fix to update my answer. :) – Omid Ariyan Sep 01 '16 at 20:38
  • @PJSCopeland It'd be safer to use `git ls-files -z` and use \0. – rosuav Nov 20 '16 at 09:00
  • Thanks for the terrific code. BTW, I had to patch it for my laptop since stat -c does not work on cranky Mac OSX. – phs Jul 21 '17 at 10:14
  • As I am using this for a website, I realized that the directory permissions might want to be saved to, so I update the Code using the following snippet ... hm, okai first have to figure out how to share the Code here because it's too long (: – Tim Jun 21 '18 at 06:25
  • Okai, here is the Code including the directory permissions for the pre-commit hook https://pastebin.com/FMEicE2a – Tim Jun 21 '18 at 06:34
  • @tim Thank you for pointing that out Tim. I now have updated the answer making it possible to backup/restore directory permissions too and it's inspired by the code you shared. – Omid Ariyan Aug 31 '19 at 21:59
  • Apart from nice automation using git hooks, this solution also provides significant security advantage over `git-cache-meta`. It doesn't store metadata as an executable script in the repository. – Simon Rozman Nov 06 '22 at 14:17
  • There is a flaw with those scripts when it comes to symlinks: `pre-commit` saves symlink modes - always 777. `post-checkout` does chmod 777 on symlinks - however, chmod actually applies mode to target, not symlink itself. This propagates mode 777 to all symlink targets?! – Simon Rozman Nov 07 '22 at 09:22
6

We can improve on the other answers by changing the format of the .permissions file to be executable chmod statements, and to make use of the -printf parameter to find. Here is the simpler .git/hooks/pre-commit file:

#!/usr/bin/env bash

echo -n "Backing-up file permissions... "

cd "$(git rev-parse --show-toplevel)"

find . -printf 'chmod %m "%p"\n' > .permissions

git add .permissions

echo done.

...and here is the simplified .git/hooks/post-checkout file:

#!/usr/bin/env bash

echo -n "Restoring file permissions... "

cd "$(git rev-parse --show-toplevel)"

. .permissions

echo "done."

Remember that other tools might have already configured these scripts, so you may need to merge them together. For example, here's a post-checkout script that also includes the git-lfs commands:

#!/usr/bin/env bash

echo -n "Restoring file permissions... "

cd "$(git rev-parse --show-toplevel)"

. .permissions

echo "done."

command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on you
r path. If you no longer wish to use Git LFS, remove this hook by deleting .git/hooks/post-checkout.\n"; exit 2; }
git lfs post-checkout "$@"
Tammer Saleh
  • 393
  • 4
  • 7
  • I really like the idea of making the `pre-commit` hook generate a script to greatly simplify the `post-checkout` hook. However, I think this `pre-commit` hook is a *too* simple; it will include all files in the tree, not just the ones committed to the Git repository. In other words, it will include everything in `.git/`, files from `.gitignore`, temporary files not checked in, etc. – jamesdlin Apr 19 '21 at 08:43
2

In case you are coming into this right now, I've just been through it today and can summarize where this stands. If you did not try this yet, some details here might help.

I think @Omid Ariyan's approach is the best way. Add the pre-commit and post-checkout scripts. DON'T forget to name them exactly the way Omid does and DON'T forget to make them executable. If you forget either of those, they have no effect and you run "git commit" over and over wondering why nothing happens :) Also, if you cut and paste out of the web browser, be careful that the quotation marks and ticks are not altered.

If you run the pre-commit script once (by running a git commit), then the file .permissions will be created. You can add it to the repository and I think it is unnecessary to add it over and over at the end of the pre-commit script. But it does not hurt, I think (hope).

There are a few little issues about the directory name and the existence of spaces in the file names in Omid's scripts. The spaces were a problem here and I had some trouble with the IFS fix. For the record, this pre-commit script did work correctly for me:

#!/bin/bash  

SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions

# Clear the permissions database file
> $DATABASE

echo -n "Backing-up file permissions..."

IFSold=$IFS
IFS=$'\n'
for FILE  in `git ls-files`
do
   # Save the permissions of all the files in the index
   echo $FILE";"`stat -c "%a;%U;%G" $FILE` >> $DATABASE
done
IFS=${IFSold}
# Add the permissions database file to the index
git add $DATABASE

echo "OK"

Now, what do we get out of this?

The .permissions file is in the top level of the git repo. It has one line per file, here is the top of my example:

$ cat .permissions
.gitignore;660;pauljohn;pauljohn
05.WhatToReport/05.WhatToReport.doc;664;pauljohn;pauljohn
05.WhatToReport/05.WhatToReport.pdf;664;pauljohn;pauljohn

As you can see, we have

filepath;perms;owner;group

In the comments about this approach, one of the posters complains that it only works with same username, and that is technically true, but it is very easy to fix it. Note the post-checkout script has 2 action pieces,

# Set the file permissions
chmod $PERMISSIONS $FILE
# Set the file owner and groups
chown $USER:$GROUP $FILE

So I am only keeping the first one, that's all I need. My user name on the Web server is indeed different, but more importantly you can't run chown unless you are root. Can run "chgrp", however. It is plain enough how to put that to use.

In the first answer in this post, the one that is most widely accepted, the suggestion is so use git-cache-meta, a script that is doing the same work that the pre/post hook scripts here are doing (parsing output from git ls-files). These scripts are easier for me to understand, the git-cache-meta code is rather more elaborate. It is possible to keep git-cache-meta in the path and write pre-commit and post-checkout scripts that would use it.

Spaces in file names are a problem with both of Omid's scripts. In the post-checkout script, you'll know you have the spaces in file names if you see errors like this

$ git checkout -- upload.sh
Restoring file permissions...chmod: cannot access  '04.StartingValuesInLISREL/Open': No such file or directory
chmod: cannot access 'Notebook.onetoc2': No such file or directory
chown: cannot access '04.StartingValuesInLISREL/Open': No such file or directory
chown: cannot access 'Notebook.onetoc2': No such file or directory

I'm checking on solutions for that. Here's something that seems to work, but I've only tested in one case

#!/bin/bash

SELF_DIR=`git rev-parse --show-toplevel`
DATABASE=$SELF_DIR/.permissions

echo -n "Restoring file permissions..."
IFSold=${IFS}
IFS=$
while read -r LINE || [[ -n "$LINE" ]];
do
   FILE=`echo $LINE | cut -d ";" -f 1`
   PERMISSIONS=`echo $LINE | cut -d ";" -f 2`
   USER=`echo $LINE | cut -d ";" -f 3`
   GROUP=`echo $LINE | cut -d ";" -f 4`

   # Set the file permissions
   chmod $PERMISSIONS $FILE
   # Set the file owner and groups
   chown $USER:$GROUP $FILE
done < $DATABASE
IFS=${IFSold}
echo "OK"

exit 0

Since the permissions information is one line at a time, I set IFS to $, so only line breaks are seen as new things.

I read that it is VERY IMPORTANT to set the IFS environment variable back the way it was! You can see why a shell session might go badly if you leave $ as the only separator.

pauljohn32
  • 2,079
  • 21
  • 28
1

In pre-commit/post-checkout an option would be to use "mtree" (FreeBSD), or "fmtree" (Ubuntu) utility which "compares a file hierarchy against a specification, creates a specification for a file hierarchy, or modifies a specification."

The default set are flags, gid, link, mode, nlink, size, time, type, and uid. This can be fitted to the specific purpose with -k switch.

Vladimir Botka
  • 58,131
  • 4
  • 32
  • 63
1

I am running on FreeBSD 11.1, the freebsd jail virtualization concept makes the operating system optimal. The current version of Git I am using is 2.15.1, I also prefer to run everything on shell scripts. With that in mind I modified the suggestions above as followed:

git push: .git/hooks/pre-commit

#! /bin/sh -
#
# A hook script called by "git commit" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message if it wants
# to stop the commit.

SELF_DIR=$(git rev-parse --show-toplevel);
DATABASE=$SELF_DIR/.permissions;

# Clear the permissions database file
> $DATABASE;

printf "Backing-up file permissions...\n";

OLDIFS=$IFS;
IFS=$'\n';
for FILE in $(git ls-files);
do
   # Save the permissions of all the files in the index
    printf "%s;%s\n" $FILE $(stat -f "%Lp;%u;%g" $FILE) >> $DATABASE;
done
IFS=$OLDIFS;

# Add the permissions database file to the index
git add $DATABASE;

printf "OK\n";

git pull: .git/hooks/post-merge

#! /bin/sh -

SELF_DIR=$(git rev-parse --show-toplevel);
DATABASE=$SELF_DIR/.permissions;

printf "Restoring file permissions...\n";

OLDIFS=$IFS;
IFS=$'\n';
while read -r LINE || [ -n "$LINE" ];
do
   FILE=$(printf "%s" $LINE | cut -d ";" -f 1);
   PERMISSIONS=$(printf "%s" $LINE | cut -d ";" -f 2);
   USER=$(printf "%s" $LINE | cut -d ";" -f 3);
   GROUP=$(printf "%s" $LINE | cut -d ";" -f 4);

   # Set the file permissions
   chmod $PERMISSIONS $FILE;

   # Set the file owner and groups
   chown $USER:$GROUP $FILE;

done < $DATABASE
IFS=$OLDIFS

pritnf "OK\n";

exit 0;

If for some reason you need to recreate the script the .permissions file output should have the following format:

.gitignore;644;0;0

For a .gitignore file with 644 permissions given to root:wheel

Notice I had to make a few changes to the stat options.

Enjoy,

1

One addition to @Omid Ariyan's answer is permissions on directories. Add this after the for loop's done in his pre-commit script.

for DIR in $(find ./ -mindepth 1 -type d -not -path "./.git" -not -path "./.git/*" | sed 's@^\./@@')
do
    # Save the permissions of all the files in the index
    echo $DIR";"`stat -c "%a;%U;%G" $DIR` >> $DATABASE
done

This will save directory permissions as well.

0

Another option is git-store-meta. As the author described in this superuser answer:

git-store-meta is a perl script which integrates the nice features of git-cache-meta, metastore, setgitperms, and mtimestore.

ishigoya
  • 161
  • 3
  • 6
0

Improved version of https://stackoverflow.com/users/9932792/tammer-saleh answer:

  1. It only updates the permissions on changed files.
  2. It handles symlinks
  3. It ignores empty directories (git can not handle them)

.git/hooks/pre-commit:

#!/usr/bin/env bash

echo -n "Backing-up file permissions... "
cd "$(git rev-parse --show-toplevel)"
find . -type d ! -empty -printf 'X="%p"; chmod %m "$X"; chown %U:%G "$X"\n' > .permissions
find . -type f -printf 'X="%p"; chmod %m "$X"; chown %U:%G "$X"\n' >> .permissions
find . -type l -printf 'chown -h %U:%G "%p"\n' >> .permissions
git add .permissions
echo done.

.git/hooks/post-merge:

#!/usr/bin/env bash

echo -n "Restoring file permissions... "
cd "$(git rev-parse --show-toplevel)"
git diff -U0 .permissions | grep '^\+' | grep -Ev '^\+\+\+' | cut -c 2- | /usr/bin/bash
echo "done."
jsaak
  • 587
  • 4
  • 17