3

When I run

proc = subprocess.Popen(['git', 'add', '-A'], stdout=subprocess.PIPE)

I get this error

fatal: not a git repository (or any parent up to mount point /media)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

But running

os.system('git add -A')

Does the job perfectly.

If you think the folder does not have a .git file,

proc = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE)

shows that it is already in the cwd.

Why Popen cannot stage the files, nor can it do the commit while os.system does both?

Update:

Here is my failed MWE

import subprocess
import os

cwd = os.getcwd()
proj_path = os.path.join(cwd, 'newproj')
os.makedirs(proj_path)
os.chdir(proj_path)
proc = subprocess.Popen(['git', 'init'], stdout=subprocess.PIPE)
proc = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE)
print(proc.stdout.read().decode('ascii'))
proc = subprocess.Popen(['git', 'add', '-A'], stdout=subprocess.PIPE)
out, err = proc.communicate()
if err:
    print('Error:\n', err.decode())
print(out.decode('ascii'))

output

.
..
.git

fatal: not a git repository (or any parent up to mount point /media)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
mercury
  • 189
  • 1
  • 15
  • does this help? https://stackoverflow.com/questions/31766655/git-add-through-python-subprocess – Lesmana Feb 10 '21 at 22:57
  • @lesmana, if the cwd is not set, then how come `ls -a` works on the cwd? – mercury Feb 10 '21 at 23:22
  • That is ... odd. Do you have a [mcve]? – torek Feb 10 '21 at 23:38
  • @torek, Added MWE. – mercury Feb 11 '21 at 01:10
  • i was not sure how it would help. the other question was similar in that git add did not work. it was a shot in the dark. i was not able to reproduce your problem and was baffled. this was before you added the minimal working example. now it is clear that it is a race condition. you have to communicate or wait for each subprocess. – Lesmana Feb 11 '21 at 08:57
  • @lesmana, no the questions are not the same. in the other question the problem was robustness of `git filter-branch` under network directories. – mercury Feb 11 '21 at 09:07
  • I was facing a similar problem - the same error message. It was due to attempts to run git commands without being in the git folder (I was in the parent one). `cd repodir` was the magic. – Karel Macek Nov 02 '21 at 21:37

1 Answers1

3

My Python version is a little behind yours, but I was able to reproduce the problem, which is actually pretty simple:

proc = subprocess.Popen(['git', 'init'], stdout=subprocess.PIPE)
proc = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE)
print(proc.stdout.read().decode('ascii'))
proc = subprocess.Popen(['git', 'add', '-A'], stdout=subprocess.PIPE)

Note the lack of any calls to proc.wait() or similar. This means we spin off a git init and do not wait for it.

Next, we run ls -a. Here we do wait a little bit—long enough to read its output to EOF, which is actually relatively long as EOF only occurs because ls -a finishes—and meanwhile the git init is still running. Depending on how fast git init is doing its work, we may or may not find the .git directory here. If this takes long enough, the git init may also finish. Fortunately the git init on my system is sufficiently slower than the ls -a that I see the same effect as you. (Well, I found that the .git directory sometimes doesn't exist yet.)

Last, we run git add -A. The git init may or may not still be running. As it turns out, it is still running, and has not yet gotten far enough to establish the .git directory as a Git repository. So git add -A complains, with the error you observed.

If we add proc.wait() right after the git init—ideally, we should check the return code, or simply use subprocess.check_call or subprocess.run here—the problem goes away.

torek
  • 448,244
  • 59
  • 642
  • 775