I'm doing pip install ., where the current working directory is on a panfs filesystem. When pip tries to copy the directory tree, including the files in .git, it fails due to what is ultimately a failure in os.setxattr:
$ pip install .
Processing /home/users/gholl/checkouts/pyatmlab
Exception:
Traceback (most recent call last):
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/commands/install.py", line 280, in run
requirement_set.prepare_files(finder)
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/req/req_set.py", line 317, in prepare_files
functools.partial(self._prepare_file, finder))
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/req/req_set.py", line 304, in _walk_req_to_install
more_reqs = handler(req_to_install)
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/req/req_set.py", line 469, in _prepare_file
session=self.session)
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/download.py", line 814, in unpack_url
unpack_file_url(link, location, download_dir)
File "/home/users/gholl/venv/stable-3.4/lib/python3.4/site-packages/pip/download.py", line 699, in unpack_file_url
shutil.copytree(link_path, location, symlinks=True)
File "/home/users/gholl/lib/python3.4/shutil.py", line 343, in copytree
raise Error(errors)
shutil.Error: [('/home/users/gholl/checkouts/pyatmlab/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack', '/tmp/pip-6lrnz9vs-build/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack', "[Errno 13] Permission denied: '/tmp/pip-6lrnz9vs-build/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack'"), ('/home/users/gholl/checkouts/pyatmlab/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.idx', '/tmp/pip-6lrnz9vs-build/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.idx', "[Errno 13] Permission denied: '/tmp/pip-6lrnz9vs-build/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.idx'")]
Some deeper digging shows the error originates in:
Traceback (most recent call last):
File "/home/users/gholl/lib/python3.4/shutil.py", line 329, in copytree
copy_function(srcname, dstname)
File "/home/users/gholl/lib/python3.4/shutil.py", line 246, in copy2
copystat(src, dst, follow_symlinks=follow_symlinks)
File "/home/users/gholl/lib/python3.4/shutil.py", line 213, in copystat
_copyxattr(src, dst, follow_symlinks=follow)
File "/home/users/gholl/lib/python3.4/shutil.py", line 153, in _copyxattr
os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/tmp/pip-y3dw8fwa-build/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack' File "/home/users/gholl/lib/python3.4/shutil.py", line 329, in copytree
...and can be reproduced by:
In [55]: shutil.copystat("/home/users/gholl/checkouts/pyatmlab/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack", "/tmp/fubar")
---------------------------------------------------------------------------
PermissionError Traceback (most recent call last)
<ipython-input-55-139c9fc77184> in <module>()
----> 1 shutil.copystat("/home/users/gholl/checkouts/pyatmlab/.git/objects/pack/pack-c1449559ec4287b3830efe2913608dddf2f21391.pack", "/tmp/fubar")
/home/users/gholl/lib/python3.4/shutil.py in copystat(src, dst, follow_symlinks)
211 else:
212 raise
--> 213 _copyxattr(src, dst, follow_symlinks=follow)
214
215 def copy(src, dst, *, follow_symlinks=True):
/home/users/gholl/lib/python3.4/shutil.py in _copyxattr(src, dst, follow_symlinks)
151 try:
152 value = os.getxattr(src, name, follow_symlinks=follow_symlinks)
--> 153 os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
154 except OSError as e:
155 if e.errno not in (errno.EPERM, errno.ENOTSUP, errno.ENODATA):
PermissionError: [Errno 13] Permission denied: '/tmp/fubar'
This occurs for any source file where the user write bit is not set.
Now, who should handle this? Is this a bug in pip, shutil.copytree, shutil.copystat, setxattr, panfs, or none of the above?
Reported as issue 24538. See also this question on Unix SE.
