Python distutils error: "[directory]... doesn't exist or not a regular file"

Question

Let's take the following project layout:

$ ls -R .
.:
package  setup.py

./package:
__init__.py  dir  file.dat  module.py

./package/dir:
tool1.dat  tool2.dat

And the following content for setup.py:

$ cat setup.py 
from distutils.core import setup


setup(name='pyproj',
      version='0.1',

      packages=[
          'package',
      ],
      package_data={
          'package': [
              '*',
              'dir/*',
          ],
      },
     )

As you can see, I want to include all non-Python files in package/ and package/dir/ directories. However, running setup.py install would raise the following error:

$ python setup.py install
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/package
copying package/module.py -> build/lib/package
copying package/__init__.py -> build/lib/package
error: can't copy 'package/dir': doesn't exist or not a regular file

What gives?

A much simpler solution is given in this answer: http://stackoverflow.com/a/25375689/74632 — Boris Chervenkov, Sep 09 '15 at 15:20
I second @BorisChervenkov comment. Combination of include_package_data and changes to MANIFEST.in is far simpler and cleaner solution. — SKG, Apr 16 '21 at 16:48

score 21 · Accepted Answer · answered Sep 14 '10 at 20:34

21

In your package_data, your '*' glob will match package/dir itself, and try to copy that dir as a file, resulting in a failure. Find a glob that won't match the directory package/dir, rewriting your setup.py along these lines:

from distutils.core import setup

setup(name='pyproj',
      version='0.1',

      packages=[
          'package',
      ],
      package_data={
          'package': [
              '*.dat',
              'dir/*'
          ],
      },
     )

Given your example, that's just changing '*' to '*.dat', although you'd probably need to refine your glob more than that, just ensure it won't match 'dir'

answered Sep 14 '10 at 20:34

Jacob Oscarson

6,363
1
36
46

6

Ah. Thanks. Do you know if there's a more elegant way in specifying "include everything, recursively, in this package (on top of the modules)"? – Santa Sep 14 '10 at 21:43
2

You might get away by using a MANIFEST.in file, but it's unfortunately more difficult than ideal (as this search http://www.google.se/search?q=distutils+package_data+include+directory shows, a lot of people are asking this question, and not many get answers). The most comprehensive doc is this: http://docs.python.org/distutils/setupscript.html , but it's not very clearly written (to find your problem, I actually pdb'd into distutils source) – Jacob Oscarson Sep 15 '10 at 08:59
1

While this appears to be the solution, it's awful that distributils can't just handle sub directories out of the box! This works fine in Windows, yet fails in Linux. I have an extensive nesting of sub directories, and having to explicitly define that is going to be real pain and be more difficult to maintain. – BuvinJ Mar 15 '16 at 18:44
@BuvinJ yes, that doesn't sound nice at all. Since I made the answer I've been trying to use `MANIFEST.in` files more and more, as described e.g. [here](https://wiki.python.org/moin/Distutils/Tutorial). Can't answer much about diffs between Windows/Linux for that, though. – Jacob Oscarson Mar 16 '16 at 09:22
As a partial solution, added logic to switch between Windows and Linux package_data: `import platform if platform.system() == 'Windows' : packageData= ...` – BuvinJ Mar 16 '16 at 12:45
I don't have this issue on new versions of setuptools. But it happens on Launchpad's old build servers (Ubuntu 10.04 AFAIK), which is super confusing. Thanks for the explanation! – benjaoming Apr 26 '19 at 16:29
The later setuptools versions and the one in your Launchpad config probably deviates. – Jacob Oscarson May 02 '19 at 09:37

score 4 · Answer 2 · answered Jul 05 '11 at 07:48

You could use Distribute instead of distutils. It works basically the same (for the most part, you will not have to change your setup.py) and it gives you the exclude_package_data option:

from distribute_setup import use_setuptools
use_setuptools()

from setuptools import setup

setup(name='pyproj',
      version='0.1',

      packages=[
          'package',
      ],
      package_data={
          'package': [
              '*.dat',
              'dir/*'
          ],
      },
      exclude_package_data={
          'package': [
              'dir'
          ],
      },
     )

score 0 · Answer 3 · answered Dec 08 '16 at 04:03

I created a function that gives me all the files that I need

def find_files(directory, strip):
  """
  Using glob patterns in ``package_data`` that matches a directory can
  result in setuptools trying to install that directory as a file and
  the installation to fail.

  This function walks over the contents of *directory* and returns a list
  of only filenames found. The filenames will be stripped of the *strip*
  directory part.
  """

  result = []
  for root, dirs, files in os.walk(directory):
    for filename in files:
      filename = os.path.join(root, filename)
      result.append(os.path.relpath(filename, strip))
  return result

And used that as arugment for package_data

score 0 · Answer 4 · answered Feb 26 '17 at 19:44

0

Not quite sure why, but after some troubleshooting I realised that renaming the directories that had dots in their names solved the problem. E.g.

chart.js-2.4.0 => chart_js-2_4_0

Note: I'm using Python 2.7.10, SetupTools 12.2

answered Feb 26 '17 at 19:44

magicrebirth

4,104
2
25
22

Python distutils error: "[directory]... doesn't exist or not a regular file"

4 Answers4

Linked