4

I am attempting to publish a package on PyPI (as a newbie) which contains data stored as a pickle file. But I don't seem to be able to read the data in my main code.

My package structure is:

package/
    setup.py
    MANIFEST.in
    package/
        package.py
        __init__.py
        data.pkl

And my code in package.py is somewhat like this:

data = pickle.load(open('data.pkl', 'rb'))

def doSomething(data):
    ***code to do something on data***
    return variable

var = doSomething(data)

When I install the package and try to import it, I get the error:

FileNotFoundError: [Errno 2] No such file or directory: 'data.pkl'

It appears my package.py can't find my pickle file. Yes, I set include_package_data = True in setup.py and the the pickle file is loaded via the MANIFEST.in file.

I have tried loading the pickle file in __init__.py or specifying the full path (package/data.pkl), but none of these work.

phd
  • 82,685
  • 13
  • 120
  • 165
ProteinGuy
  • 1,754
  • 2
  • 17
  • 33
  • Have you tried a relative path like "./data.pkl"? – Angelo Jul 19 '19 at 07:18
  • Is the pickle in the package? (Every package is just a `.zip` or `.tar.gz` archive that you can list). Is it installed after the package installation? – phd Jul 19 '19 at 07:20
  • Specifying a relative path (./data.pkl) yields the same error. The pickle is in the package after installation. – ProteinGuy Jul 19 '19 at 15:10

1 Answers1

1

The path of data.pkl relative to the source code must be specified. And this can be done using the __file__ variable (as explained here).

The following code solved it for me

this_dir, this_filename = os.path.split(__file__)  # Get path of data.pkl
data_path = os.path.join(this_dir, 'data.pkl')
data = pickle.load(open(data_path, 'rb'))
ProteinGuy
  • 1,754
  • 2
  • 17
  • 33