Setting package data as default input for command line argument in a PyPI python package

Question

I have written a command line script that requires a csv file included with my python package (layout of package below). I am using argparse on the command line script and am trying to make the default csv file the one that comes with the package. How would I assign my csv file included with the package as the default csv file in my command line command?

Command Line Argparse arguments:

parser = ArgumentParser()
parser.add_argument("-d", "--database", help="Database", **default="../data/database.csv"**) ## Help Here

args = parser.parse_args()

Package Hierarchy:

package
 - bin/
    - My_Commandline_Command
 - data/
    - database.csv
 - build/ ...
 - dist/  ...
 - MANIFEST.in
 - README.md
 - setup.py
 - package/
    - __init__.py

The Error:

When I install my package outside my development environment, I can see the data folder, however, my command line script is throwing an error because the default database path is not found.

What's the problem? Finding the file name? or finding it before setting up the parser? You could set the parser after parsing if that's easier. Just check `args.database is None`. — hpaulj, Oct 19 '20 at 06:41
Hi @hpaulj, When I run my command line command after pip installing my package, the path of the database doesn't match the default parameter. args.database is None because of a path error. — Cody Glickman, Oct 19 '20 at 07:02

score 1 · Answer 1 · answered Oct 19 '20 at 08:17

1

As @hpaulj suggested, you could set the default to None and then specifically check for this case:

import argparse
from pathlib import Path


parser = argparse.ArgumentParser()
parser.add_argument("-d", "--database", help="Database", default=None)

args = parser.parse_args()

if args.database is None:
    args.database = (Path(__file__).parent / "../data/database.csv").resolve()

Here we can use __file__ to get the relative location of database.csv

answered Oct 19 '20 at 08:17

Alex

6,610
3
20
38

Hi @Alex, Thank you for the answer. I have a conda distribution and the ```Path(__file__).parent``` leads to the bin files in my conda environment, there is no data file here or upstream. I found the __init__.py file in the lib of the conda distro. I can point the database there, although I am going to have to move the file. I think I am on th right track though – Cody Glickman Oct 19 '20 at 15:25
1

How are you installing this package? Your bin would usually be calling the python files from within your environment, which would be in site packages (either global or venv). If you've specified your `database.csv` in your manifest it should be installed with the package and accessible relative to the code that you are running. If that makes sense? – Alex Oct 19 '20 at 15:29
I think an issue is that the database is not being included in the MANIFEST.in file. I currently have within the MANIFEST.in file: ```include data/database.csv``` Am I doing that right? Or does the data file need to be in the same directory as the __index__.py file? After looking through the bin (just the My_Commandline_Command here) and site-packages folder (only the __index__.py file), I think the data is not being built, I will check my setup.py file. Any thoughts? – Cody Glickman Oct 19 '20 at 15:59
I forgot to include: ```include_package_data=True``` in the setup. I will come back in a bit, to let you know if this works. – Cody Glickman Oct 19 '20 at 16:05
I am struggling to connect this post: https://stackoverflow.com/questions/6028000/how-to-read-a-static-file-from-inside-a-python-package/58941536#58941536 with the ability to access a static file from a command line script – Cody Glickman Oct 19 '20 at 17:15

Cody Glickman · Accepted Answer · 2020-10-21T02:01:57.763

After reviewing How to read a (static) file from inside a Python package?: I moved the data folder into the package folder with the ___init__.py and was able to access the database using importlib_resources. Don't forget to edit the MANIFEST.in paths.

My_Commandline_Command:

import importlib_resources

my_resources = importlib_resources.files("package")
print(my_resources)
data = (my_resources / "data" / "database").read_bytes()

Changes to package structure:

package
 - bin/
    - My_Commandline_Command
 - build/ ...
 - dist/  ...
 - MANIFEST.in
 - README.md
 - setup.py
 - package/
    - __init__.py
    - data/
      - database.csv

Setting package data as default input for command line argument in a PyPI python package

2 Answers2