Using Resources Module to Import Data Files

Question

Background

I have a function called get_player_call_logic_df which basically reads a csv file from the PLAYER_TEST_INPUT path. I have a module called player_test and inside that i have another folder called player_test_input where i store all the csv files that are used for processing.

Code

PLAYER_TEST_INPUT_DIR = Path("../dev/playerassignment/src/player/player/player_test/player_test_input/")
def get_player_call_logic_df() -> pd.DataFrame:
    df = pd.read_csv(
       PLAYER_TEST_INPUT_DIR / "player_call_logic.csv"
    )
    return df

Issue

I created a PR and I got a very good suggestion that I look at the importlib.resources module. You can store the data as a "resource" in the library. Then, instead of referring to the data via filepath, you can import it similar to any other player module.

I am unsure how i would use resources module here. I read up on the doc and here is what i could come up with. I can probably do something like this.

from importlib import resources
def get_player_call_logic_df() -> pd.DataFrame:
    with resources.path("player.player_test_input", "player_call_logic.csv") as df:
        return df

I feel like i am doing the same thing so i am not sure how to use the resources module correctly. Any help would be appreciated as i am new to python.

This is basically doing the same thing, except the latter always works! You used a relative path in the first code and that is relative to the current working directory, which could be anything. OTOH, `importlib` know exactly where that package is, so it does not rely on CWD being anything specific. — zvone, Dec 21 '20 at 17:25

score 1 · Answer 1 · answered Apr 16 '21 at 09:17

Please use

from importlib import resources
import pandas as pd

def get_player_call_logic_df() -> pd.DataFrame::
    with resources.path("player.player_test_input", "player_call_logic.csv") as df:
        return pd.read_csv(df)

and bear in mind the __init__.py file inside the player_test_input folder:

.
└── player
    ├── __init__.py
    └── player_test_input
        ├── __init__.py
        └── player_call_logic.csv

Very good reference and alternatives can be found here and here

Using Resources Module to Import Data Files

1 Answers1