1

I know this seems like such an obvious question but I haven't been able to find the answer to it anywhere.

Whenever I'm reading in a CSV file (or any other type of file), I always need to list the entire path to find that on my computer even though the file is saved in my python project.

This is how I've had to call a CSV file called SF1 into a pandas df:

df = pd.read_csv(r'C:\Users\George Adamopoulos\Desktop\All My Files\Code\Neptune-Financial\datas\CompleteData\FULL_SF1.csv')

I know that in the past doing this has worked for me:

df = pd.read_csv('../datas/CompleteData/FULL_SF1.csv')

It's really annoying because everything is able to run fine on my computer, but when I push things to Github, it's a mess with everyone trying to change the path to their computers.

I'm using python 3.7

George Adams
  • 331
  • 4
  • 15
  • 1
    Please check this out: https://stackoverflow.com/questions/35384358/how-to-open-my-files-in-data-folder-with-pandas-using-relative-path - the latest answers might help you. – Christian Dec 31 '20 at 00:45
  • I tried that. ```df = pd.read_csv("../datas/CompleteData/FULL_SF1.csv")``` And I get this error ```FileNotFoundError: [Errno 2] File ../datas/CompleteData/FULL_SF1.csv does not exist: '../datas/CompleteData/FULL_SF1.csv' ``` – George Adams Dec 31 '20 at 00:48
  • 1
    I can't answer specifically because I don't know what your file structure is, but for a start, you can get the path that the script is in, with the `os` module, by calling `os.getcwd()`. – Countour-Integral Dec 31 '20 at 00:50
  • 1
    Can you show the directory structure of your *package*? Is `r'C:\Users\George Adamopoulos\Desktop\All My Files\Code'` the part that would/might be different on a different installation? And `r'\Neptune-Financial\datas\CompleteData\`'` *constant*? Do you have an `__init__.py` in your package's root folder? – wwii Dec 31 '20 at 01:29
  • 1
    Related: [How to read a (static) file from inside a Python package?](https://stackoverflow.com/questions/6028000/how-to-read-a-static-file-from-inside-a-python-package) – wwii Dec 31 '20 at 01:35
  • @wwii in this specific example I don't, but if I did have an ```__init__.py``` Would that change the way that I read in the CSV? – George Adams Dec 31 '20 at 02:16

3 Answers3

2

The path is relative to your current working directory. First, check if the relative path is correct -- maybe the file doesn't exist, because you want './datas/...' path instead of going level deeper?

Try using __file__

from pathlib import Path

path = Path(__file__).parent / "../datas/CompleteData/FULL_SF1.csv"
with path.open() as f:
    test = list(csv.reader(f))
LightFelicis
  • 129
  • 1
  • 5
1

From How to open my files in data_folder with pandas using relative path?

Try this:

import os
import pandas as pd
df = pd.read_csv(os.path.join(os.path.dirname(__file__), "../datas/CompleteData/FULL_SF1.csv"))

or this:

from pathlib import Path
path = Path(__file__).parent / "../datas/CompleteData/FULL_SF1.csv"
pd.read_csv(path)

file is a variable that contains the path to the module that is currently being imported. Python creates a file variable for itself when it is about to import a module. The updating and maintaining of this variable is the responsibility of the import system.

Christian
  • 4,902
  • 4
  • 24
  • 42
1

I ended up answer my own question and figured out what I think is probably the easiest (in terms of the least number of lines of code) to do it.

When defining a path, '../', takes you back one directory. So the number of directories that you need to go back to means that you need to have the same amount of '../', at the beginning of the path.

In my case, I needed to go back to three directories.

df = pd.read_csv("../../datas/CompleteData/FULL_SF1.csv")

Link to a similar answer: https://stackoverflow.com/a/43600253/14735826

George Adams
  • 331
  • 4
  • 15