Python Virtual Env OS PATH Issue

Question

I created a virtual env using venv (Python 3.xx).

There, I installed only a few packages:

I have one Python Script that reads the files in a given directory and manipulate their data using Pandas:

import pandas as pd
import os

path = os.getcwd()+'\Ph3_Charts'+'\\'

print(path)

from os import listdir
from os.path import isfile, join

days_range = [f for f in listdir(path) if isfile(join(path, f))]

dataframes = []
count = 0

for i in days_range:

    try:

        print(i,count)

        dataframes.append(pd.read_excel(path+i, sheet_name = "Issues Jira", index_col=0))

        count += 1

    except:
        pass

The problem seems to be with the variable path, as the program breaks when it tries to append some data frames from each of the listed files.

However, the red marked passage above shows the path just fine... The strangest thing is that when I run this program locally, the iteration works ok.

Any guesses why this is happening please?

Use raw strings when you want backslashes to be literal -- or double them up, to be escaped. — Charles Duffy, Jul 27 '21 at 22:21
That said, it's not obvious to me what you're describing as the program "breaking". Can you spell out a little more what it's doing that's wrong? Nothing about the output shown is obviously broken to me as a reader who doesn't know what you're _expecting_ the code to do instead. — Charles Duffy, Jul 27 '21 at 22:22
BTW -- it's always preferred to copy-and-paste text instead of providing screenshots; you _can_ copy text from a Windows cmd.exe window, it just takes a little doing. See also [Why not upload images of code/errors when asking a question?](https://meta.stackoverflow.com/a/285557/14122) — Charles Duffy, Jul 27 '21 at 22:23
(...also, is there any particular reason you believe this problem to be virtualenv-related, or to otherwise have anything to do with which libraries are or aren't installed?) — Charles Duffy, Jul 27 '21 at 22:25
@CharlesDuffy, yes, because when i run this exactly script in the regular environment, it works... — Nunes, Jul 28 '21 at 11:19

Mavaddat Javid · Accepted Answer · 2021-07-28T02:32:18.383

The source of the issue is that you're forcing the script to use the backslash \ as the path separator. Your remote system uses Linux whereas you're using Windows locally. Unlike Windows, Linux and macOS systems prefer to use the forward slash for separating directories in a system path. That's why the difference.

Below is a properly platform independent implementation that avoids such needless specificity:

import pandas as pd
import os

from os import listdir
from os.path import isfile, join

# Get CWD
CWD = os.getcwd()

# Join CWD with `Ph3_Charts` folder
PH3_CHART_PATH = os.path.join(CWD, 'Ph3_Charts') 

print("PH3 chart path: '"+PH3_CHART_PATH+"'")

days_range = [f for f in listdir(PH3_CHART_PATH) if isfile(join(PH3_CHART_PATH, f))]

dataframes = []
count = 0

for i in days_range:

    try:

        print(i,count)

        dataframes.append(pd.read_excel(PH3_CHART_PATH+i, sheet_name = "Issues Jira", index_col=0))

        count += 1

    except Exception as e: # catch *all* exceptions
        print(e) # print it out

This solution works with or without the venv features you discussed. Please see the file diff here (comparison of differences between your version and the above code).

thanks. it worked now... i just had to install one more package (openpyxl) — Nunes, Jul 28 '21 at 11:20

Python Virtual Env OS PATH Issue

1 Answers1