How to read file paths with escape characters in pandas read_csv while accepting user input?

Question

I am trying to accept a path from a user, and then load the spreadsheet at that path:

df_path = input('What is the file path?')
df = pd.read_csv(df_path, index_col=2)

Some users, when pasting their paths in, are told it is invalid, because of escape sequences. Here, \166 is being converted to v:

df_path = input('What is the file path?')

C:\Data\166 - data\data.csv #entered by user

df = pd.read_csv(df_path, index_col=2)

FileNotFoundError: [Errno 2] No such file or directory: "C:\\Datav - data\\data.csv"

I'm aware that manually loading the path within the code, this can be accounted for:

dataset = pd.read_csv(r"C:\Data\166 - data\data.csv", index_col=2)

However, I can't find a way to make this work while accepting user input and storing it as a variable (tried many ways of attempting to do so, one example here):

df_path = input('What is the file path?')

"C:\Data\166 - data\data.csv" #entered by user

df = pd.read_csv("r'" + df_path, index_col=2)

OSError: [Errno 22] Invalid argument: "r'C:\\Datav - data\\data.csv"

It also doesn't work to try to rewrite the path, replacing \ with \\:

df_path = df_path.replace("\", "\\")

SyntaxError: unexpected character after line continuation character

-

df_path = df_path.replace(r"\", r"\\")

SyntaxError: unexpected character after line continuation character

How can this be done?

Answer 1

You're turning \166 into v, you'll need to replace with \\166

The reason it works with r'\166' is because you are telling the interpreter to take the string literally. Where the backslash is literally a backslash.

In your work around you've appended a string "r'" to the front of another string, which is why it doesn't find the directory "r'C:/".

In short, "r'string'" is not the same as r'string' where the r is outside the string quotations.

Here is a break down on raw strings.

If you add the r to the front of the string in your 2nd code block you should get a working solution, like so:

df_path = r"C:\Data\166 - data\data.csv"

Or writing the string with two backslashes will do the same:

df_path = "C:\\Data\\166 - data\\data.csv"

Answer 2

This is now clearer. Python 3.5 is the version, and the code:

df_path = input('What is the file path?')

It is clear why testing with a string when wrong. Setting df_path from a literal will give an error, and you need a raw_string to avoid that error.

However data from input should be the same as a raw_string already. No conversion is needed. Testing this with a simple program:

a= "C:\Data\166 - data\data.csv"  # the \ characters here will be mapped
b= r"C:\Data\166 - data\data.csv"  # the r means \ will not be special
c= input("? ")  # data from input is already treated like the r"\" above
print(a==c,b==c)
print(c)

With python 3.5 Produces:

? C:\Data\166 - data\data.csv
False True
C:\Data\166 - data\data.csv
>>> 

In idle, or in windows 'cmd', in powershell or in bash shell(although the example is clearly for windows) the result is as above with the input being equal to the r"" string. This indicates the normal behaviour of python is to treat input exactly as a raw string, equivalent to using r'' in code.

So it could be that from your test you though you needed to emulate the raw_string with the r"", and this is the only error.

If you are saying the people get the 'v' when typing the string, this would be something different.

Are you saying that users are entering C:\Data\166 - data\data.csv as resulting in a df_path which prints as 'C:\\Datav - data\\data.csv' instead of the correct data which would print as 'C:\\Data\\166 - data\\data.csv'

Or only that you were trying to prevent this from happening, because you assumed this would happen?

If it is actually happening, the problem is not python itself. Either the actual console environment or terminal the program is being run from is doing its own translation of \166 into the escape character, or some other processing in the application is modifying the string. If the program works with the df_path = r'C:\Data\166 - data\data.csv' in place of input, then it can only be the environment where the string is being entered, or the input itself has been changed from the standard python input.

But I suspect you found an error when trying to replace the input with setting from a string, and tried to code around a problem that would never actually exist with the input, just with your test data.

Note: As explained by @GiantsLoveDeathMetal, A python 'rawstring' with an r'' before the quotes, changes the reading of the program file itself by python. You cannot make a string with an 'r' within the string to reproduce how this works. The r is outside the quotes and is telling python how to read the original string, and this is all done before the program executes. The same rule also applies to f'' strings in python 3.6.

Answer 3

I am having a little difficulty replicating the issue. Perhaps add in an extra line to see if it helps?

df_path = "C:\Data\166 - data\data.csv"
file_path = str(df_path)
df = pd.read_csv(file_path, index_col=2)