9

In R, I was using write.table() to write a file into a location embedded in directories with long names. But it errors out as below:

Error in file(file, ifelse(append, "a", "w")) : cannot open the connection In addition: Warning message: In file(file, ifelse(append, "a", "w")) : cannot open file 'data/production/Weekly_Prod_201407_Selling_Price_Snapshot_20140930_Median_Selling_Price_Map.csv': No such file or directory

Then when I shortened the filename to Weekly_Prod.csv, it worked! So it seems the long path and the long filename caused R to error out.

I tested it a few times and found that the limit is 260 characters for the total length of path+filename. That is, R errors out when it's 261 characters or more. Is there a way to get around of this? Please help. Thanks!

jdharrison
  • 30,085
  • 4
  • 77
  • 89
smz
  • 263
  • 4
  • 11
  • What OS? What R version? Share the results of `sessionInfo()`. And do those directories already exist? – MrFlick Oct 01 '14 at 22:01
  • @RichardScriven That's in the source for `write.table`. – MrFlick Oct 01 '14 at 22:03
  • Windows 7 64bit, R x64 3.1.1, R Studio 0.98.1028. – smz Oct 01 '14 at 22:11
  • sessionInfo(): > sessionInfo() R version 3.1.1 (2014-07-10) Platform: x86_64-w64-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] tcltk stats graphics grDevices utils datasets methods base other attached packages: [1] sqldf_0.4-7.1 RSQLite.extfuns_0.0.1 RSQLite_0.11.4 DBI_0.2-7 gsubfn_0.6-5 proto_0.3-10 ... – smz Oct 01 '14 at 22:12
  • Yes. Those directories already exist on a server. It is a location shared by the team, so it has a lot of sub directories. – smz Oct 01 '14 at 22:12
  • file(file, ifelse(append, "a", "w")) is not part of my code. It is part of the error message. My code is just a simple write.table(data2,file='data/production/Weekly_Prod_201407_Selling_Price_Snapshot_20140930_Median_Selling_Price_Map.csv', row.names=F,sep=',',quote=F) – smz Oct 01 '14 at 22:14

3 Answers3

10

There is a limit on file path length on windows:

> write(1, paste0(paste(sample(letters, 150, TRUE), collapse = ''), '.txt'))
> write(1, paste0(paste(sample(letters, 250, TRUE), collapse = ''), '.txt'))
Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file 'qvxirpnlwkqfwlxhggkscxlwhhyblrwxfpikpsukrfqwhaqvsyhdpihnoknqmxgafvawxkuijqbmvgdjwwgeumfksmhtiqwvzwmjukmmmeesvcdpdbpimarxssnrngfxwjksqshjruralhtwdnfmdhzrcwcdrnwezdhwqyisbjikdhbbygtcoeechgwrewenewbrlexliiikdnwlclbzllaxcohacadxzztgmtnmppyxtxtbopxdokjnvx.txt': No such file or directory

According to this source it is 260 characters

http://msdn.microsoft.com/en-us/library/aa365247.aspx#maxpath

> nchar(getwd())
[1] 23
> write(1, paste0(paste(sample(letters, 231, TRUE), collapse = ''), '.txt'))
> write(1, paste0(paste(sample(letters, 232, TRUE), collapse = ''), '.txt'))
Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file 'topylmudgfnrkdilqbklylwtbwrgwbwmamxzhwwzlxxslqeuhpywahoxqxpkckvmkfjccbsqncctlovcnxctkyvgunnbqcwyiliwpfkjibanpmtupsxfboxnjaadovtdpxeloqjnbqgvkcilwljfswzlrlqixmwqpoemcemhdizwwwbgqruhepyrskiklkbylzjhrcchbusohkrwyzgablvngqrqiardubcbziex.txt': No such file or directory
> getwd()
[1] "C:/Users/john/Documents"

> nchar(file.path(getwd(), paste0(paste(sample(letters, 231, TRUE), collapse = ''), '.txt')))
[1] 259

One possible solution which may work for you is to create a virtual drive for your long directory path. It should give you a bit of leeway see https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/subst.mspx?mfr=true

> system("subst x: C:/Users/john/Documents")
> write(1, paste0("x://", paste(sample(letters, 251, TRUE), collapse = ''), '.txt'))

when you are done with the virtual drive you can reverse using:

system("subst x: /D")
jdharrison
  • 30,085
  • 4
  • 77
  • 89
  • This includes sub directories which OP may not have listed – jdharrison Oct 01 '14 at 22:24
  • I did not put the full path in my question, where I have already used setwd() to set the parent directory. But as I said, it errors out whenever the file path goes beyond 260. – smz Oct 01 '14 at 22:25
  • Ok. It is a server location shared by Windows and Mac users. So apparently Mac users can go beyond 260 characters to saved files there. I am still wondering if it's possible to get around this on Windows. It is already 64 bit. – smz Oct 01 '14 at 22:28
  • You could try making a virtual path to your server folder – jdharrison Oct 01 '14 at 23:04
  • Thanks. I will test it out according to this site http://ss64.com/nt/subst.html. According to it, for network path, it is 'net use' instead of 'subst'. – smz Oct 01 '14 at 23:13
1

This could be taken care of by replacing the name of the said file with its Short File Name (SFN), also known as the 8.3 file name.

Type dir /x in the command prompt over the directory where the file is located, which would list all the SFN's of the files in the directory.

Then replace the file-name in your code with its corresponding 8.3 file-name.

Rakesh R
  • 11
  • 3
0

This is not an error with R, but a limitation imposed by Windows. Since Windows 10, the 260 limit can be lifted (to 32,767 characters). According to this article:

In the Windows API... the maximum length for a path is MAX_PATH, which is defined as 260 characters.

Starting in Windows 10, version 1607, MAX_PATH limitations have been removed from common Win32 file and directory functions. However, you must opt-in to the new behavior.

The article gives instruction on how to opt-into longer path length limits. I just did it, restarted my computer, and now I don't get that error.

Lewkrr
  • 414
  • 4
  • 13