Windows operating system unable to open file names that works on linux

Question

I'm developing a small Python application on Linux, where the code runs fine on both system Python3 and Anaconda Python3 interpreters. But, when I run it on Anaconda/Windows, I get this backtrace:

Traceback (most recent call last):
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\tkinter\__init__.py"
, line 1699, in __call__
    return self.func(*args)
  File "./dataView.py", line 312, in doChop
    self.DA.chop()
  File "C:\Users\<redacted>\DataAnalyser.py", line 212, in chop
    self.df.to_hdf( filename, mode='w', key=hdfKey, data_columns = view )
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas
\core\generic.py", line 1471, in to_hdf
    return pytables.to_hdf(path_or_buf, key, self, **kwargs)
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas
\io\pytables.py", line 280, in to_hdf
    complib=complib) as store:
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas
\io\pytables.py", line 467, in __init__
    self.open(mode=mode, **kwargs)
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas
\io\pytables.py", line 580, in open
    self._handle = tables.open_file(self._path, self._mode, **kwargs)
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\site-packages\tables
\file.py", line 320, in open_file
    return File(filename, mode, title, root_uep, filters, **kwargs)
  File "C:\Users\<redacted>\AppData\Local\Continuum\anaconda3\lib\site-packages\tables
\file.py", line 784, in __init__
    self._g_new(filename, mode, **params)
  File "tables\hdf5extension.pyx", line 487, in tables.hdf5extension.File._g_new

tables.exceptions.HDF5ExtError: HDF5 error back trace

  File "C:\Users\builder\mc3\conda-bld\hdf5_1506030377716\work\src\H5F.c", line
491, in H5Fcreate
    unable to create file
  File "C:\Users\builder\mc3\conda-bld\hdf5_1506030377716\work\src\H5Fint.c", li
ne 1247, in H5F_open
    unable to open file: time = Wed Feb 28 18:26:31 2018
, name = 'chop_x:0:49.hdf5', tent_flags = 13
  File "C:\Users\builder\mc3\conda-bld\hdf5_1506030377716\work\src\H5FD.c", line
 809, in H5FD_open
    open failed
  File "C:\Users\builder\mc3\conda-bld\hdf5_1506030377716\work\src\H5FDsec2.c",
line 346, in H5FD_sec2_open
    unable to open file: name = 'chop_x:0:49.hdf5', errno = 22, error message =
'Invalid argument', flags = 13, o_flags = 302

End of HDF5 error back trace

Unable to open/create file 'chop_x:0:49.hdf5'

I tried writing to a file of the exact same name, manually, in Anaconda/Win ipython3, which also worked fine. I also tried changing the name to just a simple string, which worked, suggesting that there really is something wrong with the file name value I'm passing as constructed with str.format(). On the other hand, when I do df.to_hdf('chop_x:0:49.hdf5', 'w', 'chop') it works fine. How can I see what is wrong with the parameters I'm passing? Do you think it's a problem with another parameter?

Possible duplicate of [What characters are forbidden in Windows and Linux directory names?](https://stackoverflow.com/questions/1976007/what-characters-are-forbidden-in-windows-and-linux-directory-names) — Patrick Artner, Mar 04 '18 at 11:20
You do see the `unable to create file .... unable to open file: time = Wed Feb 28 18:26:31 2018 , name = 'chop_x:0:49.hdf5' ..... open failed (bexause it could not be created) ` - do you? — Patrick Artner, Mar 05 '18 at 19:54
Oops, I'm wrong. I didn't test correctly. Patrick is correct. Thanks! — Porcelain Mouse, Mar 05 '18 at 20:01

score 3 · Accepted Answer · answered Mar 04 '18 at 11:14

3

Different operating systems have different restrictions regarding allowed characters inside paths and filenames.

Try to create a filename with a : in windows explorer and you see why it is not working.

Naming Files, Paths, and Namespaces

Abstain from using:

< (less than)
> (greater than) 
: (colon)
" (double quote)
/ (forward slash)
\ (backslash)
| (vertical bar or pipe)
? (question mark)
* (asterisk)
Integer value zero, sometimes referred to as the ASCII NUL character. Characters whose integer representations are in the range from 1 through 31, except for alternate data streams where these characters are allowed.

(Excerpt of things from above link, there is more in there, read it.)

answered Mar 04 '18 at 11:14

Patrick Artner

50,409
9
43
69

Oh, good idea, but I think I can exclude this possibility. In iPython3, I tried to do it manually, with `df.to_hdf('chop_x:0:49.hdf5', 'w', 'chop')` and that did work. So, the string seems fine by all other measures. – Porcelain Mouse Mar 05 '18 at 19:33
@PorcelainMouse On windows you cant have files that are named like this. Its forbidden by the operating system. On linux it might work. As soon as the dataframe is persisted to the filesystem the OS throws you a curveball (exception) and it wont work. – Patrick Artner Mar 05 '18 at 19:41
Ooops, my mistake. You are right. I mis-remembered my test. – Porcelain Mouse Mar 05 '18 at 19:59

Windows operating system unable to open file names that works on linux

1 Answers1