string concatenation with python 33 / isdir() always returns True? (3 hours head bumping)

Question

[ ok I edit my question since I just got answers corresponding to the first lines of my previous post, and actually this was not the question... so if you post smtg please test before and read the whole thing before thinking I need to know how to test paths in python ;) ]

the whole script, which is detailed below : http://www.pasteall.org/38511/python

the script does not return the same on windows 8 / python33 and on winXP / python 322 and this is about dir exists tests on windows boxes.

say that Im using a windows 8 box with python 3.3

sys.version_info(major=3, minor=3, micro=0, releaselevel='final', serial=0)

I have this existing path on my hd:

D:\ImHere\meToo

a simple script that confirms me these directories exists (Im using a cmd admin prompt to execute the script) :

import os
from os import path
import sys
print(sys.version_info)

def there(p) :
    print('%s %s'%(p,'exists' if os.path.isdir(p) else 'does not exist'))

# got True, ok
print('\nusuals')
path = "D:\ImHere\meToo"
there(path)

# got False, ok
path += "\ImNot"
there(path)

# a bit more academic, tests ok
elms = path.split('\\')
jpath = ''
for elm in elms :
    jpath = os.path.join(jpath,elm)
    there(jpath)

# ... ok
rpath = elms[0] + os.sep + elms[1] + os.sep + elms[2] + os.sep + elms[3]
there(rpath)

this returns in the console :

usuals
D:\ImHere\meToo exists
D:\ImHere\meToo\ImNot does not exist
D: exists
D:ImHere exists
D:ImHere\meToo exists
D:ImHere\meToo\ImNot does not exist
D:\ImHere\meToo\ImNot does not exist

but compare this :

# now, this 'flat loop' is ok too
print('\nflat loop')
wpath = elms[0]
wpath += os.sep + elms[1]
wpath += os.sep + elms[2]
wpath += os.sep + elms[3]
there(wpath)

got :

flat loop
D:\ImHere\meToo\ImNot does not exist

with this :

# but if one calls isdir() in the process
# something is altered, and this ALWAYS returns dir exists
print('\nfails ?')
wpath = elms[0]
wpath += os.sep + elms[1]
there(wpath)
wpath += os.sep + elms[2]
there(wpath)
wpath += os.sep + elms[3]
there(wpath)

win8 / python 33: KO

fails ?
D:\ImHere exists
D:\ImHere\meToo exists
D:\ImHere\meToo\ImNot exists

winXP / python 322: OK

fails ?
D:\ImHere exists
D:\ImHere\meToo exists
D:\ImHere\meToo\ImNot does not exist

it seems the call to isdir() break something in the string ?

FIRST POST

( sorry if its a duplicate, I guess this String concatenation in Python could help about the case, but as I get mad tonight I need to share this.)

I wrote this little loop in a module to create recursive directories when they don't exist. this works fine on xp, python 3.2 :

def mkdir(path) :
    path = clean(path)  # this one generates unix like path
    parts = path.split('/')
    tpath = parts[0]
    for pi in range(1,len(parts)) :
        tpath += '/%s'%parts[pi]
        if os.path.isdir(tpath) == False :
            os.mkdir(tpath)

but when using windows8 + python 33, the directory test always return true.. ? after hours digging and experimenting about quotes and slashes stuffs.. I found the issue was this line:

tpath += '/%s'%parts[pi]

and

tpath = '%s/%s'%(tpath,parts[pi])

solved the case.

what is very odd for me, and by odd I mean absolutely psychedelic, is that the generated string whatever the code looks like to be the same :

print(os.path.isdir(tpath),tpath==path)

gives me 'True True' for the last directory in the first case : generated and input string are the same (and types are string), but the last directory does not exist..

and after the edition it returns a 'False True' as it should.

Im very frightened. the world looks like very odd since a couple of hours. what append with string += concatenation type is ok, bool() tests are ok... ? please save me from madness.. thanks a lot.

You should have a look at [`os.makedirs()`](http://docs.python.org/3.3/library/os.html#os.makedirs). — Nicolas, Jan 03 '13 at 09:16
Why aren't you using [**`os.makedirs()`**](http://docs.python.org/3/library/os.html#os.makedirs)? — johnsyweb, Jan 03 '13 at 09:16
And apart from that (as it's more generally useful), why aren't you using `os.path.join` to compose paths? (Of course, that doesn't invalidate the question.) — , Jan 03 '13 at 09:17
Also why aren't you using [**`os.path.join()`**](http://docs.python.org/3/library/os.path.html#os.path.join), which deals with the path separator being different on different platforms? — johnsyweb, Jan 03 '13 at 09:17
cause I came to python from the blender api, so Im not academic, but the question remains I think (and thanks for your replies). — jerome, Jan 03 '13 at 09:28
Also, I'm pretty damn sure that `+=` works as intended on strings, so I fear you did something else wrong. — Martijn Pieters, Jan 03 '13 at 09:29
unfortunately Im right.. this was working with python3.2/XP and its not working with p33/w8, that's all I can say :s — jerome, Jan 03 '13 at 09:33
Also, be carefull with backslashes in paths in normal strings. Or use raw strings like `path = r"D:\ImHere\meToo"` if you want to write it explicitly, or us normal slashes (fine also for Windows). `path = "D:/ImHere/meToo"`. — pepr, Jan 04 '13 at 08:48
thanks for the r tip pepr. if interested I posted a test script and updated/detailed my question about the REAL question I have ;) — jerome, Jan 04 '13 at 12:19
Have you tried Python 3.2 on Windows 8, or Python 3.3 on XP? — Mark Dickinson, Jan 04 '13 at 15:50
this could help yes, Ive no clue if it's python or w8 related — jerome, Jan 06 '13 at 00:43
@jerome: Did you ever fix this issue? There's an open Python bug report that looks related at http://bugs.python.org/issue17137 — Mark Dickinson, Feb 05 '13 at 22:03
@MartijnPieters, @jerome: And it turns out that this *is* a new bug in Python 3.3 (now fixed; the fix will show up in 3.3.1 when that gets released). So `+=` was *not* working as intended on strings. :-( — Mark Dickinson, Feb 10 '13 at 09:34
@MarkDickinson: Wow, so for Python 3.3 I was wrong! Just reproduced it on my Mac too. — Martijn Pieters, Feb 10 '13 at 10:10
@MartijnPieters: Yes, it's a bit scary. I guess the moral is: beware of a .0 release just after a major internal shakeup to the Unicode representation... — Mark Dickinson, Feb 10 '13 at 12:10
@MarkDickinson: it looks ok using join() for concatenation. many thanks for your answers (that's great, I'm not so mad) please have a look to my answer (yours actually) I did not have a lot of time to test any os/py version case. — jerome, Feb 16 '13 at 16:12

score 0 · Answer 1 · answered Feb 16 '13 at 16:06

to sum up : as Mark Dickinson mentionned, there's a bug with os.path.isdir() in python 3.3 related to the Unicode format(#17137). It looks like to be fixed for next releases and alpha.

meanwhile, to test if folder names exist in a given path, I got consistent results whatever the os and the python >= 3.2 version are, using os.path.join() rather than "path +=" string concatenation in the test loop :

path = "D:\ImHere\meToo\Imnot"

def there(p) :
    print('%s %s'%(p,'exists' if os.path.isdir(p) else 'does not exist'))

elms = path.split('\\')
jpath = ''
for elm in elms :
    jpath = os.path.join(jpath,elm)
    there(jpath)

so, don't use + concatenation with isdir() in python 3.3, and concatenate with join()

[ waiting for Mark approvals :) ]

string concatenation with python 33 / isdir() always returns True? (3 hours head bumping)

1 Answers1