Basic issue with glob in python

Question

I'm really unexpert in python, so forgive my question if stupid.

I'm trying a simple script that operates on all the files in a folder. However, I apparently can only access the folder recursively!

I explain. I have a folder, DATA, with subfolders for each day (of the form YYYY-MM-DD).

If I try

for filename in glob.glob('C:\Users\My username\Documents\DATA\2021-01-20\*'):
  print filename

I get no output.

However, if I try instead

for filename in glob.glob('C:\Users\My username\Documents\DATA\*\*'):
  print filename

the output is that expected:

C:\Users\My username\Documents\DATA\2021-01-20\210120_HOPG_sputteredTip0001.sxm
C:\Users\My username\Documents\DATA\2021-01-20\210120_HOPG_sputteredTip0002.sxm
...

I even tried different folder names (removing the dashes, using letters in the beginning, using only letters, using a shorter folder name) but the result is still the same.

What am I missing?

(BTW: I am on python 2.7, and it's because the program I need for the data is only compatible with python 2)

Replace the backslashes with forward slashes (or double them up to escape them: `C:\Users` becomes `C:\\Users`), backslashes are used to escape characters in strings — GPhilo, Jan 21 '21 at 12:12
Thanks, all suggestions actually work! Though using r is the best, no path manipulation so no errors and a reusable string — poomerang, Jan 22 '21 at 14:22

score 1 · Answer 1 · answered Jan 21 '21 at 12:24

Recursive file search is not possible with glob in Python 2.7. I.e. searching for files in a folder, its subfolders, sub-subfolders and so on.

You have two options:

use os.walk (you might need to change your code's structure however)
Use the backported pathlib2 module from PyPI https://pypi.org/project/pathlib2/ - which should include a glob function supporting the recursive search using ** wildcard.

Bjarke Sporring · Accepted Answer · 2021-02-13T10:19:44.087

1

Beware when using backslashes in strings. In Python this means escaping characters. Try prepending your string with r like so:

for filename in glob.glob(r'C:\Users\My username\Documents\DATA\*'):
    # Do you business

Edit:
As @poomerang has pointed out a shorter answer has previously been provided as to what 'r' does in Python here

Official docs for Python string-literals: Python 2.7 and for Python 3.8.

edited Feb 13 '21 at 10:19

answered Jan 21 '21 at 12:24

Bjarke Sporring

528
5
7

That was it! I had completely forgotten about escape characters, but that fixes it. Thanks! In case someone else is interested, [here](https://stackoverflow.com/questions/33729045/what-does-an-r-represent-before-a-string-in-python) I found the explanation for the `r` prefix – poomerang Jan 22 '21 at 14:13
@poomerang lovely. Glad to help :D. I've updated the answer with the link you provided and links to the official documentation – Bjarke Sporring Jan 23 '21 at 15:40

Basic issue with glob in python

2 Answers2