using glob to exclude time series files

Question

My files in a directory are monthly data spanning several years, with characters like 0001-01-01, 0001-02-01, ..., 0005-01-01, ..., 0010-12-01 (yyyy-mm-dd) in the middle of each file name.

Now, I would like to exclude say the 0001* files. If I wrote sorted(glob.glob(mydirectory/filename-000[!1]*)) only gives me 0002 ~~ 0009 files, while the 0010 files are not included.

What should I do to only exclude the 0001* files?

If I wrote sorted(glob.glob(mydirectory/filename-000[2-9]*)) also only gives me 0002-0009 files, what should I do to include the 0010* files?

I also tried filename-{000[2-9],00[10-12]}*, which does not work.

Thanks,

score 0 · Answer 1 · answered Dec 11 '19 at 22:32

0

Just add two globs together.

files = glob.glob(mydirectory/filename-000[!1]*) + glob.glob(mydirectory/filename-0010*)

answered Dec 11 '19 at 22:32

James

32,991
4
47
70

Thanks! I didn't know this simple + before. – Lin Lin Dec 14 '19 at 01:12
No problem. Remember to mark the question as answered (the little check mark). – James Dec 14 '19 at 22:41

Alexandr Shurigin · Answer 2 · 2019-12-11T22:51:08.647

Do it like this

import glob
from pprint import pprint

pprint(sorted(glob.glob("filename-???[!1]*")))

Where ? is "any character", like * is for "any characters string"

For me, this works pretty well.

(stackoverflow)  ~/PycharmProjects/stackoverflow  ls -la filename-00*
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0001
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0002
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0003
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0004
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0005
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0006
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0007
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0008
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0009
-rw-r--r--  1 dude  staff  0 Dec 11 23:31 filename-0010
(stackoverflow)  ~/PycharmProjects/stackoverflow # python test123.py
['filename-0002',
 'filename-0003',
 'filename-0004',
 'filename-0005',
 'filename-0006',
 'filename-0007',
 'filename-0008',
 'filename-0009',
 'filename-0010']
(stackoverflow)  ~/PycharmProjects/stackoverflow #

Hi, thank you very much for your solution! However, this method ???[!1]* will also get rid of the 0011 files. — Lin Lin, Dec 12 '19 at 01:24
oh, you are right :( so I don't think you can do it in a one glob call. — Alexandr Shurigin, Dec 12 '19 at 01:39

score 0 · Accepted Answer · answered Dec 11 '19 at 23:03

glob supports Unix shell pattern rules, but not more complex expressions. But, while you are in Python, you can use lots of filtering techniques, including regular expressions. If my directory looks like this:

$ find
.
./mydirectory
./mydirectory/filename-0001-01-01
./mydirectory/filename-0001-02-01
./mydirectory/filename-0010-12-01
./mydirectory/filename-0005-01-01

Then

[f for f in glob.glob(r"mydirectory/filename-*") if "0001" not in f]

will return:

['mydirectory/filename-0010-12-01', 'mydirectory/filename-0005-01-01']

This is further explained in this SO answer

I think this works in a more convenient and flexible way. – Lin Lin Dec 14 '19 at 01:11 — Lin Lin, Dec 14 '19 at 01:11

using glob to exclude time series files

3 Answers3