4

I am reading some files using glob.glob(). I want to read all the files with names 123*.txt except those with 123*error.txt. Also, is there a way to print the filenames in the for loop, which is inside pd.concat()?

fields = ['StudentID', 'Grade']
path= 'C:/script_testing/'

parse = lambda f: pd.read_csv(f, usecols=fields)
table3 = pd.concat(
[parse(f) for f in glob.glob('C:/script_testing/**/*.txt', recursive=True)]
).pipe(lambda d: pd.crosstab(d.StudentID, d.Grade))
piRSquared
  • 285,575
  • 57
  • 475
  • 624
Sikander Waheed
  • 85
  • 1
  • 1
  • 4

1 Answers1

7

Use this pattern

files = glob.glob('C:/script_testing/**/123*[!error].txt`, recursive=True)

Then proceed

fields = ['StudentID', 'Grade']
path= 'C:/script_testing/'

parse = lambda f: pd.read_csv(f, usecols=fields)
table3 = pd.concat(
    [parse(f) for f in files]
).pipe(lambda d: pd.crosstab(d.StudentID, d.Grade))

Reference this post

piRSquared
  • 285,575
  • 57
  • 475
  • 624