0

I have lists of files in folders where I want to delete the last entry; I have each of these saved as a .txt file. The lists will differ in length, so I can't say, for example, always delete the 7th member from the list. These have random names, so I just want to import a list of files (or a folder) into python and delete the last member.

I've found this bit of code: Delete Lines From a File in Python that works well for an individual file. This reads a .txt, deletes one line, and resaves as a .txt. For my purposes, this will work, because then I can input all of my updated .txt files into a merged dataframe.

with open('file1.txt', 'r+') as fp:
    lines = fp.readlines()
    fp.seek(0)
    fp.truncate()
    fp.writelines(lines[:-1])

However, I'm looking for a way to import all of my files without typing in their individual names. I've used glob for this before eg:

interesting_files = glob.glob('/Users/location/*.txt')
dataframe1 = pd.concat((pd.read_csv(f, sep= ' ', names =['folder','num_files'])
                            for f in interesting_files))

So, how can I import many files instead of just file1.txt—either one source location with all my files in there like glob does, or as a for loop? Are there other approaches that would work, like reading in each .txt file into a dataframe, then dropping the final row before reading in the next .txt file?

martineau
  • 119,623
  • 25
  • 170
  • 301
Jordan
  • 11
  • 3
  • That's weird method, why not just call [`.truncate()`](https://docs.python.org/3/library/io.html#io.IOBase.truncate) with return of current position *(using [`.tell()`](https://docs.python.org/3/library/io.html#io.IOBase.tell))*? Check [my answer](https://stackoverflow.com/a/70565132/10824407) under [similar question](https://stackoverflow.com/q/70564882/10824407). – Olvin Roght Jan 18 '22 at 21:41
  • I saw your .tell() post as I was researching this. However, that only appears to work for a single file, not iteratively working through many files. – Jordan Jan 19 '22 at 18:40
  • Well, put it inside a loop which iterate over files and proceed – Olvin Roght Jan 19 '22 at 18:49

1 Answers1

0

The read_csv method supports a skipfooter parameter. So you can use that while reading it. No need for preprocessing.

To read all files in a folder, I'd do a list comprehension.

[pd.read_csv(file, skipfooter = 1) for file in interesting_files] will return a list of DataFrames. This can then be wrapped by a pd.concat:

df = pd.concat([pd.read_csv(file, skipfooter = 1) for file in interesting_files])

flom
  • 71
  • 2