19

I am trying to read from multiple input files and print the second row from each file next to each other as a table

import sys
import fileinput

with fileinput.input(files=('cutflow_TTJets_1l.txt ', 'cutflow_TTJets_1l.txt ')) as f:
    for line in f:
        proc(line)



def proc(line):
     parts = line.split("&") # split line into parts
     if  "&" in line:    # if at least 2 parts/columns
         print parts[1] # print column 2 

But I get a "AttributeError: FileInput instance has no attribute '__exit__'"

Dan Oberlam
  • 2,435
  • 9
  • 36
  • 54
Just_Newbie
  • 447
  • 1
  • 4
  • 11
  • As best I can tell, Python closes the files in fileinput as soon as they're done - at least the docs make no mention of needing to close them. As such it wouldn't make sense to use a context manager with fileinput, and it wouldn't make sense to implement `__enter__` or `__exit__` – Dan Oberlam Jun 14 '15 at 22:20
  • sorry, I dont see how this helps me – Just_Newbie Jun 14 '15 at 22:31
  • The solution is to not use a context manager. – Dan Oberlam Jun 14 '15 at 22:36
  • What is your Python version? Is it Python 2.7? You could use `with contextlib.closing(fileinput.FileInpt(..)) as file:` on Python 2. – jfs May 10 '17 at 10:34

2 Answers2

32

The problem is that as of python 2.7.10, the fileinput module does not support being used as a context manager, i.e. the with statement, so you have to handle closing the sequence yourself. The following should work:

f = fileinput.input(files=('cutflow_TTJets_1l.txt ', 'cutflow_TTJets_1l.txt '))

for line in f:
    proc(line)

f.close()

Note that in recent versions of python 3, you can use this module as a context manager.


For the second part of the question, assuming that each file is similarly formatted with an equal number of data lines of the form xxxxxx & xxxxx, one can make a table of the data from the second column of each data as follows:

Start with an empty list to be a table where the rows will be lists of second column entries from each file:

table = []

Now iterate over all lines in the fileinput sequence, using the fileinput.isfirstline() to check if we are at a new file and make a new row:

for line in f:
    if fileinput.isfirstline():
        row = []
        table.append(row)
    parts = line.split('&')
    if len(parts) > 1:
        row.append(parts[1].strip())

f.close()                      

Now table will be the transpose of what you really want, which is each row containing the second column entries of a given line of each file. To transpose the list, one can use zip and then loop over rows the transposed table, using the join string method to print each row with a comma separator (or whatever separator you want):

for row in zip(*table):
    print(', '.join(row))                             
Eric Appelt
  • 2,843
  • 15
  • 20
  • 1
    Does that close every file? Or just the last one (i.e. the other ones are closed when they are done being used)? – Dan Oberlam Jun 14 '15 at 22:37
  • 1
    @Just_Newbie that's because you're using `proc` before you define it. Reorder things – Dan Oberlam Jun 14 '15 at 22:37
  • Great ! and how would it be possible to get them a a table ie the two columns side-by-side ? – Just_Newbie Jun 14 '15 at 22:47
  • 1
    @Just_Newbie That part is kind of tricky to answer without . Do both files have the same number of lines of the form `xxxx & xxxx`? If not, should there be empty places in the table? Should line 7 from the first file always match to line 7 from the second? – Eric Appelt Jun 14 '15 at 23:03
  • 2
    @Dannnno fileinput just iterates through each file in the sequence, and only actually opens a file when `next()` is called if the end of the previous file is reached and there are still files left. `close()` just calls `nextfile()` which closes the current file (if one is open) and advances to the next file in the sequence, without actually opening it. – Eric Appelt Jun 14 '15 at 23:05
  • @EricAppelt - Yes, all input files should have the same number of lines and format match between different files – Just_Newbie Jun 14 '15 at 23:06
  • 1
    @Just_Newbie - ok, I added a solution of how this might be done using `fileinput` – Eric Appelt Jun 15 '15 at 00:03
4

If something has open/close methods, use contextlib.closing:

import sys
import fileinput
from contextlib import closing

with closing(fileinput.input(files=('cutflow_TTJets_1l.txt ', 'cutflow_TTJets_1l.txt '))) as f:
    for line in f:
        proc(line)
kzh
  • 19,810
  • 13
  • 73
  • 97