how can I take an specific line in big file which has many similar lines, using python?

Question

have different files with same name, in different directories. In these files there are lines which are almost equal, I would like to take out only the last line of these ones( there are more lines after it) and write it in another file.

So far what I have done:

#!/usr/bin/env python

import os

def cd_grep():
   for file in os.listdir("."):
     if os.path.isfile(file):
       for line in open("graph.txt"):
                  if " 4.49" in line:                               
                       line_list=[line] 
   g = open('comparation','a') 
   g.write ("%s" % (line[0:4]))
   g.close()
os.chdir('4.294')
cd_grep()
os.chdir(os.pardir)
os.chdir('4.394')
cd_grep()
os.chdir(os.pardir)
os.chdir('4.494')
cd_grep()
os.chdir(os.pardir)
os.chdir('4.594')
cd_grep()
os.chdir(os.pardir)
os.chdir('4.694')
cd_grep()

I've created a list because I am gonna take only a specific information of the whole line.

Finally I got that this procedure only works for small files and only if the last line of the file contains the term I'm searching. For big files, I got this message ( inside the file, which I was hoping to get the line):
Voluntary context switches: 3403

Any idea or suggestion will be very appreciate.

First of all format your code so it is readable and properly indented. — dkamins, May 19 '11 at 19:13
Ok, thanks. First time I submitted a question out here. I hope it is better now. — ziulfer, May 19 '11 at 19:34
Now show the exact error you get. Is it an exception? Does your program finish? Where do you see that message? — dkamins, May 19 '11 at 19:37
I got the error inside the file, where I was hoping to get the line. I don't know if it is an exception ( because I don't know exactly what its means). The program finish. — ziulfer, May 19 '11 at 19:54
Are you sure that line `for line in open("graph.txt")` is correctly indented and why are you going through all files in directory if you are only interested in "graph.txt"? — Juha Autero, May 19 '11 at 20:08
@Juha Autero, It wasn't indented correctly here, thanks. I thought it was the right way. Please tell me a better way. — ziulfer, May 19 '11 at 20:17
I'm guessing you aren't closing your files. See my answer below or http://stackoverflow.com/questions/1478697/for-line-in-openfilename — dr jimbob, May 19 '11 at 20:19

aeter · Accepted Answer · 2011-05-20T15:43:29.057

2

Not sure about the error you are receiving (after your last edit).

I have tried to rewrite the code a bit, hope it gives you a result similar to what you need (WARNING: not tested).

with open ('comparation', 'a') as write_file:
  for path, dirs, files in os.walk(os.getcwd()):
    for filename in [f for f in files if f == "graph.txt"]:
      filepath = os.path.abspath(os.path.join(path, filename))
      with open(filepath) as f:
        for line in f:
          if " 4.49" in line:
            last = line
        write_file.write("File: %s, Line: %s\n" % (filepath, last[0:4]))

edited May 20 '11 at 15:43

answered May 19 '11 at 20:17

aeter

11,960
6
27
29

It worked pretty well. The only problem was, for some reason, it gives me the same, output, many times. File: /path1/ , line:xxxxxx File: /path1/ , line:xxxxxx File: /path1/ , line:xxxxxx File: /path1/ , line:xxxxxx File: /path2/ , line:yyyyyy File: /path2/ , line:yyyyyy File: /path2/ , line:yyyyyy File: /path2/ , line:yyyyyy Should I, write the file(comparation) outside the loop? – ziulfer May 20 '11 at 11:52
I'm sorry, I cannot test it. I think you should be able to experiment with it at the interactive python interpreter; without test dirs/files I cannot answer you well. I have also forgotten to add "\n" when appending to the file (added now), which seems to have made one long line in the write_file. Hope it works better now; have fun tweaking it. – aeter May 20 '11 at 15:43
I am so sorry. I didn't delete the two first lines of my code. After doing that worked perfect. Thank you very much. – ziulfer May 20 '11 at 22:41

score 0 · Answer 2 · answered May 19 '11 at 20:14

I'm guessing you aren't closing your files.

def cd_grep():
    for file in os.listdir("."):
        if os.path.isfile(file):
           graph_file = open('graph.txt'):
           for line in graph_file:
               if " 4.49" in line:                               
                   line_list=[line] 
           graph_file.close()
    g = open('comparation','a') 
    g.write ("%s" % (line[0:4]))
    g.close()

Or much better use with to open (and always close) your files.

def cd_grep():
    for file in os.listdir("."):
        if os.path.isfile(file):
           with open('graph.txt') as graph_file:
               for line in graph_file:
                   if " 4.49" in line:                               
                       line_list=[line] 
    with open('comparation','a') as g:
        g.write ("%s" % (line[0:4]))

thanks you very much for attention and tips. still it's taking the last line of the file, even it isn't the correct line. — ziulfer, May 20 '11 at 22:36

how can I take an specific line in big file which has many similar lines, using python?

2 Answers2