1

My issue is: I want to sort my output file based on date/time. These files were concatenated from 5 directories to one outfile. So output has filewise entries and as per that sorted date/time, not as expected date/time sort applied on whole output file my code

my Code:

f = ['Dir0/fruit.log', 'Dir1/fruit.log', 'Dir2/fruit.log','Dir3/fruit.log', 'Dir4/fruit.log']
with open("outp/out2.txt", "w") as f1:
    for fname in f:
        with open(fname) as infile:
            for line in infile:
                f1.write(line)
Chubas
  • 17,823
  • 4
  • 48
  • 48
Ashish Pandya
  • 11
  • 2
  • Paste the example content of `fruit.log` please. – bosnjak Apr 27 '15 at 11:53
  • You want these fruit.log files concatenated in the `out2.txt` sorted by time they were created? – ljk321 Apr 27 '15 at 12:22
  • DirA/fr.log: Dec 10 21:18:20 delivery /usr/local/bin/fruittool[27]: Pineapple [0x000002d7]: 1420 Dec 10 21:18:40 delivery /usr/local/bin/fruittool[27]: Apple [0x000002d8]: 1 Dec 10 21:18:40 delivery /usr/local/bin/fruittool[27]: Banana [0x000002d8]: 6683 Dec 10 21:18:40 delivery /usr/local/bin/fruittool[27]: Carrots [0x000002d8]: 2979 – Ashish Pandya Apr 27 '15 at 21:08
  • DirB/fr.log: Dec 10 22:41:29 delivery /usr/local/bin/fruittool[27]: Orange [0x000003c9]: -380 Dec 10 22:41:29 delivery /usr/local/bin/fruittool[27]: Lemon [0x000003c9]: 0 Dec 10 22:41:29 delivery /usr/local/bin/fruittool[27]: Lime [0x000003c9]: 4188 – Ashish Pandya Apr 27 '15 at 21:09
  • say above 2 files in DirA inside it fr.log file and DirB inside it fr.log file. Both have time wise entries. my 1st task of merge and put into 1 file done but not sorted date and time wise. Please help me on this – Ashish Pandya Apr 27 '15 at 21:10
  • Very similar question: http://stackoverflow.com/questions/464342/combining-two-sorted-lists-in-python – Robᵩ Apr 28 '15 at 02:59
  • Please add the log contents as part of the question (or put a link to an external file if the logs are too big) instead of comments, so the question becomes more clear. – Chubas Apr 29 '15 at 16:07

1 Answers1

0

This program reads multiple logs into a single list, sorts the list, and writes the result to a combined, sorted file.

f = ['DirA/fr.log', 'DirB/fr.log']

import datetime
def key(line):
    line = line.split(None,3)
    line = line[:3]
    line = ' '.join(line)
    return datetime.datetime.strptime(line, "%b %d %H:%M:%S")

lines = []
with open("outp/out2.txt", "w") as f1:
    for fname in f:
        with open(fname) as infile:
            lines.extend(infile.readlines())
        lines.sort(key=key)
    f1.writelines(lines)

If you have more than, say, 1,000,000 lines to sort, or if you find this algorithm takes longer than a minute or two, you may prefer to merge the lists as you go.

Robᵩ
  • 163,533
  • 20
  • 239
  • 308