0

I'm attempting to parse some event logs into a .csv from a .log format so my event manager software can read it.

I'm trying to search a folder for all files ending in a .log extension, then look at each line for the word "Security," separate the values by tab and paste those values into a .csv file automatically created.

The problem is when the data is pasted it adds extra rows and rb into the file. An example would be: row 1: data, row 2: blank, row 3: rb, row 4: blank, row 5: data, etc.

This is what I'm currently working with.

import sys, random, os, csv, fileinput, glob, re, pygame
from os import path
import time as t

date = t.localtime(t.time())
destination = raw_input("Give the path of your file: ")

for logtest in glob.glob(destination +"*.log"):
    logname = logtest[:-4]
    assert os.path.exists(logtest), "No file was found at, "+str(logtest)

name = "Log_Filter_%d_%d_%d_%d_%d_%d.csv" %(date[1], date[2], (date[0] %100), date[3], date[4], date[5])
auditlog = open(logtest, "r")

if not(path.isfile(destination + name)):
    splittestfile = csv.writer(open(destination + name, "a"))
    for x in auditlog:
        if "Security" in x:
            splitstr = csv.reader((x, "rb"), delimiter = "\t")
            splittestfile.writerows(splitstr)
skcuf
  • 1
  • 2
  • Try opening your csv output file in binary mode. See http://stackoverflow.com/questions/3348460/python-getting-rid-of-extra-line and http://stackoverflow.com/questions/8746908/why-does-csv-file-contain-a-blank-line-in-between-each-data-line-when-outputting for detailed discussions on Windows/Python2/Python3... – sal Feb 12 '16 at 17:08
  • That god rid of the blank line, but I still have the rb between lines. – skcuf Feb 12 '16 at 18:17
  • Well...I figured out that the "rb" is what's writing to the file. But if I remove it I just get each individual character from every word within the line. I'm now looking for why it seems to be ignoring the delimiter = "\t" if I remove the "rb" – skcuf Feb 12 '16 at 18:36
  • I have found if I change to splitstr = csv.reader(x.split("\n\n"), delimiter = "\t") it works. Is there a more elegant way? – skcuf Feb 12 '16 at 18:56
  • Oh, got it... I think your mistake is the fact that you are passing a line to the csv.reader: you can actually pass the file to it. Check the very first reader example on https://docs.python.org/2/library/csv.html – sal Feb 12 '16 at 18:59
  • Scrap that: the mistake is that the csv.reader expects a csv file, not a text file... – sal Feb 12 '16 at 19:00
  • Can you add some lines from the logtest file to the description? Also, please provide python version and platform. – sal Feb 12 '16 at 19:01
  • I can't post any log files here due to the information contained. Using a text format with headers "Date, Time, Log Type, Keyword, Username" will be fine. I didn't import the entire file because I need to filter out for keywords, like "Security", and grab only that line. I'm not sure of the python version I'm using. I have 2.7, 3.4, and 3.5 on my machine, but am editing with Geany and don't know how to check which version I'm using. – skcuf Feb 12 '16 at 19:38
  • Ok, my wild guess is that you could just read the lines, split them in a list and feed the list to the csv writer. Maybe some cvs power users will be able to guide you on what you describe, which I seem to understand should be supported as well. – sal Feb 12 '16 at 23:53
  • I was able to get it working using split, so now I have x.split instead of trying to import the string as a list. – skcuf Feb 24 '16 at 13:01

0 Answers0