Find a syslog file based on the date in its name

Question

I am working on a Python script that will allow me to better find specific syslogs files. The script simply consists in typing the IP address of the concerned machine and then the date (in YYYYMMDD format) since there is one log file per day on the server..

The script is largely copied and adapted from a similar script that a colleague programmer did before me. But I don't have as much expertise as he does on Python, and he didn't leave comments on his code.

# This Python file uses the following encoding: utf-8
import os, sys, re
name = raw_input("IP Address? \n")

dir = os.listdir(os.getcwd())

flag_equip = False
flag_h_start = True
flag_h_end = True
flag_date = True
for n in dir :
        if(n!="syslog_hosts" and n!="syslog_hosts.avec.domain" and n!="syslog_hosts.BKP"):
                for nn in os.listdir("/applis/syslog/syslog_cpe"):
                        if(re.search(name,nn)):
                                flag_equip = True
                                print("Equipment found!")
                                while(flag_date):
                                    date = raw_input("date AAAAMMJJ ?\n")
                                    if(re.search("[\d]{8}$",date)):
                                        flag_date = False
                                for nnn in os.listdir("/applis/syslog/syslog_cpe"+"/"+n+"/"+nn):
                                        raw_filename = nnn.split(".")
                                        for i in raw_filename :
                                                if(i==date):
                                                        break
          
         
          
         
if(flag_equip==False):
        print("Equipment not found")

I have a problem with the part where I have to enter the date. No matter what I put in the date, it always gives me this error below. The IP address matches the one I entered but not the date which is always the first one of the machine.

  File "scriptsyslog.py", line 21, in <module>
    for nnn in os.listdir("/applis/syslog/syslog_cpe"+"/"+n+"/"+nn):
OSError: [Errno 2] No such file or directory:'/applis/syslog/syslog_cpe/.bash_history/100.117.130.80.20220404.log'

My code may seem a bit strange but that's because it's not complete. I hope that's clear enough. The log files are all named in the format "[IP address][YYYYMMDD].log" For example 100.117.130.80.20220410.log

Thank you in advance. This is my first question on StackOverflow.

Answer 1

Here's an attempt to fix the script.

Rather than require interactive I/O, this reads the IP address and the date as command-line arguments. I think you will find that this is both more user-friendly for interactive use and easier to use from other scripts programmatically.

I can't guess what your file and directory tree looks like, but I have reimplemented the code based on some guesswork. One of the hard parts was figuring out how to give the variables less misleading or cryptic names, but your question doesn't explain how the files are organized, so I could have guessed some parts wrong.

Your original script looks like you were traversing multiple nested directories, but if your prose description is correct, you just want to scan one directory and find the file for the specified IP address and date. This simply assumes that you have all your files in /applis/syslog/syslog_cpe and that the scanning of files in the current directory was an incorrect, erroneous addition to a previously working script.

# Stylistic fix: Each import on a separate line
import os
import sys

if len(sys.argv) != 3:
    print("Syntax: %s <ip> <yyyymmdd>" % sys.argv[0].split('/')[-1])
    sys.exit(1)

filename = os.path.join(
    "/applis/syslog/syslog_cpe",
    sys.argv[1] + "." + sys.argv[2] + ".log")
if os.path.exists(filename):
    print(filename)
else:
    # Signal failure to caller
    sys.exit(2)

There were many other things wrong with your script; here is an attempt I came up with while trying to pick apart what your script was actually doing, which I'm leaving here in case it could entertain or enlighten you.

# Stylistic fix: Each import on a separate line
import os
import sys
import re


# Stylistic fix: Encapsulate main code into a function
def scanlog(ip_address, date, basedir="/applis/syslog/syslog_cpe"):
    """
    Return any log in the specified directory matching the
    IP address and date pattern.
    """
    # Validate input before starting to loop
    if not re.search(r"^\d{8}$", date):
        raise ValueError("date must be exactly eight digits yyyymmdd")
    if not re.search(r"^\d{1,3}(?:\.\d{1,3}){3}$", ip_address):
        raise ValueError("IP address must be four octets with dots")

    for file in os.listdir(basedir):
        # Stylistic fix: use value not in (x, y, z) over value != x and value != y and value != z
        # ... Except let's also say if value in then continue
        if file in ("syslog_hosts", "syslog_hosts.avec.domain", "syslog_hosts.BKP"):
            continue
        # ... So that we can continue on the same indentation level
        # Don't use a regex for simple substring matching
        if file.startswith(ip_address):
            # print("Equipment found!")
            if file.split(".")[-2] == date:
                return file
    # print("Equipment not found!")


# Stylistic fix: Call main function if invoked as a script
def main():
    if len(sys.argv) != 3:
        print("Syntax: %s <ip> <yyyymmdd>" % sys.argv[0].split('/')[-1])
        sys.exit(1)
    found = scanlog(*sys.argv[1:])
    if found:
        print(found)
    else:
        # Signal error to caller
        sys.exit(2)

if __name__ == "__main__":
    main()

There is no reason to separately indicate the encoding; Python 3 source generally uses UTF-8 by default, but this script doesn't contain any characters which are not ASCII, so it doesn't even really matter.

I think this should work under Python 2 as well, but it was written and tested with Python 3. I don't think it makes sense to stay on Python 2 any longer anyway.