Shell script: Merge files from within a date range

Question

I'd like to merge several log files within a given date range. For example, I have 5 days of log files in a directory:

server.log.2016-04-14-00 
server.log.2016-04-14-01
. . .
server.log.2016-04-18-23
server.log.2016-04-19-00
server.log.2016-04-19-01

I know I can use cat to merge the files, but how can I code in a shell script so that only the files between, for example, 2016-04-17-22 and 2016-04-18-01 are selected?

Please learn to use the search feature **before** asking a question. Searching for `[bash] log file filter` returns 6 items while `[bash] log file` returns over 4000. Please look ;-) ... Good luck. — shellter, Apr 19 '16 at 23:20

Rany Albeg Wein · Answer 1 · 2016-04-20T00:27:16.043

The following script accepts server's log file as its first argument. Two important variables are from_date and to_date which control the from-to range. They're hard-coded in the script, and you might want to change this to enhance script's usage flexibility.

#!/bin/bash

# Server's log file.
server_log_file=$1
# The date from which the relevant part of the log file should be printed.
from_date='2016/04/14 00:00'
# The date until which the relevant part of the log file should be printed.
to_date='2016/04/19 01:00'

# Uses 'date' to convert a date to seconds since epoch.
# Arguments: $1 - A date acceptable by the 'date' command. e.g. 2016/04/14 23:00
date_to_epoch_sec() { 
    local d=$1
    printf '%s' "$(date --date="$d" '+%s')"
}

# Convert 'from' and 'to' dates to seconds since epoch.
from_date_sec=$(date_to_epoch_sec "$from_date")
to_date_sec=$(date_to_epoch_sec "$to_date" )

# Iterate over log file entries.
while IFS=. read -r s l date; do
    # Read and parse the date part.
    IFS=- read -r y m d h <<< "$date"
    # Convert the date part to seconds since epoch.
    date_sec=$(date_to_epoch_sec "$y/$m/$d $h:00")

    # If current date is within range, print the enire line as it was originally read.
    if (( date_sec > from_date_sec && date_sec < to_date_sec )); then
        printf '%s.%s.%s\n' "$s" "$l" "$date"
    fi

done < "$server_log_file"

In order to test it I created the following file, named logfile:

server.log.2016-04-14-00
server.log.2016-04-14-01
server.log.2016-04-18-23
server.log.2016-04-19-00
server.log.2016-04-19-01

Usage example ( script name is sof ):

$ # Should print logs from 2016/04/14 00:00 to 2016/04/19 01:00 
$ ./sof logfile 
server.log.2016-04-14-01
server.log.2016-04-18-23
server.log.2016-04-19-00

Shell script: Merge files from within a date range

1 Answers1