bash script to get the spent time from a file

Question

I have a log file that shows switch time between my scripts:

Tue Oct 24 11:57:54 IRST 2017 Script switched from abc to XYZ 
Tue Oct 24 14:03:41 IRST 2017 Script switched from XYZ to ZEN 
Tue Oct 24 15:43:16 IRST 2017 Script switched from ZEN to XYZ 
Tue Oct 24 17:07:25 IRST 2017 Script switched from XYZ to ZEN 
Tue Oct 24 18:40:48 IRST 2017 Script switched from ZEN to XLS 
Tue Oct 24 19:52:26 IRST 2017 Script switched from XLS to XYZ 
Tue Oct 24 20:20:30 IRST 2017 Script switched from XYZ to ZEN 
Tue Oct 24 20:36:06 IRST 2017 Script switched from ZEN to XLS 
Tue Oct 24 21:01:03 IRST 2017 Script switched from XLS to XYZ 
Tue Oct 24 21:47:47 IRST 2017 Script switched from XYZ to ZEN

How do I get total time spent on each script with bash So the output shows like this:

abc 2 hours 30 min 40 sec
XYZ 3 hours 23 min 45 sec
zen ...
XLS ...

StackOverflow is a forum for helping you with your programs. You have to write your own program first, then really try to get it working, and once you've hit a final roadblock with some good debugging, then you ask your question. You still need to do the first step of writing a program yourself first. — Mort, Nov 11 '17 at 02:03

hek2mgl · Answer 1 · 2017-10-26T05:11:11.033

You can use the following gawk program:

time_spent.awk

BEGIN {
    months["Jan"] = "01"
    months["Feb"] = "02"
    months["Mar"] = "03"
    months["Apr"] = "04"
    months["May"] = "05"
    months["Jun"] = "06"
    months["Jul"] = "07"
    months["Aug"] = "08"
    months["Seb"] = "09"
    months["Oct"] = "10"
    months["Nov"] = "11"
    months["Dec"] = "12"
}

{
    split($4, time, ":")
    # mktime() manual: https://www.gnu.org/software/gawk/manual/html_node/Time-Functions.html
    now = mktime($6" "months[$2]" "$3" "time[1]" "time[2]" "time[3])
    prv = $(NF-2)
    cur = $(NF)
    start[cur] = now
    spent[prv]+=start[prv]?now-start[prv]:0
}

END {
    for(i in spent) {
        printf "%s seconds spent in %s\n", spent[i], i
    }
}

Save it into a file time_spent.awk and execute it like this:

gawk -f time_spent.awk input.log

Output from the above example:

5795 seconds spent in XLS
0 seconds spent in abc
17084 seconds spent in XYZ
12514 seconds spent in ZEN

Nice, anyway to convert seconds to hour:min:sec ? So output become ```XYZ: 02:34:25``` — papampi, Oct 25 '17 at 21:13
gives me this error : ``` awk: spent_time: line 32: function mktime never defined ``` — papampi, Oct 25 '17 at 22:24

eddiem · Answer 2 · 2017-10-26T20:57:34.847

#!/usr/bin/env python

import sys
from time import strptime
from datetime import datetime

intervals = (
    ('weeks', 604800),  # 60 * 60 * 24 * 7
    ('days', 86400),    # 60 * 60 * 24
    ('hours', 3600),    # 60 * 60
    ('minutes', 60),
    ('seconds', 1),
    )

def display_time(seconds, granularity=2):
    result = []

    for name, count in intervals:
        value = seconds // count
        if value:
            seconds -= value * count
            if value == 1:
                name = name.rstrip('s')
            result.append("{} {}".format(value, name))
    return ' '.join(result[:granularity])

with open(sys.argv[1], "rb") as df:
    lines = df.readlines()

totals = {}

for i in range(len(lines)-1):
    (_,t1,t2,t3,_,t4,_,_,_,_,_,scr) = lines[i].strip().split(' ')
    st = datetime.strptime(' '.join([t1,t2,t3,t4]), "%b %d %H:%M:%S %Y")
    (_,t1,t2,t3,_,t4,_,_,_,_,_,_) = lines[i+1].strip().split(' ')
    et = datetime.strptime(' '.join([t1,t2,t3,t4]), "%b %d %H:%M:%S %Y")
    if scr not in totals:
        totals[scr] = 0
    totals[scr] += (et-st).seconds

    print("{} {}".format(scr,display_time((et-st).seconds, 3)))

print("\nTotals:")
for scr in totals:
    print("{} {}".format(scr,display_time(totals[scr], 3)))

Here is the output, assuming your times are in a file named logfile:

$ ./times.py logfile
XYZ 2 hours 5 minutes 47 seconds
ZEN 1 hour 39 minutes 35 seconds
XYZ 1 hour 24 minutes 9 seconds
ZEN 1 hour 33 minutes 23 seconds
XLS 1 hour 11 minutes 38 seconds
XYZ 28 minutes 4 seconds
ZEN 15 minutes 36 seconds
XLS 24 minutes 57 seconds
XYZ 46 minutes 44 seconds

Totals:
XLS 1 hour 36 minutes 35 seconds
XYZ 4 hours 44 minutes 44 seconds
ZEN 3 hours 28 minutes 34 seconds

$

Note: I lifted the handy display_time function from here: Python function to convert seconds into minutes, hours, and days.

Got it, it gives separate times spent on each, how to add them and make a sum of each ? — papampi, Oct 25 '17 at 22:28

iamauser · Accepted Answer · 2017-10-26T22:20:21.820

Assuming you have a log file named test.txt, following script should work,

#!/bin/bash

dtime=0
sname=""
while read line
do
    _dtime=$(echo "$line" | awk '{print $1,$2,$3,$4}')
    _sname=$(echo "$line" | awk '{print $10}')

   _dtimesec=$(date +%s -d "$_dtime")
   _timediff=$(( _dtimesec - dtime ))
   [ "x$sname" != "x" ] && printf "$sname %d hours %d minutes %d seconds\n" $(($_timediff/3600)) $(($_timediff%3600/60)) $(($_timediff%60))
   dtime=$_dtimesec
   sname=$_sname

done < test.txt

This will produce an output like the following:

 ]$ ./test
 abc 2 hours 5 minutes 47 seconds
 XYZ 1 hours 39 minutes 35 seconds
 ZEN 1 hours 24 minutes 9 seconds
 XYZ 1 hours 33 minutes 23 seconds
 ZEN 1 hours 11 minutes 38 seconds
 XLS 0 hours 28 minutes 4 seconds
 XYZ 0 hours 15 minutes 36 seconds
 ZEN 0 hours 24 minutes 57 seconds
 XLS 0 hours 46 minutes 44 seconds

EDIT

In order to find total amount of time spent by each script, this modified script should do the job:

   #!/bin/bash

   dtime=0
   sname=""
   namearr=()
   timearr=()
   while read line
   do
       _dtime=$(echo "$line" | awk '{print $1,$2,$3,$4}')
       _sname=$(echo "$line" | awk '{print $10}')
       _dtimesec=$(date +%s -d "$_dtime")
       _timediff=$(( _dtimesec - dtime ))

       _rc=1
       for n in "${!namearr[@]}"
       do
              if [ "${namearr[$n]}" == "$_sname" ]; then
                  export _rc=$?
                  export ind=$n
                  break;
              else
                  export _rc=1
              fi
       done
       if [ $_rc -eq 0 ]; then
       timearr[$ind]=$(( ${timearr[$ind]} + _timediff ))
       else
              if [ $dtime -eq 0 ] && [ "x$sname" == "x" ]; then
                  :
              else
                  namearr+=($_sname)
                  timearr+=($_timediff)
              fi
      fi

       dtime=$_dtimesec
       sname=$_sname

   done < test.txt

   echo "Total time spent by each script:"
   echo
   for i in "${!namearr[@]}"
   do
       _gtime=${timearr[$i]}
       printf "${namearr[$i]} %d hours %d minutes %d seconds\n" $(($_gtime/3600)) $(($_gtime%3600/60)) $(($_gtime%60))
   done

Result:

 $ ./test
 Total time spent by each script:

 XYZ 4 hours 44 minutes 44 seconds
 ZEN 3 hours 28 minutes 34 seconds
 XLS 1 hours 36 minutes 35 seconds

bash script to get the spent time from a file

3 Answers3

EDIT