How to calculate average response time Bash script

Question

I have a bash script that is calculating response time for specific java methods from given log. The script is constituted by 2 loops, one is iterating through last 5 minutes for (( i = 5; i > 0 ; i--)) and identifies the "ID's" which I need for the 2nd loop so I can match the starting and ending times. At the second loop I'm doing the calculations diff = $(($endTime - $startTime)); echo $diff

Now I want to get the average response time from all results from last 5 minutes. This can be easily done with awk if I call my script ./responseTime.sh | awk '{sum+=$1} END {print sum / NR}' How can I calculate the avg without using this pipe (I don't know the number of results which I will get for every 5 minutes it runs)?

I do not have such log at my local machine to show you how it works. Here is my script

#!/bin/bash

logfile=/var/log/examples.txt

#loop through last 5 min

  for (( i = 5; i >=0; i-- ))

    do

     debugDate=$(date --date="$i minutes ago" "+%b%d %H:%M")
     folderId=$(grep "$debugDate" $logfile  | sed -rn '/.*folderId:([^ ]*).*/ s//\1/p' ) # identify id's and store results

      for j in $folderId

       do

        startTime=$(grep "$j" $logfile | grep "Starting" | awk '{split ($2,Time,":"); print Time[1]*3600000+Time[2]*60000+Time[3]*1000}') #get the value in ms
        endTime=$(grep "$j" $logfile | grep "process finished" | awk '{split ($2,Time,":"); print Time[1]*3600000+Time[2]*60000+Time[3]*1000}')

      if [ ${#startTime} -ne 8 ] || [ ${#endTime} -ne 8 ] #exclude midnight results and retries

      then

         continue

      fi

       responseTime=$(($endTime - $startTime))
       echo $responseTime #test results

      done

  done

cat examples.txt

[Jan12 18:26:00] Initialized session for folderId:8005
[Jan12 18:26:15] Starting process [8005]
[Jan12 18:27:00] Initialized session for folderId:8004
[Jan12 18:27:17] Starting process [8004]
[Jan12 18:28:00] Initialized session for folderId:8003
[Jan12 18:28:16] Starting process [8003]
[Jan12 18:29:00] Initialized session for folderId:8002
[Jan12 18:29:15] Starting process [8002]
[Jan12 18:30:00] Initialized session for folderId:8001
[Jan12 18:30:12] Starting process [8001]
[Jan12 18:31:00] Initialized session for folderId:8000
[Jan12 18:31:16] Starting process [8000]
[Jan12 18:26:31] process finished [8005]
[Jan12 18:27:41] process finished [8004]
[Jan12 18:28:55] process finished [8003]
[Jan12 18:29:49] process finished [8002]
[Jan12 18:30:33] process finished [8001]
[Jan12 18:31:35] process finished [8000]

Can you set up a minimum working example, so that anyone can copy, paste, try, edit, and possible answer your question? — Enlico, Jan 12 '20 at 12:32
@EnricoMariaDeAngelis added some more information hopefully it helps. — krisla, Jan 12 '20 at 14:04
Bash doesn't have floats but all of this is probably better written as a single Awk script. Without samples of your input, it is hard to know exactly how to refactor this. In particular, in what form and in which field(s) is the time stamp for each log entry? — tripleee, Jan 12 '20 at 14:36
I think it would be easier if I just ask how do I store all these results($responseTime) into an array, then I can get its number and calculate the avg. — krisla, Jan 12 '20 at 15:12
@KristianSlavov, you can also write a sample simple log file yourself and add it to the question. However, as @tripleee mentioned, `bash` is not the right tool if you have to deal with floating point numbers. `awk` is, and in principle it can accomplish the whole task. — Enlico, Jan 12 '20 at 16:03
You are using `awk` (and `sed` and ??? other subprocesses) on each line of input to calculate your result here. Your previous Q, all in `awk` is the way to go to solve this sort of problem. I see you don't need help in translating your `bash` code to `awk`, so better to post a Q with any problems you encounter using `awk`. Good luck. — shellter, Jan 12 '20 at 16:22
@KristianSlavov, does this [answer](https://stackoverflow.com/a/11088034/5825294) the question you asked in the comment? — Enlico, Jan 12 '20 at 16:23
@tripleee added examples txt please do keep in mind this is a probe not a real scenario. here I may not need the 2 loops, but the script works. You may have to increase the minutes counter since it's already later. — krisla, Jan 12 '20 at 17:17
@EnricoMariaDeAngelis yes, it did, but I still don't like my solution — krisla, Jan 15 '20 at 19:30

score 2 · Answer 1 · answered Jan 18 '20 at 18:04

For simple math, you can use bash built-ins. Note that bash does not have floating-point math, so you will get result in full second resolution.

sum=0
count=0
for (( i = 5; i >=0; i-- ))
do
   ...
   Original Loop Here
   responseTime=$(($endTime - $startTime))
   echo $responseTime #test results
   ...
   # Update sum and count
   sum=$((sum+responseTime))
   count=$((count+1))   
done
# Print Average
[ "$count" -gt 0 ] && echo "Average: $((sum/count))"

How to calculate average response time Bash script

1 Answers1