3

I have a task that is very well inside of a bash for loop. The situation is though, that a few of the iterations seem to not terminate. What I'm looking for is a way to introduce a timeout that if that iteration of command hasn't terminated after e.g. two hours it will terminate, and move on to the next iteration.

Rough outline:

for somecondition; do
   while time-run(command) < 2h do
     continue command
   done
done
user857990
  • 1,140
  • 3
  • 14
  • 29
  • Your system may have a `timeout` command that can be used to impose a time limit on a process. – chepner Sep 22 '13 at 20:07
  • A few options here that you might be able to integrate: http://stackoverflow.com/questions/5161193/bash-script-that-kills-a-child-process-after-a-given-timeout – Kyle Sep 23 '13 at 03:19

2 Answers2

3

One (tedious) way is to start the process in the background, then start another background process that attempts to kill the first one after a fixed timeout.

timeout=7200   # two hours, in seconds
for somecondition; do
    command & command_pid=$!
    ( sleep $timeout & wait; kill $command_pid 2>/dev/null) &  sleep_pid=$!
    wait $command_pid
    kill $sleep_pid 2>/dev/null   # If command completes prior to the timeout

done

The wait command blocks until the original command completes, whether naturally or because it was killed after the sleep completes. The wait immediately after sleep is used in case the user tries to interrupt the process, since sleep ignores most signals, but wait is interruptible.

chepner
  • 497,756
  • 71
  • 530
  • 681
0

If I'm understanding your requirement properly, you have a process that needs to run, but you want to make sure that if it gets stuck it moves on, right? I don't know if this will fully help you out, but here is something I wrote a while back to do something similar (I've since improved this a bit, but I only have access to a gist at present, I'll update with the better version later).

#!/bin/bash

######################################################
# Program:      logGen.sh
# Date Created: 22 Aug 2012
# Description:  parses logs in real time into daily error files
# Date Updated: N/A
# Developer:    @DarrellFX
######################################################
#Prefix for pid file
pidPrefix="logGen"
#output direcory
outDir="/opt/Redacted/logs/allerrors"
#Simple function to see if running on primary
checkPrime ()
{
  if /sbin/ifconfig eth0:0|/bin/grep -wq inet;then isPrime=1;else isPrime=0;fi
}


#function to kill previous instances of this script
killScript ()
{
  /usr/bin/find /var/run -name "${pidPrefix}.*.pid" |while read pidFile;do
    if [[  "${pidFile}" != "/var/run/${pidPrefix}.${$}.pid" ]];then
      /bin/kill -- -$(/bin/cat ${pidFile})
      /bin/rm ${pidFile}
    fi
  done
}


#Check to see if primary
#If so, kill any previous instance and start log parsing
#If not, just kill leftover running processes


checkPrime
if [[ "${isPrime}" -eq 1 ]];then
  echo "$$" > /var/run/${pidPrefix}.$$.pid
  killScript
  commands && commands && commands #Where the actual command to run goes. 
else
  killScript
  exit 0
fi

I then set this script to run on cron every hour. Every time the script is run, it

  • creates a lock file named after a variable that describes the script that contains the pid of that instance of the script
  • calls the function killScript which:
    • uses the find command to find all lock files for that version of the script (this lets more than one of these scripts be set to run in cron at once, for different tasks). For each file it finds, it kills the processes of that lock file and removes the lock file (it automatically checks that it's not killing itself)
  • Starts doing whatever it is I need to run and not get stuck (I've omitted that as it's hideous bash string manipulation that I've since redone in python).

If this doesn't get you squared let me know.

A few notes:

  • the checkPrime function is poorly done, and should either return a status, or just exit the script itself
  • there are better ways to create lock files and be safe about it, but this has worked for me thus far (famous last words)
drldcsta
  • 413
  • 3
  • 8