1

So timeout can be used to set the ultimate time limit on a process / command as mentioned here and here . For example, timeout 300 sleep 1000 will return to prompt after 300 seconds itself instead of 1000.

But is there any way to modify this limit on the fly while process is still running ? So this is what I am looking for. 

at time 0 : timeout 300 python long_run.py
at time 250 : <some way to extend the timeout limit by another 300 minutes or so>

I tried following two ways but couldn't make it work.

Through GDB

I tried to attach to timeout process with gdb. It showed following call stack but I couldn't find a variable whose value I could update to increase time limit. 

(gdb) where
#0  0x00007f10b49f6e8c in __libc_waitpid (pid=15753, stat_loc=0x7fff0c799f30, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
#1  0x00000000004022d8 in ?? ()
#2  0x00007f10b4643f45 in __libc_start_main (main=0x401fc0, argc=4, argv=0x7fff0c79a0e8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff0c79a0d8)
    at libc-start.c:287
#3  0x0000000000402479 in ?? () 
0  0x00007f10b49f6e8c in __libc_waitpid (pid=15753, stat_loc=0x7fff0c799f30, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
31      ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
(gdb) p *(stat_loc)
$2 = 4204240

Through /proc/

Is there anything we can do in /proc//limits or stat file to update timeout limit for timeout process or child process.

Community
  • 1
  • 1
ViFI
  • 971
  • 1
  • 11
  • 27
  • Do you _have_ to do it specifically via `timeout`? – ilkkachu May 10 '17 at 21:45
  • What necessitates extending the timeout? Why not just set the timeout for 550 seconds to begin with? – chepner May 10 '17 at 22:03
  • 1
    Put another way, what happens at time 250 that makes you decide to extend the timeout? You could schedule the longer timeout to start, then at time 250, decide whether to kill the process early, or let it continue to the original timeout. – chepner May 10 '17 at 22:13
  • 1
    Easiest thing to do may be to modify [the source code for timeout](http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/timeout.c). For example, you can add a `-i` option to specify an extension time, and trap the signal `SIGUSR1` to call a function that calls `timer_gettime` and `timer_settime` to extend the current timer value by adding the extension time to it. – Mark Plotnick May 10 '17 at 22:36
  • @chepner : I tried to reduce the scale of limits for easier understanding here but actual timeouts ( in our production environments) are very large , like in 10 hours. Sometimes what happens is we run it with 10 hours timeout overnight but when we come in the morning we want to debug a failure which happened overnight. But since 9:30 hours are already elapsed, it gives very small window to debug the issue. – ViFI May 10 '17 at 22:38
  • 1
    If that's all you need, you can just do `kill -STOP pid_of_timeout` Its child will continue running. – Mark Plotnick May 10 '17 at 22:44
  • @MarkPlotnick: `kill -STOP` seems to be doing the trick but leaves the parent timeout process forever even after child process finishes. So it seems a good workaround but I guess modifying the original source code to add '-i' option seems like a nice idea too. – ViFI May 10 '17 at 23:09
  • 1
    `kill -CONT pid` will continue a stopped process. – Mark Plotnick May 10 '17 at 23:12
  • @ViFI, so, if you expect that you might not want to terminate the program at the given time anyway, why set the timeout in the first place? Just kill it when you come in the next morning if you don't need it any more. Or use something other than `timeout` to kill it, say using `at` to schedule sending the signal, and then change the at job as required. – ilkkachu May 11 '17 at 11:11

1 Answers1

0

Converging the solution to question based on Mark Plotnick's much helpful comments.

Solution 1

kill -STOP stops a process and kill -CONT resumes it.

So kill -STOP pid_of_timeout will stop the timeout / timer running while child process will continue running. We can resume the counter when we want by kill -CONT pid_of_timeout

This means to add a delay of, say 50 in the timer, following will do the trick.

kill -STOP pid_of_timeout ; sleep 50 ; kill -CONT pid_of_timeout

Solution 2

Attach the timeout process to gdb. It will freeze timeout process. Detach from GDB when done.

Solution 3

Modify the source code of timeout, compile and run it instead of system "timeout" process.

Community
  • 1
  • 1
ViFI
  • 971
  • 1
  • 11
  • 27