0

I have a python program running on my university machine and I realized that its going to take more time than I expected. One loop of the code prints some output. I want to save this output in a file and also send the currently running program in the background as i need to logout from the system.

This(How do I put an already-running process under nohup?) explains how can I send it to background. Is there any way by which I can also save the output that is being shown on the screen every second, for eg by using tee

Community
  • 1
  • 1
Abhishek Thakur
  • 16,337
  • 15
  • 66
  • 97
  • Redirect the output to a file and use `tail -f` to read the file? I don't know if you can redirect the output after the program has started though. – rm5248 Nov 06 '13 at 14:20
  • I think it is technically possible, but it involves using a debugger to modify the open file handles of the running process. Not for the faint of heart, and probably more trouble than it is worth. – chepner Nov 06 '13 at 14:32

2 Answers2

2

I've not tried it, but this evil hack might work.

  1. Find what file descriptors the program is using:

    ls -l /proc/<your-pid>/fd

  2. Attach a debugger to the process (the executable is something like /usr/bin/python):

    gdb --pid=<your-pid> <executable>

  3. Open the new file you want (you may have to look up the numeric values of O_CREAT and O_WRONLY):

    (gdb) set $newfd = open("myfile", O_CREAT | O_WRONLY)

  4. Replace the old file descriptor you found in step 1 with the new one.

    (gdb) call dup2(<old-fd>, $newfd)
(gdb) call close($newfd)

  5. Set the program running again and disconnect the debugger

    (gdb) detach
(gdb) quit

This will only work if the open and dup2 functions are present in the program.

Warning: there is some danger this could crash your program!

ams
  • 24,923
  • 4
  • 54
  • 75
1

If the program has already started, it's a matter of hacking (see below) to redirect its output. Instead, you can/should take precautionary measures beforehand.

I found it very useful to always run remote commands in a screen session. This allows you to:

  1. attach and detach to the session at will, and without disturbing the running programs. This means you can detach the session, logout, then login later and find all your "windows" and running programs as you left them (or, hopefully, finished computing). This also works when your ssh connection is dropped unexpectedly, or if you need to attach from multiple places at the same time (e.g. from your own laptop/machine via ssh, and directly from the university machine's terminal);
  2. have multiple (text mode) "windows" running on the same ssh connection, and switch between them (and their output) via keyboard shortcuts;
  3. have a scrollback buffer for each "window", so that even if you forgot to use tee, you can scroll up and see what the program has printed;
  4. set a session password, so that only you can reattach to it.

About the hacking part:

If you are stuck running a very long and important program, and can't restart it to use the method described above, there is still hope.

The open file descriptors for each process (including its current stdout) can be found via the /proc filesystem. Using that, you can swap a file descriptor out and have the command switch to outputting somewhere else on the fly. This script (fdswap.sh) uses gdb to swap out a file descriptor from under a running process, and there's an article explaining how to use it.

In short: find the pid of the running process, then run ls -l /proc/$pid/fd/0 to find where its current output goes, then use the above script to swap it out: fdswap.sh /dev/pts/$vty /path/to/new/output $pid.

Historical reference: http://www.phrack.org/issues.html?issue=51&id=5

Sir Athos
  • 9,403
  • 2
  • 22
  • 23