1

I've created a python (and C, but the "controlling" part is Python) program for carrying out Bayesian inversion using Markov chain Monte Carlo methods. Unfortunately, McMC can take several days to run. Part of my research is in reducing the time, but we can only reduce so much.

I'm running it on a headless Centos 7 machine using nohup since maintaining a connection and receiving prints for several days is not practical. However, I would like to be able to check in on my program occasionally to see how many iterations it's done, how many proposals have been accepted, whether it's out of burn-in, etc.

Is there something I can use to interact with the python process to get this info?

Georgia S
  • 602
  • 4
  • 14
  • 3
    you might be interested in screen, or tmux – Jay Kominek Apr 21 '16 at 22:08
  • You can use a signal handler for eg `SIGUSR1` to cause the process to print some info. Checkout `mosh` as a "better ssh" too. With `mosh` you can easily keep your `tmux` or `screen` session open over an unreliable connection. – John La Rooy Apr 21 '16 at 22:23
  • you can [attach a debugger (`gdb`)](http://stackoverflow.com/q/32941251/4279) but it might be more convenient to provide a remote (`ssh`) (i)python REPL that runs in the context of your application (i.e., you can play with your program's object interactively), see [twisted manhole example](https://twistedmatrix.com/documents/current/conch/examples/) – jfs Apr 22 '16 at 19:43

1 Answers1

2

I would recommend SAWs (Scientific Application Web server). It creates a thread in your process to handle HTTP request. The variables of interest are returned to the client in JSON format .

For the variables managed by the python runtime, write them into a (JSON) file on the harddisk (or any shared memory) and use SimpleHTTPServer to serve it. The SAWs web interface on the client side can still be used as long as you follow the JSON format of SAWs.

gdlmx
  • 6,479
  • 1
  • 21
  • 39
  • That looks really neat, but unfortunately most of the info comes from the python part of the program. Do you know if there's something similar for python? – Georgia S Apr 22 '16 at 04:44
  • Then you can keep updating a file (maybe use [JSON](https://docs.python.org/2/library/json.html)) on the harddisk (or any shared memory) and use [SimpleHTTPServer](https://docs.python.org/2/library/simplehttpserver.html) to serve it. If you mimic the JSON format of SAWs, you can reuse its web application. – gdlmx Apr 22 '16 at 13:20
  • Thanks @gdlmx. I think I'll do this with flask since that's what I'm most familiar with. – Georgia S Apr 27 '16 at 20:20