0

I have a python script, that calls HandBrakeCli as a subprocess on Linux. During normal operations, HandBrakeCli takes almost 100% of available CPU resources. But sometimes it seems to starve or whatever - it seems not to be active, but does not return for a long time. This seems to be due to a broken DVD (Handbrake is a dvd ripping tool) - it seems that Handbrake is not handling this so well. It will finish after a long while (hours), but bevor it does nothing. Whereas in normal operation it will use lots of cpu cycles.

So my idea to solve this was to have a watch dog that checks if there is a HandBrakeCli process, and if so monitors if it does use a good part of the cpu. If that is not the case for maybe 5 minutes, it will kill that process, so that the parent script can continue its operations.

It does not seem too hard to program this, but I have a feeling it could involve some tedium. Also it seems not unlikely that this problem has been solved before. Is there a solution for this, possibly in python? If there is a standalone program or daemone that does the job on Linux I would not mind if its not python, as long as it is open source.

Isaac
  • 810
  • 2
  • 13
  • 31
  • If you want to do it manually, this can be your help: http://stackoverflow.com/questions/1420426/calculating-cpu-usage-of-a-process-in-linux – Michał Fita Nov 26 '15 at 16:51

1 Answers1

0

Well perhaps the most obvious answer for something to monitor and control child process in python would have to be supervisord. It's very mature, and has a lot of features and documentation.

It is written in Python, and has a plugin API, as well as a service API and a web dashboard.

Your problem description is a little bit vague though, re 'starvation'. It does sound as if your job is blocking on some resource, but you might want to diagnose this a little more precisely before reaching for a complicated software scaffolding to prop it up.

i.e. perhaps it's blocked on I/O (network or disk bandwidth), or it's stalled reading from a pipeline that's failed. You might want to look into process control (e.g. the renice command ) to manage it's CPU allocation a little better etc.

cms
  • 5,864
  • 2
  • 28
  • 31
  • I already looked at supervisord - it seems that it would monitor for *failed* processes - whereas mine is not in a failed state, it is just not doing much anymore. The only way it is noticable to me is that it is not using almost 100% cpu anymore. I have updated the question with a more verbose description of what I call "starved". - probably the wrong term. – Isaac Nov 26 '15 at 18:17