0

In order to operate a 3rd party device, I use python's win32com module to interact with the provided UI. This works as expected in general, but rarely (every 10k calls or so) the COM API throws the error -2147024888 (0x80070008) with the description not enough storage is available to process this command. This is very annoying, because it usually kills the device during the night and the running procedure needs to be repeated entirely.

The question now is, what actually is out of memory, because I am quite certainly not hitting any RAM limitations. Digging deeper into the problem, I found that I also cannot close and re-open the COM connection, since this results in error -2147024882 (0x8007000e) described as out of memory. However, this answer hints that actually no memory problem is involved. As another explaination, too many used handles are suggested here.

That idea would be supported by the observation, that I cannot fork threads anymore at this point. Calling start() on a threading.Thread I obtain

RuntimeError: can't start new thread

although only 4 threads are running according to threading.active_count(). At the same time, none of these problems are observed from a different process running python. Hence, I suspect that my process runs out of some ressource, but I don't know which. For the moment I see two options on how to proceed.

  • Option 1: Find out what ressource my process is running out of. However, I am not really an expert on debugging. What could I try to do, in order to understand better what is going on behind the scenes?

  • Option 2: Live with the problem and work around it. Error catching appears not to be sufficient in this case. As soon as the error occurs, there seems to be no way to get everything working again without killing the python process. Of course, this is not possible from within python.

    As a solution I thought about using the multiprocessing module and having all device-related calls running in a separate process. If something goes wrong, I kill the proces form my main programm and start it again with the possibility to continue at the point, where the error occured.

For both options I would be happy for suggestions or guidelines. Maybe there is also something entirely different, which I haven't thought about. Any help is appreciated.

Edit: Motivated by Ben's comment, I looked up some properties of the still running process, in which the errors occured. Here is a list of the data, which I consider as potentially useful:

  • 1.7G commit size with only 21M used and 500M peak memory (on Win7 x86)
  • 800k paging errors
  • 772 handles
  • 20 threads
Community
  • 1
  • 1
ranguwud
  • 285
  • 3
  • 12
  • Use Task Manager details or processes tab, and enable the optional columns. One of them is "handle count" another is "thread count", also see "commit charge" which is the memory allocation of the process. You will soon see what you are running out of. Performance Monitor will also help. – Ben Mar 11 '16 at 13:20
  • Thanks for the suggestion. I made an edit to include data, which I consider potentially relevant. – ranguwud Mar 11 '16 at 14:44
  • What about all the other processes? Any have high number of handles or threads or Commit Size? – Ben Mar 11 '16 at 14:46
  • Unfortunately I don't know what "high" means here. Every other process has not more than 2000 handles or 100 threads. The commit size of 1.7G of the process in question is by far the highest. The next smaller thrad only has 200M commit size.The only interesting, which I can read off, is that the process with 100 threads belongs to the python IDE (Spyder), from which my program is being launched. – ranguwud Mar 11 '16 at 15:05
  • How do those numbers change as you run the program? Do they gradually increase until crash? Or do they just stay at that number? – Ben Mar 11 '16 at 15:06
  • 1
    For the time being I don't know. Since the crash only occurs every 10k calls or so, I never witnessed it in real time but only saw the aftermath with the whole program being stuck. Also there is no obvious way to me, how to reproduce the crash. Any ideas? – ranguwud Mar 11 '16 at 15:54
  • Just look at the numbers while you run the program! Do they go up? – Ben Mar 11 '16 at 15:55
  • 1
    After my tests there is no coherent picture I can draw. Tonight I _did_ see a continuous increase in RAM usage, but today running the precisely same code again I _don't_. So my interpretatoin is that python's garbage collector just was somewhat lazy tonight. Also, when I attatch a memory profiler, I don't get any hints towards a memory leak. Hence, with very high probability, the problem must lie elsewhere. – ranguwud Mar 12 '16 at 12:38
  • I come up with nearly the same problem when using `os.system` or `subprocess`. It's so weird that every executing about 280 `popen` or `os.system` commands, my python script will give out this error. I have tried `gc.collect()` and `subprocess._cleanup()` but none of them worked. – tigertang Nov 14 '19 at 06:55
  • Plus, I never saw an increase in memory usage as the script went on and there were merely several subprocesses. – tigertang Nov 14 '19 at 06:57
  • 1
    @tigertang The problem described in my question does not exist anymore, but I never really found out what made it disappear. I *believe*, however, that the underlying cause was a variable initialized with the wrong type. I have written down my insights about this [here](https://stackoverflow.com/a/42255914/5804088). Whether this translates somehow to your `subprocess`, I cannot say. – ranguwud Nov 26 '19 at 10:04
  • @rangud I found the bug several days after commenting on this question. It's not relatable with `subprocess`. It is because a third-party library kept appending system paths until the path maximum length is reached. Damn, how can an error called "not enough storage is available" can be reported? It's so misleading! – tigertang Nov 26 '19 at 11:34

0 Answers0