12

I searched a lot about problem VMware with python, but I didn't find any information about my problem. My problem is that python programs freeze, process is still running but it doesn't use CPU and memory usage doesn't change. Program doesn't return any exception or anything... it just freeze and it never back to execution. It looks that it happens randomly and it is not a problem with no memory available for my machine, because in the same time I can execute it in another session.

My machine is virtual machine with: Windows Server 2008 64-bit, VMware Tools 9.4.5

I tried python: 2.7, 3.3 and 3.4

my example script: (but not only this script freeze)

print("START")
for i in range(0, 1000): 
  print("step: " + str(i)) 
  file = open("./test_file.csv", "r") #file size is 1.2GB but I have 10GB RAM
  for line in file.readlines(): 
    pass    
  file.close() #close the file
print("END")

example output is:

START
step 0
step 1
step 2
step 3
step 4

and it freeze, it is randomly on which step (sometimes 4, 15, 34...) All what I can do is kill the process and run it again. During the execution I can see that program repeatable use 1,2GB RAM and release... use and release. Freeze is always after release memory and from this time memory usage is stable and CPU usage 0% for this process.

I run the script in IDLE to play with debugger and stack viewer, but when the program freeze, the whole idle is not responding. Also I tried it on others no-virtual machine and there is no problem.

I would be grateful for any suggestions, how to solve or debug that kind o problems.

Alex
  • 411
  • 5
  • 12
  • You mention that you have 10GB of RAM, but... did you mean that the host machine has 10GB of physical RAM, or you have allocated 10GB of RAM to your VM? It sure sounds like your VM is running out of memory. – evadeflow Jul 04 '14 at 12:35
  • Same solution here - https://serverfault.com/questions/204150/sometimes-powershell-stops-sending-output-until-i-press-enter-why – rdmolony Apr 29 '22 at 09:07

2 Answers2

27

I solved it, the problem was not with python or VMware.... only with my knowledge about Microsoft products.

I did not mentioned that I execute python scripts using windows command line, and the reason of the "freeze" was cmd. I had no idea that cmd suspends executed task if you mark sth in command line.

Only on this virtual machine I had set up “QuickEdit Mode”. You can change it following steps: right click on panel of cmd -> properties -> tab: “Options” -> section: “QuickEdit Mode”)

In this mode if you click on command line it will mark sth and... suspend executed task… I turned off the “QuickEdit Mode” and program works perfectly. Without QuickEdit Mode you can not mark anything in command line by clicking.

I know... I should shame myself... and I do.

Alex
  • 411
  • 5
  • 12
  • 4
    Oh god, thank you! I've spent a lot of time trying to find a problem in a script or an interpreter. Should've googled it earlier... Problem was absolutely the same. Python freezes in the middle of nowhere without any signs of it and continues if I pressed any key in cmd. – Serge Norin Jun 16 '17 at 09:28
  • 1
    OMG! I had exactly the same issue running uvicorn web server. Sometimes it just freezes and the server did not accept requests. Every requests finished with timeout with any logs from the web server or the program running it. Disabling QuickEditMode fix the issue. Thanks! – hpeev Mar 24 '21 at 10:05
  • 1
    You shouldn't shame your self, Microsoft should... This is a bad design for the cmd. You saved my time/life, I would have to suffer trough a painful search for the real reason of the freeze, to find this type of "feature", which cause unintended problems for the users. – Károly Szabó Jan 27 '22 at 09:27
0

Propably you should use readline() not readlines() that reads the whole file.

PeterMmm
  • 24,152
  • 13
  • 73
  • 111
  • I used readlines(), because it load whole file to memory and I can see what is happening with memory ;-) When I changed it to readline() and range of iteration it also freeze after 1 100 steps ;-) – Alex Jul 04 '14 at 10:38