3

I want to have a python script running on my Ubuntu instance constantly. It's an AWS EC2 a1.medium instance and the script is taking pretty much 100% of the CPU constantly.

I start the script with screen <run script>, however it gets killed randomly and I have no idea why. One time I had it running in the foreground and I saw "Killed" as the last output of the process.

I tried running it as root with sudo screen <run script> but I get [screen is terminating]. Not sure if root would even help though.

Is there anything I can do to troubleshoot this? I tried with -L -Logfile <filename> with proper permissions for the file, but it stays empty, so I really have no idea how to tell what went wrong.

Nikolay Dyankov
  • 6,491
  • 11
  • 58
  • 79

1 Answers1

1

What is nature of your script? Is it CPU/Memory bound ? I am assuming you are over utilising vCPUs provide in a1.medium EC2 instance

You can also take a look at this to understand steal time

Shakeel
  • 1,869
  • 15
  • 23
  • The script consumes a web API, about 1MB/sec worth of data, processes and writes a small amount of data in a postgres database. I picked the A1 family of instances because they don't have a credit system and you can use the vCPU as much as you like. – Nikolay Dyankov Mar 28 '21 at 00:57
  • Okay so its I/O, can you please share script so that can make an attempt to handle errors or can take a look if something can be done to catch that errors ? – Shakeel Mar 28 '21 at 01:12
  • Are you running consumer script under `while True` ? – Shakeel Mar 28 '21 at 01:36
  • Yes, but I didn't have any issues on my machine with that script, it could run for days without interrupting. – Nikolay Dyankov Mar 28 '21 at 03:01
  • Maybe I should look into this - http://upstart.ubuntu.com/ - "Services may be respawned if they die unexpectedly" but it looks like a bandaid rather than fixing the issue. – Nikolay Dyankov Mar 28 '21 at 03:07
  • I made you a pastebin of the script if you want to take a look - https://pastebin.com/WuSJ0LvL – Nikolay Dyankov Mar 28 '21 at 04:59
  • Thanks for sharing clean readable code. I suspect there is some exceptions are raised in threads. Can you please take look at this https://stackoverflow.com/a/53779560/9592801 plan to handle exceptions and give a try ? – Shakeel Mar 28 '21 at 12:23