2

This question has been asked before but it was two years ago and I want to know that has there been any update regarding this in python 2.7.11?
Basically my program hangs for several hours and when i do a keyboard interrupt i get this error:

    File "/usr/lib/python2.7/ssl.py", line 579, in __init__
      self.do_handshake()
    File "/usr/lib/python2.7/ssl.py", line 808, in do_handshake

  self._sslobj.do_handshake()

Does anyone has any idea why is this happening? As self-answered by Matt Vukas in the fore-mentioned thread, I scanned this thread but i don't get what's the error is about in the first place. I started python a couple of weeks ago and this is all new to me.

As as far as my memory serves me, it's not an exception so try/catch is out of question.
One more thing I'd like to mention (which may or may not matter) is that there is no specific pattern for this, sometimes it happens after days. Sometime just 10 minutes after I've restarted the program. URL changes after every call.

Here's the code:

username = "foobar"
password = "123456"
url = http://api.example.com
try:
   r = requests.get(url, auth=(username, password))
except requests.exceptions.ConnectionError:
   time.sleep(2)
   self.requests_exception_error(url)

EDIT: As asked by @Sasha Pachev here's the strace.out file (Last entry to database is at line #128 And last print statement is at line #285) The program froze at line #430.

When the program freezes the python deamon ends automatically

Community
  • 1
  • 1
Jarwin
  • 1,045
  • 1
  • 9
  • 30
  • Log the URLs before you issue the request and see if you can isolate the problem to a particular URL. Once you get there, you might be able to create a minimal, complete, and verifiable example, and we can go from there. – Sasha Pachev Dec 09 '15 at 19:43
  • @Sasha Pachev I did logged the variable and even made a `curl` call. As I mentioned in the question the "problem" is completely random. And I am pretty sure the problem is not with the URL (The URLs are read from file and are saved in MySQL db and I resume the program from the same URL with with the execution froze) Also, the program make around a thousand calls every couple of hour (in case it's related) – Jarwin Dec 09 '15 at 19:51
  • Use `strace -tt -f -o /tmp/strace.out -s 1024 -p $(pidof your_daemon)` , make sure /tmp or wherever you log has enough disk space, replace `your_daemon` with the actual name of the process or the whole thing with just the `pid`, collect the data, then record the time when you run into trouble and peruse the `strace` log around that time. If the problem does not become apparent post it where we can read it. – Sasha Pachev Dec 09 '15 at 19:56
  • @SashaPachev Since I am running the python script, I'll have to replace `your_daemon` with Python's pid, right? – Jarwin Dec 09 '15 at 20:42
  • 1
    The most reliable method would be to determine the pid of python with `top` or `ps` and then just use `-p the_pid_you_have_found`. `-p $(pidof python)` might be ambiguous if you have another instance of python running. – Sasha Pachev Dec 09 '15 at 21:01
  • @Sasha Pachev After I attached the process I was monitoring it via `tail -f` when I noticed that my API keys are DB password are being displayed(as plain ASCII). I am not sure if it will be safe to post the log file. Is there any alternative for this? – Jarwin Dec 09 '15 at 21:13
  • @Sasha Pachev additionally, with the rate the size of the file is increasing, and with the randomness of the specified "problem", in the worst case scenario, the size of the file could be in the magnitude of *gigabytes* the next time problem occurs(which could be days from now) and uploading the file would be out of option. – Jarwin Dec 09 '15 at 21:21
  • The problem window would be restricted to a certain timestamp, and the `-tt` option will mark the trace so you can isolate to a certain window. You can use a combination of `head`, `tail`, and `grep` to figure out the lines and extract the window, and then `sed` (or other tools) to edit out the sensitive information. `strace` has a `-e` option which you can use to filter out obviously irrelevant calls reducing the size as well. – Sasha Pachev Dec 09 '15 at 21:47
  • @Sasha Pachev As I read your last comment I went to create the log file by the command you provided, but the program was already frozen. And when I tried to grep python's id from` top`. Python *wasn't* running? Here, take a look: http://i.imgur.com/VEz98b4.png?1 If I am not mistaken I should probably update the question... – Jarwin Dec 09 '15 at 22:28
  • @SashaPachev I have updated the question – Jarwin Dec 10 '15 at 20:11

1 Answers1

0

What I see from strace is that at some point either after or during the SSL negotiation, the client is waiting for a reply from the remote host and it never comes. There can be numerous reasons for that and it is hard to tell without further troubleshooting. Next steps:

  • Collect packet traffic on port 443 into a PCAP file while your script is running (eg with tcpdump -s 65535 -w /tmp/port443.pcap port 443)
  • Decode it using ssldump -r /tmp/port443.pcap -d
  • Post the relevant part of the output
Sasha Pachev
  • 5,162
  • 3
  • 20
  • 20