1

I'm on version 1.9.9 of the SDK and I'm having issues with the devserver. I have a manually scaled module with 1 instance. I created a webapp2.RequestHandler for /_ah/start. In that handler I start a background thread. When I run my app in the devserver, the _ah/start handler returns a 200, but /_ah/background will randomly return 500 errors for a while. After sometime (usually a minute or two, but sometimes more), the 500 errors stop, but will randomly occur again every few hours. It also seems that everytime I open a new browser tab (Chrome), I get the same error. Anyone know what could be causing this?

Here is the RequestHandler for /_ah/start:

class StartupHandler(webapp2.RequestHandler):
    def get(self):
        runtime.set_shutdown_hook(shutdown_hook)

        global foo
        if foo is None:
            foo = Foo()

        background_thread.start_new_background_thread(do_foo, [])

        self.response.http_status_message(200)

Here is the 500 error:

ERROR    2014-08-18 07:39:36,256 module.py:717] Request to '/_ah/background' failed
Traceback (most recent call last):
  File "\appengine\tools\devappserver2\module.py", line 694, in _handle_request
    environ, wrapped_start_response)
  File "\appengine\tools\devappserver2\request_rewriter.py", line 311, in _rewriter_middleware
     response_body = iter(application(environ, wrapped_start_response))
  File "\appengine\tools\devappserver2\module.py", line 1672, in _handle_script_request
    request_type)
  File "\appengine\tools\devappserver2\module.py", line 1624, in _handle_instance_request
    request_id, request_type)
  File "\appengine\tools\devappserver2\instance.py", line 382, in handle
    request_type))
  File "\appengine\tools\devappserver2\http_proxy.py", line 190, in handle
    response = connection.getresponse()
  File "E:\Programing\Python27\lib\httplib.py", line 1030, in getresponse
    response.begin()
  File "E:\Programing\Python27\lib\httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "E:\Programing\Python27\lib\httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "E:\Programing\Python27\lib\socket.py", line 430, in readline
    data = recv(1)
error: [Errno 10054] An existing connection was forcibly closed by the remote host
INFO     2014-08-18 07:39:36,257 module.py:1890] Waiting for instances to restart
INFO     2014-08-18 07:39:36,262 module.py:642] lease: "GET /_ah/background HTTP/1.1" 500 -
Harshal Patil
  • 6,659
  • 8
  • 41
  • 57
Eliezer
  • 7,209
  • 12
  • 56
  • 103

2 Answers2

2

Well this might be not the answer , but how long will it take to complete a specific task to assign to a backend? Seems like an issue with concurrency

not 0x12
  • 19,360
  • 22
  • 67
  • 133
  • 1
    From what I understand `Queue.put` and `Queue.get` have a locking mechanism built in. I've discovered one of the issues, but that led to another issue. I'm in middle of editing the question. – Eliezer Aug 18 '14 at 05:03
  • That is fine , but from my prior experience it is not advisable to put heavy load to back ends and the task queue because they might behave strangely and um not quit sure we have a control over the locking mechanism – not 0x12 Aug 18 '14 at 06:39
1

Looks like the issue (as far as I can currently tell) is that I'm using PyCharm, which synchronizes the project's files when its window is entered or exited. This rewrites the project files even if there are no changes, which causes the devserver to restart all instances, leading to the 500 errors.

More info on PyCharm Synchronization

Link to issue at PyCharm

Eliezer
  • 7,209
  • 12
  • 56
  • 103