App Engine socket invalid argument

Question

I am using PyAPNS to connect to APNs from Python. When running as a stand alone script, all is working fine and push messages get delivered. When running in the Google App Engine Development environment, I get the following error:

Traceback (most recent call last):
  File "/usr/local/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/usr/local/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/usr/local/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/usr/local/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/usr/local/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/usr/local/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/mnt/dev/OmegaHandler.py", line 173, in apns
    apns.gateway_server.send_notification("70f23022d76aae0176844087c97f0ff068dd3e3686dbac086b8f82a630d0196a", payload)
  File "/mnt/dev/apns.py", line 544, in send_notification
    self.write(self._get_notification(token_hex, payload))
  File "/mnt/dev/apns.py", line 273, in write
    return self._connection().write(string)
  File "/mnt/dev/apns.py", line 254, in _connection
    self._connect()
  File "/mnt/dev/apns.py", line 230, in _connect
    self._ssl = wrap_socket(self._socket, self.key_file, self.cert_file)
  File "/usr/local/lib/python2.7/ssl.py", line 911, in wrap_socket
    ciphers=ciphers)
  File "/usr/local/lib/python2.7/ssl.py", line 535, in __init__
    if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM:
  File "/mnt/dev/rsocket.py", line 225, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 22] Invalid argument

I used this answer to help the "Permission denied" error in the development environment. Again, when running stand alone, all is working fine using the exact same library and code. The error seems to come from getsockopt being called with options (1, 3). I have no clue anymore.

Why is this occurring in the development environment, monkey patched with the stock Python socket.py and not occurring in a stand alone script. Please do not answer that I need to enable billing in order to use the Socket API, this is only happening in the development environment. Thanks.

Edit: This is also happening on a different machine and with a different server (regular SSL socket connection to some server outside the Google IP address range).

Answer 1

Not sure if this is still useful or if you found answers elsewhere.

I had a pretty similar issue trying to use the requests library in pretty standard app engine code.

tl;dr - App Engine patches/provides a sandboxed socket. The ssl module snapshots objects from socket's namespace on its own import. If socket (and _socket) is not the correct version when ssl is imported, ssl will end up with GAE-patch/sandbox based values for SOL_SOCKET and SO_TYPE (probably incorrect), rather than what should have come from a base system installation. If you make sure your copy of socket (and _socket) is correct, ssl will work.

I filed a support ticket with Google and got push-back to implement this.

It's not actually that trivial for me to just swap out the physical socket.py implementation, so I went the route of this solution instead.

Which got me most of the way there, but the issue still cropped up because of SSL.

I have tested something practically identical to the below code to get SSL working with raw sockets (rather than URLFetch) when making HTTP requests in an App Engine development sandbox (i.e dev_appserver.py):

import os

def _patch_ssl_support():

    # Only patch SSL support if it's local dev.
    if not os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
        return

    import imp
    import inspect
    from google.appengine.tools.devappserver2.python import sandbox

    # Allow the sandbox to read _ssl and _socket.
    sandbox._WHITE_LIST_C_MODULES += ['_socket', '_ssl']

    # Use the system socket.
    # I used inspect here, but many core modules should work. 
    # It ultimately depends on your python installation.
    runtime_path = os.path.realpath(inspect.getsourcefile(inspect))
    runtime_dir = os.path.dirname(runtime_path)

    # Patch and reload the socket module implementation.
    system_socket = os.path.join(runtime_dir, 'socket.py')
    imp.load_source('socket', system_socket)

    # Patch and reload the ssl module implementation.
    system_ssl = os.path.join(runtime_dir, 'ssl.py')
    imp.load_source('ssl', system_ssl)

    # Patch and/or reload any other libraries you suspect may have copied values
    #   from the socket or ssl namespaces.

# Patch SSL support before you do anything else.
_patch_ssl_support()

import webapp2

# Setup app engine application, or something that runs in an app engine runtime:
app = webapp2.WSGIApplication(routes)