Python 3 intermittent ssl.SSLEOFError

Question

I'm doing a fetch from google sheets using pygsheets python module every 90 secs.

During early hours of morning (usually between 2-3 AM) this operation fails, and I get this error logged:

Traceback (most recent call last):
  File "/etc/naemon/naemon-automation/exec/pull-GSheets-CSV.py", line 42, in <module>
    wks.export(pygsheets.ExportType.CSV, path=outputDir + '/', filename=outputFileName)
  File "/usr/local/lib/python3.5/dist-packages/pygsheets/worksheet.py", line 1306, in export
    self.client.drive.export(self, file_format=file_format, filename=filename, path=path)
  File "/usr/local/lib/python3.5/dist-packages/pygsheets/drive.py", line 210, in export
    status, done = downloader.next_chunk()
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/http.py", line 686, in next_chunk
    'GET', headers=headers)
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/http.py", line 183, in _retry_request
    raise exception
  File "/usr/local/lib/python3.5/dist-packages/googleapiclient/http.py", line 164, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/google_auth_httplib2.py", line 198, in request
    uri, method, body=body, headers=request_headers, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1926, in request
    cachekey,
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1595, in _request
    conn, request_uri, method, body, headers
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1501, in _conn_request
    conn.connect()
  File "/usr/local/lib/python3.5/dist-packages/httplib2/__init__.py", line 1291, in connect
    self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
  File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 752, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:645)

My code:

import pygsheets
import sys
from pathlib import Path

# Import Arguments required for generating Configuration. If one is not set - Do not continue
# str(sys.argv[1]) # Argument denoting the Google Service Account API Authentication File
# str(sys.argv[2]) # Argument denoting the Google Sheets URL Key found when logging into the Google Sheets Manually via browser
# str(sys.argv[3]) # Argument denoting the Google Worksheet Name, for unique identification within the Google Spreadsheet
# str(sys.argv[4]) # Argument denoting the File Name and output destination

# Variable Definitions
try:
        gServiceAccAuthFile = str(sys.argv[1])
        gSheetKey = str(sys.argv[2])
        gWorksheetName = str(sys.argv[3])
        outputFile = str(sys.argv[4])
        outputDir = str(Path(outputFile).parents[0])
        outputFileName = str(Path(outputFile).stem)
except IndexError:
        print('Not enough Arguments Specified, Required Arguments:\nPOS 1.)\tGoogle Service Account API Authentication File\nPOS 2.)\tGoogle Sheets URL Key\nPOS 3.)\tGoogle Worksheet Name\nPOS 4.)\tOutput File Path & Name')
        sys.exit()

# Authorize Spreadsheet Access
gc = pygsheets.authorize(service_file=gServiceAccAuthFile, retries=1)
# Open spreadsheet
sh = gc.open_by_key(gSheetKey)
# Open Worksheet
wks = sh.worksheet_by_title(gWorksheetName)
# Export as CSV
wks.export(pygsheets.ExportType.CSV, path=outputDir + '/', filename=outputFileName)

Proposed solutions:

Issue with SSL Module: Update to newer binary?
Try / except: What would be the except statement? except ssl.SSLEOFError?
Pygsheets: Is there a wks.export() retry function?

AKX · Accepted Answer · 2019-07-11T10:06:17.077

1

Try / except: What would be the except statement? except ssl.SSLEOFError?
Pygsheets: Is there a wks.export() retry function?

Combining these – I'm using logging for logging, but adapt as you like.

import ssl
import logging

log = logging.getLogger(...)

for attempt in range(1, 6):  # Try at most 5 times
   try:
       wks.export(pygsheets.ExportType.CSV, path=outputDir + '/', filename=outputFileName)
   except ssl.SSLError as e:
       log.warning('Attempt %d to export sheet failed: %s' % (attempt, e), exc_info=True)
   else:
       break  # success!
else:  # executed if we didn't `break` out
    raise RuntimeError('All attempts to export the sheet failed!')

edited Jul 11 '19 at 10:06

answered Jul 11 '19 at 09:26

AKX

152,115
15
115
172

Thanks! - I've noticed that you've imported the `ssl` module - whereas I think I've had pygsheets do that for me. Is this necessary to raise the exception? Or can I get around it by calling `pygsheets.ssl.SSLError` instead? – itChi Jul 11 '19 at 09:42
1

You will need to `import ssl` in your module to bring it to your namespace to catch the error. It's still the same module. There likely is no `pygsheets.ssl` import available anyway, since if you look at the traceback, it's `googleapiclient`, then `httplib2` eventually using the SSL module. – AKX Jul 11 '19 at 09:44
quick one - where did `s` come from from `(attempt, s)`? – itChi Jul 11 '19 at 10:03
1

Oops, typo. Meant it to be `e`, the exception. – AKX Jul 11 '19 at 10:06

Python 3 intermittent ssl.SSLEOFError

1 Answers1