2

I'm using get_all_values() function which some times throws an error 

content = worksheet.get_all_values()
File "/usr/local/lib/python2.7/dist-packages/gspread/models.py", line 339, in get_all_values
cells = self._fetch_cells()
File "/usr/local/lib/python2.7/dist-packages/gspread/models.py", line 226, in _fetch_cells
feed = self.client.get_cells_feed(self)
File "/usr/local/lib/python2.7/dist-packages/gspread/client.py", line 251, in get_cells_feed
return ElementTree.fromstring(r.read())
File "/usr/lib/python2.7/httplib.py", line 541, in read
return self._read_chunked(amt)
File "/usr/lib/python2.7/httplib.py", line 590, in _read_chunked
value.append(self._safe_read(chunk_left))
File "/usr/lib/python2.7/httplib.py", line 649, in _safe_read
raise IncompleteRead(''.join(s), amt)
IncompleteRead: IncompleteRead(16360 bytes read, 26 more expected)

I don't know why this happens only some times. can some one explain all the possible cases so that I can handle this.

I have read here https://stackoverflow.com/a/25214394/3981589 looping over it for n number of trails does that mean it might be problem with connectivity problem or some thing else.

Community
  • 1
  • 1
ck reddy
  • 494
  • 7
  • 17
  • I'm guessing that is not the actual error, please include the full error message and traceback and maybe some code that can produce this error (especially if only some of the time) – Tadhg McDonald-Jensen Jun 01 '16 at 14:13
  • @TadhgMcDonald-Jensen can you check now – ck reddy Jun 01 '16 at 14:46
  • How long does it take to pull all of the values? I query a sheet every second at all times of the day and there are plenty of connectivity errors thrown by Google API in addition to expiry of OAuth. – roganjosh Jun 01 '16 at 17:59
  • @roganjosh on an average it is taking 5 minutes to read all values – ck reddy Jun 02 '16 at 04:40
  • @ckreddy I can't give a definitive answer on this, but if it's intermittent and requires 5 minutes of continuous connection, it could well be a 500 error from Google that is being masked. If that is the case then you either have to scrap the download for 30 seconds and start again or maybe you can read column-by-column and rebuild the sheet locally, with a 30 second hang on any error. – roganjosh Jun 02 '16 at 08:52
  • 1
    You could try to connect with a keep-alive connection: http://stackoverflow.com/questions/36786253/gspread-keep-alive-after-update-google-auth2 – Jacques Gaudin Jun 02 '16 at 09:29
  • @JacquesGaudin not at all my area of expertise, how does `keep-alive` handle something like an internal server error? Currently I catch them with `except gspread.exceptions.HTTPError:` because I do single reads, but if I were to get a server error during a large pull of data, will it just hang and then resume? – roganjosh Jun 05 '16 at 16:25
  • 1
    @roganjosh: I am not 100% sure to be honest but I thought it would be worth giving it a try. What I can tell from the error message is that the `gspread` code has changed since. It now uses the `requests` module to handle the `GET` request and the line `return ElementTree.fromstring(r.read())` was changed for `return ElementTree.fromstring(r.content)`. I would suggest to update to the latest version and see if the problem persists. If it does, `requests` provides a `raiseforstatus` method that would tell us what error the server is returning. – Jacques Gaudin Jun 05 '16 at 19:53
  • @JacquesGaudin are you active on this then? I got the impression that `gspread` was not developed anymore. If you are developing this then I will try and capture it and give you any feedback I can. It's a 500 code with a _huge_ apology message from Google. We have a niche use of the library that makes literally thousands of reads a day from a single sheet. – roganjosh Jun 05 '16 at 20:08
  • @JacquesGaudin I misunderstood 'us' sorry. Yes, OP runs a different version from me too. I will set up a test case if I can for your suggestion as handling Google API fails is clumsy for me when you're trying different things. – roganjosh Jun 05 '16 at 20:25
  • @roganjosh: The last update of gspread dates back to Oct 2015 which is not so long ago. I am not active as such on the project but the size of the code is small enough that you can understand what it is doing and maybe try to patch it. If the size of the response is an issue, it could be broken down into smaller chunks with the help of parameters passed to the `GET` request in `Client.get_cell_feed`. – Jacques Gaudin Jun 05 '16 at 20:47

0 Answers0