GCloud Upload httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header

Question

I am attempting to upload a small file to gcloud using a simple python program

client = storage.Client(project=GCLOUD_PROJECT)
bucket = client.get_bucket(GCLOUD_BUCKET)
blob = bucket.blob(GCLOUD_FILE_ON_CLOUD)
blob.upload_from_filename(GCLOUD_FILE_LOCAL)

It had been working until recently and something changed. Now, whenever I upload a file greater than 5MB I get the below error. Files less than or equal to 5MB goes through. The size isn't large enough to implement resumable upload, is it?

Traceback (most recent call last):
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/gcloud_upload.py", line 40, in <module>
    blob.upload_from_filename(GCLOUD_FILE_LOCAL)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/storage/blob.py", line 597, in upload_from_filename
    encryption_key=encryption_key, client=client)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/storage/blob.py", line 543, in upload_from_file
    http_response = upload.stream_file(use_chunks=True)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/transfer.py", line 1086, in stream_file
    response = send_func(self.stream.tell())
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/transfer.py", line 1215, in _send_chunk
    return self._send_media_request(request, end)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/transfer.py", line 1125, in _send_media_request
    self.bytes_http, request, retries=self.num_retries)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/http_wrapper.py", line 423, in make_api_request
    check_response_func=check_response_func)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/gcloud/streaming/http_wrapper.py", line 371, in _make_api_request_no_retry
    redirections=redirections, connection_type=connection_type)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/httplib2/__init__.py", line 1986, in request
    cachekey,
  File "/Users/mmumshad/PycharmProjects/quiz-python-flask-angular/venv36/lib/python3.6/site-packages/httplib2/__init__.py", line 1685, in _request
    content,
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.

When I debug I see the below.

{
 'content-type': 'text/plain; charset=utf-8', 
 'range': 'bytes=0-1048575', 
 'content-length': '0', 
 'date': 'Sun, 19 Jan 2020 23:52:13 GMT', 
 'server': 'UploadServer', 
 'alt-svc': 'quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000', 
 'status': '308'}

My pip list for reference. OS is MAC OSx. I tested on Linux as well. Same issue.

adal                     1.2.2    
bson                     0.5.8    
cachetools               3.1.1    
certifi                  2019.9.11
cffi                     1.13.1   
chardet                  3.0.4    
cryptography             2.8      
dnspython                1.16.0   
gcloud                   0.18.3   
gitdb2                   2.0.6    
GitPython                3.0.5    
google-auth              1.6.3    
googleapis-common-protos 1.51.0   
httplib2                 0.16.0   
idna                     2.7      
itsdangerous             1.1.0    
MarkupSafe               1.1.1    
oauth2client             4.1.3    
oauthlib                 3.1.0    
pip                      19.0.3   
protobuf                 3.11.2   
pyasn1                   0.4.7    
pyasn1-modules           0.2.7    
pycparser                2.19     
PyJWT                    1.7.1    
python-dateutil          2.8.0    
requests                 2.19.1   
requests-oauthlib        1.2.0    
rsa                      4.0      
setuptools               40.8.0   
six                      1.12.0   
smmap2                   2.0.5    
urllib3                  1.23     
websocket-client         0.56.0   
Werkzeug                 0.16.0   

This was working last week. Has something changed recently?

Answer 1

I solved this problem with:

pip install httplib2==0.15.0

pip install google-api-python-client==1.6

Edit: faster on load is: pip install httplib2==0.15.0 pip install google-api-python-client==1.7.11

Answer 2

Solutions

gcloud package is deprecated twice and is not compatible with httplib2>=0.16. Proper solution is to use google-cloud-* packages family.

google-api-python-client>=1.7.12 is using redirect_codes API, please upgrade, it just works.

httplib2 v0.17.0 is just released with ability to modify set of response codes treated as redirects. This is the best option if you can modify code that creates Http object:

http = httplib2.Http()
http.redirect_codes = http.redirect_codes - {308}

Iff that's not possible, edit your requirements.txt to pin down httplib2<0.16.0

---

Long story

google cloud storage server uses HTTP 308 for special resumable uploads feature which somewhat resembles "retry same method to same location", but not quite.

Above is (probably) rationale for PyPI package google-resumable-media which is used by more recent incarnations of gcloud related packages and handles 200 and 308 in similar manner, unlike generic HTTP client should.

History context:

• 2016 package gcloud was deprecated in favor of package google-cloud
• 2017 google-cloud-python switches HTTP transport from httplib2 to requests
• 2018 package google-cloud was deprecated again in favor of google-cloud-* packages
• 2020 httplib2 v0.16 gained support for 308 redirects per RFC7538

Sorry for bad news. As HTTP enthusiast, I'm biased towards 308 support. Please reach if you have better idea how to handle this situation more gracefully.

Answer 3

Downgrade your httplib2 version to 0.15.0. Worked for me on the python google-cloud-sdk.

Answer 4

I had this error on Dataflow (masquerading also as a BrokenPipe). Downgrading google-api-python-client to the versions >=1.7.8,<1.7.12 fixed it, because 1.7.12 introduced a dependency on httplib2 0.17.0 which was somehow incompatible.