Decompress remote .gz file in Python

Question

i've a issue with Python.

My case: i've a gzipped file from a partner platform (i.e. h..p//....namesite.../xxx) If i click the link from my browser, it will download a file like (i.e. namefile.xml.gz).

So... if i read this file with python i can decompress and read it.

Code:

content = gzip.open(namefile.xml.gz,'rb')
print content.read()

But i can't if i try to read the file from remote source. From remote file i can read only the encoded string, but not decoded it.

Code:

response = urllib2.urlopen(url)
encoded =response.read()
print encoded

With this code i can read the string encoded... but i can't decoded it with gzip or lzip.

Any advices? Thanks a lot

score 4 · Accepted Answer · edited Jan 16 '18 at 17:58

4

Unfortunately the method @Aya suggests does not work, since GzipFile extensively uses seek method of the file object (not supported by response).

So you have basically two options:

Read the contents of the remote file into io.StringIO, and pass the object into gzip.GzipFile (if the file is small)
download the file into a temporary file on disk, and use gzip.open

There is another option (which requires some coding) - to implement your own reader using zlib module. It is rather easy, but you will need to know about a magic constant (How can I decompress a gzip stream with zlib?).

edited Jan 16 '18 at 17:58

Brad Solomon

38,521
31
149
235

answered Apr 26 '13 at 20:15

newtover

31,286
11
84
89

Thanks, my files is very large and i think that the "temporary folder" is the best method. – Nothing Apr 29 '13 at 13:40

Aya · Answer 2 · 2018-01-17T15:49:07.373

0

For Python v3.2 or later, you can use the gzip.GzipFile class to wrap the file object returned by urllib2.urlopen(), with something like this...

import urllib2
import gzip

response = urllib2.urlopen(url)
gunzip_response = gzip.GzipFile(fileobj=response)
content = gunzip_response.read()
print content

...which will transparently decompress the response stream as you read it.

edited Jan 17 '18 at 15:49

answered Apr 26 '13 at 17:03

Aya

39,884
6
55
55

As @newtover mentions, this method won't work with Python <3.2 (including all 2.x versions). – Tom Morris Aug 27 '13 at 13:30

score 0 · Answer 3 · answered Aug 27 '13 at 13:28

0

If you use Python 3.2 or later the bug in GzipFile (requiring tell support) is fixed, but they apparently aren't going to backport the fix to Python 2.x

answered Aug 27 '13 at 13:28

Tom Morris

10,490
32
53

Decompress remote .gz file in Python

3 Answers3