Zipfile module for Python3.6: write to Bytes instead of Files for Odoo

Question

I've been trying to use the zipfile module for Python 3.6 to create a .zip file which contains multiple objects.
My problem is, I have to manage files from an Odoo database that only allows me to use bytes objects instead of files.

This is my current code:

import zipfile

empty_zip_data = b'PK\x05\x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
zip = zipfile.ZipFile(empty_zip_data, 'w')

# files is a list of tuples: [(u'file_name', b'file_data'), ...]
for file in files:
    file_name = file[0]
    file_data = file[1]
    zip.writestr(file_name, file_data)

Which returns this error:

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/zipfile.py", line 1658, in writestr
  with self.open(zinfo, mode='w') as dest:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/zipfile.py", line 1355, in open
  return self._open_to_write(zinfo, force_zip64=force_zip64)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/zipfile.py", line 1468, in _open_to_write
  self.fp.write(zinfo.FileHeader(zip64))
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/zipfile.py", line 723, in write
  n = self.fp.write(data)
AttributeError: 'bytes' object has no attribute 'write'

How am I supposed to do it? I followed the ZipFile.writestr() docs, but that got me nowhere...

EDIT: using file_data = file[1].decode('utf-8') as second parameter is not useful either, I get the same error.

The problem is not with the `ZipFile.writestr()` line, it's with the `zipfile.ZipFile(empty_zip_data, 'w')` line. It's because `ZipFile` expected to `write` to a file determined by the `path-like` object `empty_zip_data`, but instead it is a `byte` object which doesn't have the `write` method. One inelegant way to do this is to temporarily write the zip file to a physical local first, and then read it back in as `bytes` to be passed back to your Odoo database. — r.ook, Jan 15 '19 at 15:09

r.ook · Accepted Answer · 2019-01-15T15:52:46.047

As mentioned in my comment, the issue is with this line:

empty_zip_data = b'PK\x05\x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
zip = zipfile.ZipFile(empty_zip_data, 'w')

You're trying to pass a byte object into the ZipFile() method, but like open() it is expecting a path-like object.

In your case, you might want to utilize the tempfile module (in this particular example we'll use SpooledTemporaryFile from this relevant question:

import tempfile
import zipfile

# Create a virtual temp file
with tempfile.SpooledTemporaryFile() as tp:

    # pass the temp file for zip File to open
    with zipfile.ZipFile(tp, 'w') as zip:
        files = [(u'file_name', b'file_data'), (u'file_name2', b'file_data2'),]
        for file in files:
            file_name = file[0]
            file_data = file[1]
            zip.writestr(file_name, file_data)

    # Reset the cursor back to beginning of the temp file
    tp.seek(0)
    zipped_bytes = tp.read()

zipped_bytes
# b'PK\x03\x04\x14\x00\x00\x00\x00\x00\xa8U ... \x00\x00'

Note the use of context managers to ensure all your file objects are closed properly after being loaded.

This gives you zipped_bytes which is the bytes you want to pass back to Odoo. You can also test the zipped_bytes by writing it to a physical file to see what it looks like first:

with open('test.zip', 'wb') as zf:
    zf.write(zipped_bytes)

If you are handling file size that are considerably large, make sure to pay attention and make use of max_size argument in the documentation.

score 3 · Answer 2 · answered Jan 31 '19 at 16:10

If you want to handle all of this in memory without a temporary file then use io.BytesIO as the file object for ZipFile:

import io
from zipfile import ZIP_DEFLATED, ZipFile

file = io.BytesIO()
with ZipFile(file, 'w', ZIP_DEFLATED) as zip_file:
    for name, content in [
        ('file.dat', b'data'), ('another_file.dat', b'more data')
    ]:
        zip_file.writestr(name, content)

zip_data = file.getvalue()
print(zip_data)

You may also want to set the compression algorithm as shown because otherwise the default (no compression!) is used.

Wow, after disbelieved testing this (no compression!) seems to be true when not using `ZIP_DEFLATED`. Wouldn't have expected *no compression* to be the default. — philmaweb, Jul 14 '20 at 15:27

Zipfile module for Python3.6: write to Bytes instead of Files for Odoo

2 Answers2

Linked