4

If I was to create a file-like csv object in memory like so:

output_stream = io.StringIO()
sheet = pyexcel.get_sheet(records=data)
sheet.save_to_memory(file_type='csv', stream=output_stream)

What can I do to save the file like object in output_stream to a file on my default_storage backend with Django?

class Example(models.Model):
    model_file = models.FileField(upload_to='', max_length=255, blank=True, null=True)

I've tried something like:

self.model_file.save(filename, ContentFile(output_stream.read()))

But I get the following error:

"TypeError: ('data must be bytes, received', <class 'str'>)"

pyexcel only supports io.StringIO streams for csv type files.

Traceback:

...
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\files\storage.py", line 49, in save
    return self._save(name, content)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\storages\backends\gcloud.py", line 167, in _save
    content_type=file.mime_type)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\cloud\storage\blob.py", line 1034, in upload_from_file
    size, num_retries, predefined_acl)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\cloud\storage\blob.py", line 947, in _do_upload
    size, num_retries, predefined_acl)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\cloud\storage\blob.py", line 759, in _do_multipart_upload
    transport, data, object_metadata, content_type)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\resumable_media\requests\upload.py", line 94, in transmit
    data, metadata, content_type)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\resumable_media\_upload.py", line 270, in _prepare_request
    raise TypeError(u'`data` must be bytes, received', type(data))
TypeError: ('`data` must be bytes, received', <class 'str'>)
coler-j
  • 1,791
  • 1
  • 27
  • 57

1 Answers1

3

You can read the contents from a StringIO and convert to utf8 encoded bytes like this.

self.model_file.save(filename, ContentFile(output_stream.getvalue().encode()))

getvalue() is similar to read(), but you will always get the full content of the StringIO regardless of the current stream position. With read() you might have to rewind using seek(0).

Håken Lid
  • 22,318
  • 9
  • 52
  • 67