Decoding problems when returning xlsxwriter response with pyramid

Question

I though this question would solve my problem, and I followed the Simple HTTP Server example but I'm getting different issues that I can't find a solution for.

I want to generate an Excel file in my server and return it to the user with an Http response. I'm using xlsxwriter to build the file and a pyramid framwork in my server. I've managed to build the file and return it, but then first issue is LibreCalc (I'm testing on Ubuntu 14) asks me how I want to import the file. It doesn't matter what I select I get this error message

General Error. General input/output error.

If I just build and save the file without returning it as a response, it opens fine.

My code to build the file:

output = StringIO()  
workbook = xlsxwriter.Workbook(output)
worksheet = workbook.add_worksheet()

# add the data

workbook.close()

excelcontent = output.getvalue()
response = Response(content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
                    body=excelcontent)
response.headers.add('Content-Disposition',
                     "attachment; filename=%s.xlsx" % nice_filename)
response.headers.add('Access-Control-Expose-Headers','Content-Disposition')
response.headers.add("Content-Length", str(len(excelcontent)))
response.headers.add('Last-Modified', last_modified)
response.headers.add("Cache-Control", "no-store")
response.headers.add("Pragma", "no-cache")

return response

And I handle the response:

    $http({
        method: 'POST',
        url: url,
        data: data},
        {responseType: 'arraybuffer'}
    ).success(function (response, status, headers, config) {
        var file = new Blob([response], {type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'});
        var fileURL = URL.createObjectURL(file);
        var result = document.getElementsByClassName("excel_hidden_download");
        var anchor = angular.element(result);
        var filename_header = headers('Content-Disposition');
        var filename = filename_header.split('filename=')[1];
        anchor.attr({
            href: fileURL,
            target: '_blank',
            download: filename
        })[0].click();
    })

I thought at first it could have something to do with the fact that I use UTF-8 encoding in the python file, but since I can build and open the file otherwise, I don't think it does.

# -*- coding: utf-8 -*-

Can you download the file directly from the browser address bar URL and see if it works any better? — Mikko Ohtamaa, Sep 15 '15 at 11:35
If not you need to examine the file with a text editor and see what's inside if it gives any hint - it's XML so it should be human readable. — Mikko Ohtamaa, Sep 15 '15 at 11:35
Downloading it from the address bar doesn't work cause it needs extra user input parameters etc. I'll edit it to see if I can try make it work that way. — Niel, Sep 15 '15 at 11:45
I saved the file and tried to open it with gedit, it said there was a problem opening the file. `The file you opened has some invalid characters. If you continue editing this file you could corrupt this document. You can also choose another character encoding and try again.` — Niel, Sep 15 '15 at 11:46
Doesn't make sense, gedit should be able to open any file if you have permissions. — Mikko Ohtamaa, Sep 15 '15 at 11:46
Aaah. I think the problem must be in the file payload, part where you do "add data". Start experimenting with empty files first, then narrow down the problem. — Mikko Ohtamaa, Sep 15 '15 at 11:47
It does open the file, but it's clearly got the wrong encoding for the characters. — Niel, Sep 15 '15 at 11:48
I'll try an empty file, but as I said, if I generate and save the file directly without returning it in the response, it opens fine. — Niel, Sep 15 '15 at 11:49
When I return an empty file, not even adding a worksheet, sometimes it has the same issue, and other times Ubuntu says it experienced an internal error :/ — Niel, Sep 15 '15 at 11:54
I don't know much about returning file reponses, is the `content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'` part correct, because I just copied and pasted it. — Niel, Sep 15 '15 at 11:55
Check the file with hex editor and compare it to a working file. — Mikko Ohtamaa, Sep 15 '15 at 11:58
That that there is no encoding issue byte/unicode string with `excelcontent` — Mikko Ohtamaa, Sep 15 '15 at 12:04
If this is Python 2 then you might be converting some unicodes to bytes implicitly. — Mikko Ohtamaa, Sep 15 '15 at 12:04
I opened it with a hex editor (bless), and I'm comparing it with a working version of the same file. What am I looking for? — Niel, Sep 15 '15 at 12:11
But I think you need to use step debugger (pdb) to go through every payload, content etc. variable in the script and it looks correct and encoding is not messed up. — Mikko Ohtamaa, Sep 15 '15 at 12:31
Well I can't see the data that's supposed to be in the excel file, more strings like `PK.......p/G,.........xl/worksheets/sheet1.xml` — Niel, Sep 15 '15 at 12:43

score 1 · Accepted Answer · edited May 23 '17 at 12:03

I was able to get to a solution by following the answers to this question. I'm not sure exactly what did the trick, but here is my resulting code:

 $http({
     method: 'POST',
     url: url,
     data: data,
     responseType: 'arraybuffer'
 }).success(function (response, status, headers, config) {
     var blob = new Blob([response], {
         type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
     });
     var filename = headers('Content-Disposition').split('filename=')[1];

     var config = {
         data: blob,
         filename: filename,
     };

     FileSaver.saveAs(config);
 })

And it works perfectly, I only needed to change the http code. By the way I used angular-file-saver for the saveAs function.

Decoding problems when returning xlsxwriter response with pyramid

1 Answers1