3

I would like to stream a csv file via post request from a node.js server to a Flask app. Here is the node.js code that generates the request:

// node.js route that handles form-based csv file upload
app.post('/route', upload.single('file'), function(req,res) {
    var file = req.file;

    // Streams file to Flask app API      
    fs.createReadStream(file).pipe(
        request.post('http://localhost:5000/upload_dataset',
        function(error, response, body){
          console.log(body);
        })
    );
});

And here is the Flask request handler:

@app.route('/upload_dataset', methods=['POST'])
def upload_dataset():
    # Read from the request buffer and write it to a new file
    with open(app.root_path + "/tmp/current_dataset", "bw") as f:
        chunk_size = 4096
        while True:
            chunk = request.stream.read(chunk_size)
            if len(chunk) == 0:
                return

            f.write(chunk)

    return 'Some Response'

The problem is that when the route is hit, len(chunk) is 0 so nothing gets written.

Q1) The transfer-encoding header of the request is type "chunked". I've heard that wsgi does not handle "chunked" requests. Is this what is causing my issue? If so, what is a good alternative to the present implementation?

Q2) request.stream seems to be empty. Is it possible that the request file is ending up in a different parameter of the request object? If so, which one?

Otherwise, if you see any other problems with the code, let me know.

Thanks.

Allen More
  • 858
  • 3
  • 14
  • 25
  • Flask has direct support for file uploads, just use that instead of reimplementing it. – pvg May 30 '17 at 22:12
  • what does `upload.single('file')` do? where is the body in `request.post`? – wolfsgang May 30 '17 at 22:15
  • pvg: Flask support for uploads seems to be limited to form mediated uploads which is why request.file is empty. I need to be able to stream a file from a node.js app to flask, not from an html form. – Allen More May 30 '17 at 22:19
  • wolfsgang: upload.single.('file') is just some middleware to handle file uploads to node. It's part of multer, a node.js package for parsing multipart data. – Allen More May 30 '17 at 22:20
  • @AllenMore (you should @-prefix your mentions if you want people to be notified of your responses). As to the form thing, that's just a matter of encoding/correctly forming your request. Nothing at all stopping you from doing it from node. You almost certainly want wsgi/the-bowels-of-flask to handle this for you. – pvg May 30 '17 at 22:27
  • @pvg thanks for the tip. According to the flask docs, incoming post request data gets exposed via request.form or request.files if a file is uploaded. So my question is, how do I structure my response such that flask is able to get a hold of the file? I can't find any info on uploading files to flask from sources other than html forms. – Allen More May 30 '17 at 22:47
  • 1
    I'm not super-familiar with node but it looks like there are SO questions and doc pointers that cover this. https://stackoverflow.com/questions/25344879/uploading-file-using-post-request-in-node-js – pvg May 30 '17 at 22:52
  • Although, frankly, looking at this more closely it seems saner to have your front-end webserver just proxy the request to your flask app, if possible. – pvg May 30 '17 at 22:54
  • @pvg, thanks, with your help I have found a solution. Issue was I needed to add contentype = multipart/form-data to the request. Unfortunately I can't proxy the request, node server needs to mutate file before querying flask api. Anyway, thanks again. – Allen More May 30 '17 at 23:22
  • Ah, of course, since you're already reading multipart/form-data. You can answer your own questions, incidentally. – pvg May 30 '17 at 23:27
  • flask request depends Werkzeug, you maybe update Werkzeug. try 0.14.1 or greater. – jie xing Jun 14 '19 at 12:36

0 Answers0