Tuesday 30 May 2017

How to handle streamed post request in Flask?

I would like to stream a csv file via post request from a node.js server to a Flask app. Here is the node.js code that generates the request:

// node.js route that handles form-based csv file upload
app.post('/route', upload.single('file'), function(req,res) {
    var file = req.file;

    // Streams file to Flask app API      
    fs.createReadStream(file).pipe(
        request.post('http://localhost:5000/upload_dataset',
        function(error, response, body){
          console.log(body);
        })
    );
});

And here is the Flask request handler:

@app.route('/upload_dataset', methods=['POST'])
def upload_dataset():
    # Read from the request buffer and write it to a new file
    with open(app.root_path + "/tmp/current_dataset", "bw") as f:
        chunk_size = 4096
        while True:
            chunk = request.stream.read(chunk_size)
            if len(chunk) == 0:
                return

            f.write(chunk)

    return 'Some Response'

The problem is that when the route is hit, len(chunk) is 0 so nothing gets written.

Q1) The transfer-encoding header of the request is type "chunked". I've heard that wsgi does not handle "chunked" requests. Is this what is causing my issue? If so, what is a good alternative to the present implementation?

Q2) request.stream seems to be empty. Is it possible that the request file is ending up in a different parameter of the request object? If so, which one?

Otherwise, if you see any other problems with the code, let me know.

Thanks.



via Allen More

No comments:

Post a Comment