Wednesday, 17 May 2017

How to handle Node Express connection break

I have a simple Express service that does some operations on a large image file. This is the skeleton:

app.post("/extract", function(req, res) {
    let path = "/tmp/" + uuid.v4() + ".tif";
    let stream = req.pipe(fs.createWriteStream(path));

    stream.on('finish', function() {
        console.log("Working on " + req.headers['filename'])
        if (fs.existsSync(path) && fs.statSync(path).size > 0) {
            console.log("Working on path " + path);
        }
    });
});

It does call res.send(), the rest is left out for the sake of brevity.

It uses the GDAL library to do the extraction, and sometimes GDAL just crashes. However, I know for a fact that it is not GDAL itself that is broken because I have taken the same business logic (exact same) outside of the Express handler and run it against the entire data set where crashes randomly occur, and GDAL never has any problems with the data.

So the the only thing I can think is happening here is that the service layer is running into problems such as a premature connection close where GDAL would be operating on a broken file because the write stream would be prematurely closed.

Can someone give me any ideas on how to catch if/when there is a client error such as a premature connection close? I want to be able to detect that and avoid processing broken data.



via Mike Thomsen

No comments:

Post a Comment