Uploading files to restify server using streams

I am working on a server that manages pdf files. This server needs to be able to upload, download and store pdfs.

Client download has been implemented without any issues.

The client upload is the problem:

What I have Tried

1. Removing bodyParser for upload endpoint

When I included the bodyparser the stream was not writing any data into the file. My endpoint is defined before the bodyParser

2. Using streams

Because of the possible size of files I do not want to transfer the file in its entirety and want to use streams. When I use streams, the header of the request is also pushed to the stream. This is not acceptable and requires me to ignore the first chunk that arrives in the response (see code examples below).

The header being pushed to the file stream:

----------------------------487602989105343035284942
Content-Disposition: form-data; name="file"; 
filename="myfileuploadtest.pdf"
Content-Type: application/pdf

My current code

This is my current client code for upload

var req = request.post({url: `http://127.0.0.1:${PORT}/pdf/file`, headers:
{filename: "myfileuploadtest.pdf"}}, (err, resp, body) => {
    if(err){
        console.log("ERROR", err.message);
    } else {
        console.log("body: ", body);
    }
});

var form = req.form();
form.append( "file", fs.createReadStream("myfileuploadtest.pdf"));

This is my current code for upload to the server

let doneIgnoreFirst = false;
req.on("data", (chunk) => {
    if(!doneIgnoreFirst){
        doneIgnoreFirst = true;
    } else {
        writeStream.write(chunk);
    }
});

//dont know if this event is actually fired
req.on("error", (err) => {
    console.log("Error while receiving PDF");
    res.send(500);
});

req.on("end", () => {
    writeStream.end(() => { 
        res.send(500);
    });
});

If not for my issue with the headers being included in the stream, I would have used something like this:

req.pipe(writeStream);
writeStream.on("finish", () => { res.send(200); });
writeStream.on("error", () => { res.send(500); });

I am doing something wrong

There is something that I am missing. I suspect that the restify bodyParser is the fault here. The headers are not parsed from the stream which introduces the issue that I am having and requires me to skip the first chunk.

This is not a solution for this issue. Depending on the request sent, parts of the pdf can be included in the first chunk. this breaks the pdf because not all of its data is included in the file on the server. This makes the endpoint very unstable which I do not want.

Any help is greatly appreciated!

Versions

• restify: 4.3.0
• request: 2.81.0

via Sisso