Saturday, 18 March 2017

Cannot pipe s3 object readstream to PUT request

I have a simple scenario. I need to read an object from S3 and pipe its output to PUT request. Here is my code using request.

let readStream = s3.getObject({
  Bucket: bucket,
  Key: "/my/key/location"
}).createReadStream();

let formData = {
  "key1": "value1",
  "key2": "value2",
  upload: {
    value: readStream,
    options: {
      filename: "myfilename",
      contentType: "application/gzip"
    }
  }
};

request.put({
  url: "http://" + host + ":" + port + "/bootstrap",
  formData: formData
}, function (error, proxyResponse, body) {
  if (error) {
    reject(error);
  } else {
    resolve();
  }
});

I am getting this error:

File [upload] got 58 bytes
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: Unexpected end of multipart data
    at /pots/cnc/node_modules/dicer/lib/Dicer.js:62:28
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

The funny thing is, if I save the file locally first and then createReadStream for that local file, it works:

let formData = {
  ...
  upload: {
    value: fs.createReadStream(localPath + "/" + filename),
    options: {
      ...
    }
  }
};

The server on receiving end has following code:

busboy.on("file", function (field, file, filename, encoding, mimetype) {
  path += "/" + filename;
  file.pipe(fs.createWriteStream(path));
});
busboy.on("field", function (field, val, fieldTruncated, valTruncated, encoding, mimetype) {
  console.log(field + " " + val);
});
busboy.on("finish", function () {
    ...
});



via Rash

No comments:

Post a Comment