I am piping data from a textfile to another pipe which is downloading images from some urls. Now as expected this sends a large number of requests in quick succession and remote server shuts me down. I would like to handle next chunk only after the first is processed. My code is:
read.pipe(JSONStream.parse('*'))
.pipe(es.map(function (d, cb) {
download_images(x,y)
.then(function(r) ...)
.fail(function(r) ...)
.fin(function(f) cb())
})
.pipe(xyz)
Since I have just started looking into streams, I might have missed a very simple point, or in my zeal to use streams I could have ignored a better approach
- Extremely Large json file
- Download images with a delay
via RBS
No comments:
Post a Comment