Friday, 28 April 2017

Request stream ECONNRESET for long processing

I am using the request module for fetch a CSV file and do some processing on each row.

var stream = request.get(url)
.pipe(csvParser(delimiter))
.pipe(action1(xx))
.pipe(action2(yy))
.pipe(buffer())
.pipe(index());

The issue is that the processing can be long for each row so the entire process can take 30min/1h. At some point I get a read ECONNRESET error because I guess that the request remains opens.

Is there a way to make request close the connection when the CSV file is done being downloaded?

This should prevent this error if I am not wrong.



via Maxime L

No comments:

Post a Comment