I have a project that requests about 10000 consecutive requests to apache solr for a mathematical calculation.
For individual query, express api works fine. How it works is that it makes query to the apache solr when I make an api request to my local server. However, when the number of requests increase to 10000 consecutive individual requests, it throws ECONNRESET error for some request (they all make request to the same url except with slightly different query parameter). Which query parameter ends up throwing error is not consistent.
Below is example of what I would like to demonstrate (I have deleted the domain because I am not allowed to release it but I checked that it works)
app.get('/morans/test', (req, res) => {
console.log(`start test request`);
let targetUrl = `http://-domain-/solr/genea_expression/smplGeoclust?q=text:"GO:0003674"&stats.facet=geohash_3&rows=10320`;
for (let i = 0; i < 1000; i++){
console.log(`making request ${i}`);
http.get(targetUrl, (result) => {
result.on('data', (chunk) => {
console.log(`working ${i}`);
}).on('end', () => {
console.log(`ending ${i}`);
}).on('error', (err) => {
console.log(`this is error message ${err}`);
})
})
}
});
So I am making 1000 requests to exactly the same URL and mostly I get below result for most of the i values which makes sense.
working 534
working 534
working 534
working 534
working 534
working 534
working 534
ending 534
However sometimes, irregularly I get the following (like one in 200)
{ Error: read ECONNRESET
at exports._errnoException (util.js:1033:11)
at TCP.onread (net.js:584:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: socket hang up
at createHangUpError (_http_client.js:302:15)
at Socket.socketCloseListener (_http_client.js:334:23)
at emitOne (events.js:101:20)
at Socket.emit (events.js:191:7)
at TCP._handle.close [as _onclose] (net.js:513:12) code: 'ECONNRESET' }
Why is this? I know that ECONNRESET error means the communication has abruptly closed but why would that only happen to some of the requests?
via serendipity
No comments:
Post a Comment