The task is:
- Cluster creates workers
- Each worker loads some remote data line by line and output them to its own process.stdout
- The output of the hole cluster is redirected to file
Expected result: all lines from all workers are concatenated to resulting file
Actual result: on some machines the result is such as expected, on others the result is interlaced.
Tested on 0.10.47 and 0.12.7 nodes under Ubuntu OS. On two machines under Ubuntu 16.04.2 the result is good. On one under Ubuntu 14.04.5 is interlaced.
I need help to understand this situation and fix it on 14.04.
cluster.js:
var cluster = require("cluster");
cluster.setupMaster({
exec: "./worker.js"
});
cluster.fork();
cluster.fork();
worker.js:
var cluster = require("cluster");
var fs = require("fs");
var workerId = cluster.worker && cluster.worker.id || 1;
var stream = fs.createReadStream("./data" + workerId + ".log");
stream.on("end", function () {
process.exit();
});
stream.pipe(process.stdout);
You need two files in the same folder named data1.log and data2.log with data to read:
$ wc -l data*
9260 data1.log
111636 data2.log
120896 total
The piping always gives good result:
$ node cluster.js | wc -l
120896
The redirection to file gives good result only on Ubuntu 16.04 but never on Ubuntu 14.04. Even more on Ubuntu 14.04 it gives every time different results:
$ node cluster.js >result.log; wc -l result.log
114135 <= MUST BE 120896
$ node cluster.js >result.log; wc -l result.log
110136 <= MUST BE 120896
The redirection to file for addition always gives good result:
$ rm result.log
$ node cluster.js >>result.log; wc -l result.log
120896
via TheRoSS
No comments:
Post a Comment