Friday 2 June 2017

Algorithm run from within Node HTTP request takes much longer to run

I have a node app which plots data on an x,y dot plot graph. Currently, I make a GET request from the front end and my back end node server accepts the requests, loops through an array of data points, draws a canvas using Node Canvas and streams it back to the front end where it's displayed as a PNG image.

Complicating things is that there are can be polygons so my algorithm calculates if a point is inside a polygon, using the point in polygon package, and colors that data point differently if it is.

This works fine when there are less than 50,000 data points. However, when there are 800,000 the request takes approximately 23 seconds. I have profiled the code and most of that time is spent looping through all the data points and figuring out where to plot it on the canvas and what color (depending on if it's in one or more polygons). Here's a plunker i made. Basically i do something like this:

for (var i = 0; i < data.length; i++) {

  // get raw points
  x = data[i][0];
  y = data[i][1];

  // convert to a point on canvas
  pointX = getPointOnCanvas(x);
  pointY = getPointOnCanvas(y, 'y');

  color = getColorOfCell(pointX, pointY);

  color = color;

  plotColor.push({
      color: color,
      pointX: pointX,
      pointY : pointY
  });

}

// draw the dots down here

The algorithm itself it's the problem. The issue I have is that when the algorithm is run within a HTTP request, it takes a long time to calculate what color a point is - about 16 seconds. But if do it in chrome on the front end, it takes just over a second (see the plunker). When I run the algorithm on the command line with Node, it takes less than a second. So the fact that my app runs the algorithm within a HTTP request is slowing it down massively. So couple of questions:

Why would this be? Why does running an algorithm from within a HTTP request take so much longer?

What can I do to fix this, if anything? Would it somehow be possible to make a request to start the task, and then notify frontend when finished and retrieve the PNG?



via Mark

No comments:

Post a Comment