Wednesday 26 April 2017

Is it right to run a Parse server in a NodeJS cluster?

We deployed a Parse server which is causing us a lot of problems on our platform. I think the problem is in the deployment rather than the code. However, I can't figure out how to optimise it. Two of the main problems are: 1) slowness of the web platform 2) no concurrencies between requests

THE PROBLEM: When profiling the querying of my dashboard I noticed that the NodeJS process on the server would spike to around 80% and sit there for about 5 to 10 minutes while the dashboard was being loaded. At the same time, the MongoDB (database) processes would hover around 10% or so. This leads me to believe that the problem is in the NodeJS (Parse) layer, not the database per se (even though the database could probably need some indexes here and there)

MY UNDERSTANDING OF THE ISSUE: When I load the dashboard, a lot of simultaneous calls go out to the Parse server. Because NodeJS is a single threaded server, it means that rather than spawning more processes for concurrent connections that are made to it, NodeJS utilises an "event loop". (i.e all connections are serviced by a single process). This means that NodeJS can basically only do one thing at the time while spending a little bit of time on each connection in quick succession. However, if this "event loop" is blocked it means that all other connections that are waiting for a response are, just waiting.

POSSIBLE WORK AROUND: A work around for this would be to simply start multiple NodeJS services, i.e. start multiple single processes. I'm not exactly sure if there are any specific Parse issues with this, but this is how it works when we want to scale NodeJS.

==> Is that correct?

So, to rework my Problem a little bit:

Description:

  • Parse is a specialised application server running in NodeJS.
  • Currently when the dashboard is loaded many simultaneous requests for data go out the the Parse server to query the database.
  • Currently the Digital Ocean server only runs a single Parse (NodeJS) process.
  • This single process is used to serve all requests made to the Parse server.
  • If the single Parse (NodeJS) process is blocked, or takes a long time to complete, it will result in other connections not being served, i.e. placed in a waiting state

Solution:

Investigate if the Parse server is being "blocked" when being queried by the PHP backend. In NodeJS this is usually the case when so called "sync" calls are made, e.g. "fs.open(..)" vs "fs.openSync(..)" where the latter one "blocks" the event loop (i.e. "pauses" the server).

==> Investigate if it's possible to run a Parse server in a NodeJS cluster: https://nodejs.org/docs/latest-v5.x/api/cluster.html

==> Do you have any other recommendations or is it the right things to do?

Thanks a lot!!!



via rom_stas

No comments:

Post a Comment