Wednesday, 26 April 2017

How do I move code to a worker when it depends on code outside of the worker?

Running a node.js app that has various components that are required in the main index.js file, like this:

// index.js

require('./component-1.js')
require('./component-2.js')
require('./component-3.js')
// ...etc.

yield App.init()

Basically, by requireing those components, they attach themselves to the App, and I can reference component-1 inside of component-2.

The problem is that component-2 does a lot of tasks that take a lot of time to complete (minutes to hours). So, I want to move component-2 to a separate worker. With Heroku I would do this in the Procfile (telling it to spin up a separate dyno):

# Procfile
worker: node ./component-2.js

Problem: component-2 depends on component-1 having been registered with the App. Therefore if I want to run component-2 as a worker, I'll have to register component-1 inside the worker as well.

One solution I tried was to basically spin up a "mini app" inside the separate worker:

// workers/worker.js

require('component-1')
require('component-2')

yield App.start()

Then do worker: node workers/worker.js inside my Procfile, but now any time I want to extract a component into a worker, I have to track down every single other component it depends on, and move them into the "mini app".

Any other good solutions for this?



via Josh Beam

No comments:

Post a Comment