RabbitMQ multiqueue consumer (independently consumed queues)

TL;DR

Is there any (Node.JS) pattern to stop consuming a queue after a time (10 seconds), and closing the connection, without interrupting the processing of messages were not yet acked?

Longer version

We are sending out millions of push notifications daily for our customers. After generating the messages, we insert them into RabbitMQ, and then send them out with hundreds of consumers. Often customers would like to send out a huge amount of messages almost the same time, and if these messages are going to the same queue, then one customer's messages will have the processing time of the other customer's messages.

We would like to create a dynamic multiqueue setup, where a campaign's messages are going to a messages. queue, and the customers are dynamically subscribing, processing thousands of messages then switching to an other queue. The idea is that we are sharing our resources between campaigns, so they will be sent at the same time in parallel.

Our current solution is generating the messages slower than we are sending them out, so they are inserted into the queue in mixed order. It's not the ideal solution, because it's hard find out what is slow enough, but not too slow.

I've tried creating a simple setTimeout, and forcing the consumer to finish consuming (Channel#cancel) after a few seconds, then subscribing to a queue has the least consumers. The problem is that this interrupts the processing of some messages. I've tried using Channel#get to process a bunch of messages (then choosing a different queue), but the documentation says it is ideal only for a few messages, but not for millions of messages, using Channel#consume is much better in performance.

Any ideas how to do it well?

via András Bártházi