Saturday 22 April 2017

Live Notification - DB Polling - Best Practice

I'm providing my users with live notifications.

I'm debating two options and can't decide which is the best way to go when polling the DB.

(The notifications are transmitted using WebSockets.)

Option 1 (current):

I hold a list of all the logged in users. Every 1000 ms, I check the db for new notifications and if there is any, I send a message via WS to the appropriate user.

Pros:

  • This task is rather not expensive on resources

Cons:

  • In off-times, where's there's only a new notification every 1 minute, I poll the DB 60 times for no reason.
  • In a sense, it's not real-time because it takes 1 full second for new notifications to update. Had it been a chat service, 1 second is a very long time to wait.

Option 2:

Create a hook that whenever a new notification is saved (or deleted), the db get polled.

Pros:

  • The db does not get polled when there are no new notifications
  • Actual real-time response.

Cons:

  • In rush-hour, when there might be as many as 10 new notifications generated every second, the db will be polled very often, potentially blocking its response time for other elements of the site.
  • In case a user was not logged in when their notification was generated, the notification update will be lost (since I only poll the db for logged in users), unless I also perform a count whenever a user logs in to retrieve their notifications for when they were offline. So now, not only do I poll the DB when ever my notification hook is triggered, but also I poll the db every time a user logs-in. If I have notifications generated every second, and 10 log-ins every second, I will end up polling my DB 20 times a second, which is very expensive for this task.

Which option would you choose? 1, 2? or neither? Why?

Here is the code I am currently using (option 1):

var activeSockets = [] //whenever a user logs in or out, the array gets updated to only contain the logged-in users in any given moment

var count = function () {
    process.nextTick(function () {
        var ids = Object.keys(activeSockets) //the ids of all the logged in users
        //every user document has a field called newNotification that updates whenever a new notification is available. 0=no new notifications, >0=there are new notifications
        User.find({_id:{$in:ids}}).select({newNotifications:1}).lean().exec(function (err,users) {
            for(var i=0;i<users.length;i++) {
                var ws = activeSockets[String(users[i]._id)]
                if(ws!=undefined) {
                    if (ws.readyState === ws.OPEN) {
                        //send the ws message only if it wasn't sent before. 
                        if(ws.notifCount!=users[i].newNotifications) {
                            ws.send(JSON.stringify({notifications:users[i].newNotifications}));
                            activeSockets[String(users[i]._id)].notifCount = users[i].newNotifications
                        }

                    }
                    else {
                        //if the user logged out while I was polling, remove them from the active users array
                        delete activeSockets[String(users[i]._id)]
                    }
                }
            }
            setTimeout(function () {
                count()
            },1000)
        })
    })

}

The implementation of Option 2 would be just as simple. Instead of calling

count()

using

setTimeout()

I only call it in my "new notification", "delete notification", and "log-in" hooks.

Code:

var activeSockets = [] //whenever a user logs in or out, the array gets updated to only contain the logged-in users in any given moment

var count = function () {
    process.nextTick(function () {
        var ids = Object.keys(activeSockets) //the ids of all the logged in users
        //every user document has a field called newNotification that updates whenever a new notification is available. 0=no new notifications, >0=there are new notifications
        User.find({_id:{$in:ids}}).select({newNotifications:1}).lean().exec(function (err,users) {
            for(var i=0;i<users.length;i++) {
                var ws = activeSockets[String(users[i]._id)]
                if(ws!=undefined) {
                    if (ws.readyState === ws.OPEN) {
                        //send the ws message only if it wasn't sent before. 
                        if(ws.notifCount!=users[i].newNotifications) {
                            ws.send(JSON.stringify({notifications:users[i].newNotifications}));
                            activeSockets[String(users[i]._id)].notifCount = users[i].newNotifications
                        }

                    }
                    else {
                        //if the user logged out while I was polling, remove them from the active users array
                        delete activeSockets[String(users[i]._id)]
                    }
                }
            }
        //setTimeout was removed
        })
    })

}

Hooks:

hooks = {
    notifications : {

        add: function () {
            count()
            //and other actions
        },
        remove: function () {
            count()
            //and other actions
        }
    },
    users: {
        logIn: function () {
            count()
            //and other actions
        }
    }
}

So, Which option would you choose? 1, 2? or neither? Why?



via Michael Seltenreich

No comments:

Post a Comment