Tuesday 6 June 2017

improve mongo query performance

I'm using a node based CMS system called Keystone, which uses MongoDB for a data store, giving fairly liberal control over data and access. I have a very complex model called Family, which has about 250 fields, a bunch of relationships, and a dozen or so methods. I have a form on my site which allows the user to enter in the required information to create a new Family record, however the processing time is running long (12s on localhost and over 30s on my Heroku instance). The issue I'm running into is that Heroku emits an application error for any processes that run over 30s, which means I need to optimize my query. All processing happens very quickly except one function. Below is the offending function:

const Family = keystone.list( 'Family' );

exports.getNextRegistrationNumber = ( req, res, done ) => {

    console.time('get registration number');

    const locals = res.locals;

    Family.model.find()
        .select( 'registrationNumber' )
        .exec()
        .then( families => {

            // get an array of registration numbers
            const registrationNumbers = families.map( family => family.get( 'registrationNumber' ) );

            // get the largest registration number
            locals.newRegistrationNumber = Math.max( ...registrationNumbers ) + 1;

            console.timeEnd('get registration number');

            done();

        }, err => {

            console.timeEnd('get registration number');

            console.log( 'error setting registration number' );

            console.log( err );

            done();
        });

};

the processing in my .then() happens in milliseconds, however, the Family.model.find() takes way too long to execute. Any advice on how to speed things up would be greatly appreciated. There are about 40,000 Family records the query is trying to dig through, and there is already an index on the registrationNumber field.



via autoboxer

No comments:

Post a Comment