Tuesday, 23 May 2017

Fast way to upload json file to mongodb using node and mongoose.

The task i'm trying to achieve is:

1) User upload a large xml file.(200k+ entries)

2) I convert that from to json using xlsx.

3) I now need to upload that json into mongodb quickly.

4) The file needs to update a nested array for that specific user.

What I have tried: Ive actually tried a bunch of methods but this is the closest ive come.

"contacts" is the object with my json array.
"contact" prints out 1 object.
"numbers" is a nested array.

for (var index = 0; index < contacts.length; index++) {
    var contact = contacts[index];

    console.log("index",index);
    var query = List.update({ owner: owner, _id: id }, { $push: { 
        numbers: contact
    } });

    query.stream().on("data", function(d) { 
        // console.log(d); 
        // console.log(index);
        index++;
    });

    query.stream().on("end", function() { 
        // console.log("done"); 
        // response(null, { msg: 'Your List was updated successfully.' }); 
    });}

The output for console.log(d) is :
{ _id: 5923f3cba9d4aae3e4f12e32, owner: 58fe37d7582f56375aa6a141, __v: 0, created_on: 2017-05-23T08:33:15.052Z, hlr: false, campaigns: [], numbers: [], name: 'test' }

My model looks like this:

var number = new mongoose.Schema({
name: { type: String, default: "" },
surname: { type: String, default: "" },
phone_number: { type: String, default: "" },
language: { type: String, default: "English" },
hlr_status: { type: String, default: "" }});

module.exports = mongoose.model('List', new mongoose.Schema({
name: { type: String, default: "" },
numbers: [number],
campaigns: [{ type: String, default: "" }],
hlr: { type: Boolean, default: false },
owner: { type: mongoose.Schema.Types.ObjectId, ref: 'User' },
created_on: { type: Date, default: Date.now }}));

Any ideas where im going wrong?



via JCom09

No comments:

Post a Comment