Thursday 13 April 2017

How to retrieve a lot of objects in MongoDB?

What I need to do:

I need to export a huge amount (undetermined size) of data to CSV from MongoDB according to a date range. (Minimum is a day and there's no maximum)

The amount of data is different between each days.

My problems:

I thought about using mongoexport but I need to add columns that doesn't exist in the DB.

My MongoDB objects are not huge individually, they are composed with texts only but I have a lot of them in the database and need to retrieve a lot of them. So when I am using Mongo through NodeJS to query my objects, I have:

JavaScript heap out of memory

I am searching a long-term solution that will allow me not to worry about how many days of data I am trying to retrieve.

Any solution(s)?

Thank you,



via Juju013

No comments:

Post a Comment