Wednesday, 17 May 2017

Problematic JSON prop/field with MongoDB

I have a response from 'request' using the following code:

collection.distinct('id', (err, docs) => {
    docs.forEach(id => {
        let url = 'url.com/id=' + id
        request(url, (error, response, body) => {
            var resp = JSON.parse(response.body.replace('._', '_'));
            collection2.insert(resp);
        });
    });
});

response.body returns a stringified JSON with some fields and properties containing a period, for example:

MISC._EXTERIOR_FEATURES": {
    "id": 29,
    "name": "MISC._EXTERIOR_FEATURES",
    "attributes": {
      "ROOF_RACK": {
      "id": 0,
      "name": "ROOF_RACK",
      "value": "roof rack"
    }
}

Code explained: for every distinct id in collection, request the url for id. Then, using response.body (string), I used .replace() to "clean up" the stringified JSON, then I parse it, and I insert it into collection2.

Errors:

(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 33): Error: key MISC._EXTERIOR_FEATURES must not contain '.'
(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 34): Error: key MISC._INTERIOR_FEATURES must not contain '.'
(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 35): Error: key MISC._EXTERIOR_FEATURES must not contain '.'
(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 36): Error: key MISC._INTERIOR_FEATURES must not contain '.'
(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 37): Error: key MISC._INTERIOR_FEATURES must not contain '.'
(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 38): Error: key MISC._INTERIOR_FEATURES must not contain '.'
(node:24552) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 39): Error: key MISC._EXTERIOR_FEATURES must not contain '.'



via Moshe

No comments:

Post a Comment