I want to store a large amount (~ 10 million objects) of high frequency (upto 5-10 objects per sec) logging data in JSON format from a node.js application, in such a way that visually examining and searching through the logged data is as easy as possible later.
I don't know how this is done in node.js.
The objects may not be in a consistent format (keys may differ).
In short I'm looking for a package, tool or a combination/set of tools together that
- can store JSON data logs efficiently (something like a table with automatically added columns for each new key, instead of something that just serializes the object) from node.js
- make the logs visually accessible (even better if it's possible in real time)
Can you recommend me something like this?
(I'm not sure if this is the correct SE community for this. If this kind of question is invalid here, could someone please point me in the right direction?)
via tarrasque
No comments:
Post a Comment