Friday 5 May 2017

Javascript - Remove Keys from TypedArray in JSON

I'm currently trying to store large, typed-arrays from Javascript into JSON-Files.

After stringifying one typed-array, I stored the JSON string in a new File using nodes' file stream system.

So far so good, but, once I checked the newly created file, I noticed that the size of this file is huge. After looking through the actual text stored in the file I noticed quite a lot of potential for improving for reducing the files size.

This exemplaric structure illustrates the structure of the array in the actual JSON string:

{"data": [{"my_key": "my_val1"},{"my_key": "my_val2"},{"my_key": "my_val3"},{"my_key": "my_val4"},{"my_key": "my_val5"},{"my_key": "my_val6"},{"my_key": "my_val7"},{"my_key": "my_val8"},{"my_key": "my_val9"}]}

Now, after googeling around a bit, I found this libary (json-compressed) which is exactly what I want to do with my array. Unfortunately this libary is made for php, I tried other libaries like JSONC and various other libaries that used some sort of compression algorithm. (LZW, DEFLATE/gzip).

I found that all libaries that use some sort of compression algorithm aren't able to provide a solution for my problem as I have to remove the keys before trying to compress the file even further.

Unfortunately, promising libaries like JSONC simply crashed my Application without throwing an error, notification or something else whenever I used their function JSONC.compress(). Their JSONC.pack() function crashes as well, but with giving away an error message. After looking through the issues related to JSONC on github I realized that this project is probably dead anyways.

Now back to my original question, is it possible to remove the keys from each value in the array? And if it is: Whats the smartest approach to do so? A result like this would be perfect and decrease the size of my file tremendously:

{"data": {"keys": ["my_key"], "data":["my_val1","my_val2","my_val3","my_val4","my_val5","my_val6","my_val7","my_val8","my_val9"]}}


Thanks in advance for any useful input!

Best Regards.



via mxl

No comments:

Post a Comment