Thursday, 8 June 2017

Function lambda that write data from JSON Document in Apache Cassandra

I have a lambda function in Node.js that allows me to write to an Apache Cassandra database.

I generate queries from a JSON document event, by retrieving the fields.

In Cassandra, there is no possibility of making an alter table if the column does not exist. So I make an alter table for all columns ignoring the error, insert is done for the JSON elements the first time.

enter image description here

The problem arises when I want to modify the JSON and add fields.

The problem is that I can only insert in my table the last element of the JSON document that I add. Example :

{
"humidte" : 18.75,
"temperature" : 14.45,
}

And I want to add another field to the end like this:

{
"humidte" : 18.75,
"temperature" : 14.45,
"air" : "fait",
 }

enter image description here

There it's done.

But if I insert an element in my JSON document in the middle or at the beginning like this :

{
"humidte" : 18.75,
"air" : "fait",
"temperature" : 14.45,
"topic" : "sk1_000/data"
 }

There is the following error : enter image description here

And I have not my new column.

Here is a part of my code where I make the alter table, and insertion. I use async.

How I can proceed to modify my JSON and add tables as I want in Cassandra from my JSON document.

var keys = Object.keys(event); var count = Object.keys(event).length;
var q = async.queue(function (x, callback) {
                                   x = client.execute (queryadd, { prepare: true }, function (err) {
                                          console.log(err);
                                          callback();

                                   });
             }, Infinity);



                       for( var i = 0; i < keys.length; i++ ) {

                       var queryadd1 = "ALTER TABLE "
                       var queryadd2 = " ADD "
                       var queryadd = queryadd1 + ' ' + keyspace + '.' + table + queryadd2 + '(' +  keys[i] + ' ' + typeLabel(event[keys[i]]) + ')';
                       console.log(queryadd);
                       q.push(console.log(queryadd));
                      }



             q.drain = function() {

                                  var queryinsert1 = "INSERT INTO "
                                  var queryinsert2 = "(nom_entreprise, date_serveur"
                                  var queryinsert3 = "VALUES ('"+ keyspace +"', '"+ Date.now() +"'"
                                  var queryinsert4 = "if not exists "
                                  var queryinsert = queryinsert1 + keyspace +'.'+ table + queryinsert2 + donnees + ') ' + queryinsert3 + valeurs + ') ' + queryinsert4 + ';';
                                  console.log(queryinsert);
                                  client.execute(queryinsert, { prepare: true }, function (err) {
                                     console.log(err); 
                                     client.shutdown ();

                                    });



via Amine Hamouche

No comments:

Post a Comment