Monday, 29 May 2017

How to perform a sum on a lot of entitities in Google Datastore?

I am trying to understand if Google Datastore can fit my needs.

I have a lot of entities and I need to perform a sum on a certain property. Basically, I would like to be able to do select count(value1) from entity1 where [some filter], entity1 is an entity that keeps track of some sort of data in its field/property value1.

I know these kind of functions are not available in datastore since it's not a relational database, so the most immediate solution would be to perform a select and then calculate the sum on the result set in the application. So I would have something like (using nodejs, but i don't care about the language):

query = client.query(kind='Task')
query.add_filter('done', '=', False)
results = list(query.fetch())

total = 0
for(v in results)
  total += v.value

The problem is that I have thousands of records, so results may be like 300 000 records.

What's the best way to do this without getting a bottleneck?



via Marco Galassi

No comments:

Post a Comment