Decentralized Databases: The Power of Layered Computation

Photo by Tom Hermans on Unsplash

Recently I have been thinking about establishing my own best practices when working with decentralized databases. While this endeavor is primarily for my benefit, I though I would share it so that others might benefit, or perhaps just provide some food for thought… — after all, I’m just a passionate enthusiast trying to make sense of it all!

Over the coming weeks I plan on publishing a couple of stories with whatever insights I discover. 🕵️ I’d love your company, hit that follow button and all that jazz 🎷, otherwise — happy reading!

Principle: Use layered calculations

• Create a pyramid of computation where each peer is in charge of specific calculations,
• that can be based on values created by other peers,
• typically sums etc. calculated by that peer own their own data.

Reason and explanation

When doing calculations on streaming data, the amount of calculations needed can quickly become unwieldy.
This is because we essentially need to re-calculate every time new data arrives.
This is inefficient as every peer has to redo the same work over and over again, and individually — i.e. all of them has to do the work.

For this reason, it is better to do calculations locally, and then store the result (or intermediate result) in the decentralized database.

This way other peers can run further calculations based on the stored values, that may not need to change as often.

The goal of this principle is that as computations get more complex, the data on which they rely should change less and less.
By establishing a pyramid of computation, where each layer is building on the calculations of other peers, no peer is overwhelmed.

Some Examples

In the examples I am using a decentralized database system called gun. You don’t need to use gun to gain some take aways from the examples, but if you are curious I have written about gun in some of my other articles! :D

What I don’t think is very nice to do is the following:

collective_pet_age = 0;
// get the total age of all the pets of all the friends.
gun.get("friends").map().get("pets").map().get("age").once(age => {
 collective_pet_age += age;
});

Here we are looping over all friendsand all of their pets, then getting the age of these, and calculating a total. Every peer in the network is doing this, and as the network begin to grow, these calculations will become more and more resource intensive.

Instead what we might try is something like the following:

collective_pet_age = 0;
gun.get("friends").map().get("total-pet-age").once(total_age => {
 collective_pet_age += total_age // total_age of own pets is kept up to date by friend.
});
// remember to store the age of your own pets…
gun.get("me").get("total_pet_age").put(the_total_age_of_my_pets);

Here we are iterating over all friends and then retrieving the sum of their pets’ ages from a property that should be kept up to date by that friend. If every peer is keeping a record of this total-pet-age variable, it makes it easier for everyone!

So, to recap

Store calculated values in a node on each peer, and let that peer be in charge of keeping the value up to date with their own data.

I hope this principle has made sense to you. Let me know what you think! Thanks for reading along, I hope to see you in another article… :)