Building a Blockchain: Key Value Stores
In other posts, I described the idea, the entities, the serialization implementation of my personal blockchain project (source code repository in Java). I really enjoy writing this code: it pushes me to really understand the involved use case, in the simplest way possible.
Today I want to describe how to store the information needed to run the blockchain, even after the restart of the node.
First, I want to keep the blocks and the world state. And the list of blocks that makes the best chain: maybe the node received more than one block per height, that is, a block could have siblings. And for some other uses case, the node needs also the result of each transaction execution (called usually the transaction receipt).
All this information does not need a full fledged relational database. Simple key value stores are enough. So, I have key value stores, where a key is associated with a value:
At the end, every key and value is a byte array: original keys and values are serialized to byte arrays. Ie, I have a key value store where the key is a hash (serialized to bytes), and the value is the corresponding block entity (serialized to bytes using RLP (Run Length Prefix)). The accounts and the contract storage are saved as tries, each node trie having as key its own hash.
This is the interface for KeyValueStore.java:
When running some code tests, I don’t need a disk-based key value store, a simple map is good enough, HashMapStore.java:
And when the node starts, the services (message communication with other nodes; attending JSON RPC request from external applications like wallets; etc) need a list of stores, I have KeyValueStores.java:
Notice that the transaction receipts store is not yet used nor implemented. Using TDD, I will add it when I wrote a use case and tests that need it. Don’t cross the bridge before you come to it.
File-based Key Value Store
Not only I have an in-memory implementation, I also have a file-based implementation, KeyValueDb.java:
It’s oriented to have values that are not likely to change a lot. I should improve to support some few use case of many writings of different values for the same key.
Remote Key Value Store
Exploring some use cases, related to synchronized a node with a network that is already running, I discovered the usefulness of having access to the key value stores of peer nodes. So, now I have a RemoteKeyValueStore.java:
Its getValue method broadcast a query to peer nodes, and when the response message arrived, the future is resolved (code not shown):
And the key value store that a node runs could access the remote ones, using the DualKeyValueStore.java:
Its getValue method checks one store, but if the value is missing, the other store is used:
All this code allows the execution of a new block, without having ALL the world state available, so I could implement a quick synchronization (in Ethereum jargon, it is called a beam sync). But this use case deserves a separated post. It could be also used in a light client implementation.
Angel “Java” Lopez