Building a Blockchain: Key Value Stores

In other posts, I described the idea, the entities, the serialization implementation of my personal blockchain project (source code repository in Java). I really enjoy writing this code: it pushes me to really understand the involved use case, in the simplest way possible.

Today I want to describe how to store the information needed to run the blockchain, even after the restart of the node.

First, I want to keep the blocks and the world state. And the list of blocks that makes the best chain: maybe the node received more than one block per height, that is, a block could have siblings. And for some other uses case, the node needs also the result of each transaction execution (called usually the transaction receipt).

All this information does not need a full fledged relational database. Simple key value stores are enough. So, I have key value stores, where a key is associated with a value:

Key value stores, and two use cases

At the end, every key and value is a byte array: original keys and values are serialized to byte arrays. Ie, I have a key value store where the key is a hash (serialized to bytes), and the value is the corresponding block entity (serialized to bytes using RLP (Run Length Prefix)). The accounts and the contract storage are saved as tries, each node trie having as key its own hash.

This is the interface for KeyValueStore.java:

Key value interface

When running some code tests, I don’t need a disk-based key value store, a simple map is good enough, HashMapStore.java:

An in-memory key value store

And when the node starts, the services (message communication with other nodes; attending JSON RPC request from external applications like wallets; etc) need a list of stores, I have KeyValueStores.java:

The key value stores

Notice that the transaction receipts store is not yet used nor implemented. Using TDD, I will add it when I wrote a use case and tests that need it. Don’t cross the bridge before you come to it.

File-based Key Value Store

Not only I have an in-memory implementation, I also have a file-based implementation, KeyValueDb.java:

Implemeting a file-based key value, using ValueFile and KeyFile.

It’s oriented to have values that are not likely to change a lot. I should improve to support some few use case of many writings of different values for the same key.

Remote Key Value Store

Exploring some use cases, related to synchronized a node with a network that is already running, I discovered the usefulness of having access to the key value stores of peer nodes. So, now I have a RemoteKeyValueStore.java:

The remote key value store can access the key value store of peer nodes

Its getValue method broadcast a query to peer nodes, and when the response message arrived, the future is resolved (code not shown):

Using a completable future to query remote peer nodes

And the key value store that a node runs could access the remote ones, using the DualKeyValueStore.java:

Its getValue method checks one store, but if the value is missing, the other store is used:

All this code allows the execution of a new block, without having ALL the world state available, so I could implement a quick synchronization (in Ethereum jargon, it is called a beam sync). But this use case deserves a separated post. It could be also used in a light client implementation.

Angel “Java” Lopez

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store