Using a DSL (Domain Specific Language) for RSK Java implementation tests

I have already described something of the RSK project in Java in other post. Today I would like to comment on an addition I wrote a few years ago to facilitate some tests. I could say that those tests are integration tests. I’m not an expert on test nomenclature, but see Integration testing for more information and definitions.

The project is a fork of EthereumJ project (now deprecated). When forked, that project didn’t have many integration tests, and in general, it was difficult to write them: to create the necessary objects (from a blockchain to a block storage or contract storage) it was a process quite arduous or convoluted. Tests of this kind, which tested various components, were based on data written in JSON (see EthereumJ original jsonsuite package or the new rskj jsonsuite package): those JSON files described the transactions and accounts that were needed at the beginning of the execution, the blocks that were to be executed and the expected results. But those JSON files were generated from one of Ethereum’s implementations, based in C++ programming language. They were then run by all the other Ethereum implementations (Python, Go, Java, etc.) to see if the results were the same. At RSK the only implementation was the same one we wanted to test. Those JSON files could have been generated from the project, and then used to test the same project behavior: a kind of inbred relationship.

Some examples

There are texts files that describes the actions to be executed, see the DSL resources:

Partial list of DSL files with actions to be executed and tested

This is the accounts01.txtfile content:

# create account with initial balance
account_new acc1 10000000

# check account balance
assert_balance acc1 10000000

There are verbs and arguments. The accoutn_new verb allows to create an account with a symbolic name (acc1), and with initial balance. That account is already created before processing the genesis block. A verb like assert_balanceallows to check that an account has an expected balance amount.

This is the blocks01.txt file content:

# Create two blocks, starting from genesis block
block_chain g00 b01 b02

# Add the two blocks, in order, to current block chain
block_connect b01 b02

# Assert best block
assert_best b02

# Assert latest connect result
assert_connect best

The block_chain verb creates a chain of blocks: new block b01 has as parent the block g00; block b02 has as parent the block b01. The block g00 is the genesis block, created at the beginning of the world process, after processing the initial accounts.

Creating blocks does not imply that they are already on the blockchain. You have to connect them using the verb block_connect that receives as arguments the list of blocks to add, in order.

The assert_best verb ensures that the best block, after this process, is block b02.

The assert_connect best checks that the last block added to the blockchain was accepted as the best block (it could have been rejected).

The uncles01.txt file content:

# Create one block and two uncles
block_chain g00 b01
block_chain g00 u01
block_chain g00 u02

# Create second block with uncles

block_build b02
    parent b01
    uncles u01 u02
    build

# Add the two blocks
block_connect b01 u01 u02 b02

# Assert best block
assert_best b02

# Assert latest connect result
assert_connect best

The block_build verb is a multiline one. It allows to specify the content of a new block, like its uncles, its transactions, its parent block. Then the last verb build creates the specified block with the symbolic name b01 or b02

transfer01.txt file:

account_new acc1 10000000
account_new acc2 0

transaction_build tx01
    sender acc1
    receiver acc2
    value 1000
    build

block_build b01
    parent g00
    transactions tx01
    build

block_connect b01

# Assert best block
assert_best b01

assert_balance acc2 1000

There is another multiline verb: transaction_build. You can specify the sender account, the receiver account, the value to transfer, the gas limit, etc… Then, the transactions could be added to a block using the block_build verb.

But what about create and execute contracts? See create01.txt file:

account_new acc1 10000000

# Create empty.sol contract

transaction_build tx01
    sender acc1
    receiverAddress 00
    value 0
    data 60606040523415600e57600080fd5b603580601b6000396000f3006060604052600080fd00a165627a7a72305820b25edb28bec763685838b8044760e105b5385638276b4768c8045237b8fc6bf10029
    gas 1200000
    build

block_build b01
    parent g00
    transactions tx01
    build

block_connect b01

# Assert best block
assert_best b01

# The code test checks the gas used

The creation of a contract could be executed adding the data to transaction_build verb. It should contain the compiled EVM/RVM bytecode of the contract to be deployed

Notice that the last comment refers to additional code asserts in the test Java code.

One of the latest addition is a multiline comment, see recursive01.txt file content:

comment

// Contracts compiled using
// Truffle v5.1.14 (core: 5.1.14)
// Solidity v0.5.16 (solc-js)
// the contract to be deployed is RecursiveParent

// the contracts source code

// RecursiveInterface.sol

pragma solidity >=0.5.0 <0.6.0;

interface RecursiveInterface {
    function increment(uint level) external;
}

// RecursiveParent.sol

pragma solidity >=0.5.0 <0.6.0;

import "./RecursiveInterface.sol";
import "./RecursiveChild.sol";

contract RecursiveParent is RecursiveInterface {
// ....
// end of comment
end

Its main use is to specify the original source code of the smart contracts that are used along the test. The smart contracts should be deployed using the compiled bytecodes into the data field of a transaction. But the source code is useful as a reference.

Usage

This is a typical Java code test that executes a DSL file:

@Test
public void runTransfers01Resource() throws FileNotFoundException, DslProcessorException {
    DslParser parser = DslParser.fromResource("dsl/transfers01.txt");
    World world = new World();
    WorldDslProcessor processor = new WorldDslProcessor(world);
    processor.processCommands(parser);

    Assert.assertNotNull(world.getAccountByName("acc1"));
    Assert.assertNotNull(world.getAccountByName("acc2"));
    Assert.assertNotNull(world.getTransactionByName("tx01"));
    Assert.assertNotNull(world.getBlockByName("b01"));
}

So, you can execute the file with the DSL commands, and then, using code, check the final world state.

The package

World, DSL parser, DSL command, DSL processor and related classes are included into the test/dsl package:

DSL Parser, Commands, Processors and World builder

The WorldDslProcessor is the executor of the commands specified in the text file.

To do: Improve the parser, to allow more complex cases. And add new assertion verbs like assert_storage, assert_nonce, call contract view functions and check their return values, etc.

Angel “Java” Lopez