data Artisans Streaming Ledger

Serializable ACID transactions on streaming data

data Artisans Streaming Ledger is a library on top of Apache Flink, for processing event streams across multiple shared states/tables with Serializable ACID Semantics.

Instead of operating on a single key in a single operator at a time (like in vanilla Apache Flink and other stream processors) data Artisans Streaming Ledger allows you to define a set of states, connect streams of events that drive the transactions, and apply flexible business logic that operates transactionally across those states.

This repository contains the following maven modules:

A parallel runner exists as part of the dA platform, you can learn more about the dA platform here: dA Platform


Let's create a simple ledger of user accounts. An account in the ledger is identified by a String key, and has a Long value (its balance).

We start by defining the streaming ledger scope. All state definitions and transaction functions would be bound to this named scope "Account Ledger".

    StreamingLedger ledger = StreamingLedger.create("Account Ledger");

Next, we define the accounts state.

    StreamingLedger.State<String, Long> accounts = ledger.declareState("accounts")

Next, let's assume we have a DataStream of TransactionEvents, with the following schema:

final class TransactionEvent {

    private final String sourceAccountId;

    private final String targetAccountId;

    private final long accountTransfer;


    public String getSourceAccountId() {
        return sourceAccountId;

    public String getTargetAccountId() {
        return targetAccountId;

    public long getAccountTransfer() {
        return accountTransfer;


And we would like to transfer money from the source account to the target account, in response to an incoming TransactionEvent.

    DataStream<TransactionEvent> transactions = ...

    ledger.usingStream(transactions, "transaction stream")
        .apply(new TxnHandler())
        .on(accounts, TransactionEvent::getSourceAccountId, "source-account", READ_WRITE)
        .on(accounts, TransactionEvent::getTargetAccountId, "target-account", READ_WRITE);

Where TxnHandler is a TransactionProcessFunction defined as:

class TxnHandler extends TransactionProcessFunction<TransactionEvent, Void> {

        public void process(
                TransactionEvent txn,
                Context<Void> transactionContext,
                @State("source-account") StateAccess<Long> sourceAccount,
                @State("target-account") StateAccess<Long> targetAccount) {

            final long sourceAccountBalance =;
            final long targetAccountBalance =;

            // check the preconditions
            if (sourceAccountBalance > txn.getAccountTransfer()) {

                // compute the new balances
                long newSourceBalance = sourceAccountBalance - txn.getAccountTransfer();
                long newTargetBalance = targetAccountBalance + txn.getAccountTransfer();

                // write back the updated values

Note that TxnHandler will be executed with the following guaranties:

A more complete example can be found here

Building from source


git clone
cd da-streamingledger
mvn clean install

data Artisans Streaming Ledger is now available at your local .m2 repository.

Obtaining from Maven Central

Just add the following dependency to start experimenting with the SDK



The code in this repository is under the Apache license, see license