Event-Driven microservices with Kafka Streams

A skeleton example of how to use Kafka Streams with Event-Driven microservices.

This is an application that generates a simple football statistics like match scores, team and player rankings. It should be a little more comprehensible than a typical code examples based on orders, payments etc.



You can think of this application as a supplement to the basic code examples of Kafka Streams. Kafka Streams is used here extensively for many purposes like data processing, building materialized views, persisting the domain model and exposing the output of services. Furthermore, there is no traditional database, local state stores instead (PostgreSQL plays a role of legacy system only).

This codebase is trying to apply the idea of stateful streams processing with Kafka and Kafka Streams. These blog posts are a great introduction to that concept:


Additional modules:

Events and Streams

Each event represents a state change that occurred to the system.

Event Fields
MatchScheduled match id, season, home club, away club, matchDate
MatchStarted match id, home club, away club
GoalScored goal id, match id, minute, scorer, scored for
CardReceived card id, match id, minute, receiver, type (yellow/red)
MatchFinished match id
PlayerStartedCareer palyer id, name

These events are the source for stream processing. For example, in order to determine the result of a match, you must join MatchStarted and GoalScored streams and then count the goals (see the code).

Domain Model


Kafka Topics

Topics are organized under logical groups as follows:

The prefixes above aim to separate the application topics from the internal Kafka Streams and Debezium topics.


REST Endpoints

There are only two REST endpoits and both of them are accessed from the outside of the system:

How to run

Requirements: Java 10, Docker with at least 6 GB memory limit.

  1. Add kafka bootstrap address to your /etc/hosts: kafka postgres connect football-match football-player football-view-basic football-view-top football-ui
  2. Build microservices and Docker containers:
    cd football
    mvn install
  3. Build and run the demo application:

    cd football-tests
    mvn install -DskipTests
    java -jar target/football-tests-0.0.1-SNAPSHOT-jar-with-dependencies.jar

    It will start Docker containers and wait until all microservices are ready.

    If the default startup timeout (180 sec) will be insufficient, try to increase it:

    java -jar target/football-tests-0.0.1-SNAPSHOT-jar-with-dependencies.jar 400
  4. Wait a minute and launch http://football-ui:18080/. In a few seconds you should see some statistics updated in real time:



The input is historical data from English Football League Championship. The historical events (goal scored, card received etc.) are played back every few seconds.