This is an example on how to use Kafka Streams API to aggregate logs and display results online in a dashboard.
Genome Campus Software Craftsmanship Meetup
Follow step 1 and step 2 of Kafka Quickstart
Create two topics: data-in and data-out
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic data-in
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic data-out
Clone and build java project
git clone https://github.com/ebi-wp/kafka-streams-api-websockets.git
cd kafka-streaming-websockets
mvn clean package
Run producer
java -cp target/shade-kafka-streaming-websockets-0.1.0.jar uk.ac.ebi.produce.KafkaExampleProducer
Run streaming client
java -cp target/shade-kafka-streaming-websockets-0.1.0.jar uk.ac.ebi.streaming.KafkaStreamingMain
Run spring boot
java -cp target/shade-kafka-streaming-websockets-0.1.0.jar uk.ac.ebi.Application
See live dashboard in localhost:8080
When you click "Connect" you should see:
If you run without internet access you might need to add following to config/server.properties
advertised.host.name = localhost
advertised.port = 9092