Build Status


SpinalTap is a general-purpose reliable Change Data Capture (CDC) service, capable of detecting data mutations with low-latency across different data sources, and propagating them as standardized events to downstream consumers. SpinalTap has become an integral component in Airbnb's infrastructure platform and data processing pipeline, on which several critical applications are reliant on. More information from our engineering blog.


SpinalTap does not support message authentication yet as part of its current open-source offering.

Message authentication is a requirement to enforce authenticity and accessibility of messages propagated in the message bus, as well as source verification of transmitted messages. Given SpinalTap is by design a source & destination agnostic library, we believe the burden of supplying a message authentication workflow lies on the message bus solution used, or as an extension to the destination component implement adopted. This is inline with the usage within Airbnb’s infrastructure, where SpinalTap is just another client to our internal message transport which in itself should supply message authentication as a general offering for all its clients. If the need arises in the future, message authentication can be seamlessly integrated within the library as a potential supported feature.

Getting Started

Install Thrift 0.9.3 Compiler

Building SpinalTap Standalone JAR

cd spinaltap-standalone
../gradlew shadowjar

Download and Build Kafka

Recommend the version or above (

cd kafka
./gradle build

MySQL Configuration

MySQL server should be configured to use Row-based binlog:


SpinalTap Standalone Configuration

zk-connection-string: localhost:2181 
zk-namespace: spinaltap-standalone
  bootstrap_servers: localhost:9092
mysql-user: spinaltap
mysql-password: spinaltap
mysql-server-id: 12345
  port: 3306
  database: schema-store
  archive-database: schema-store-archive
  - name: mysql-db-1
    host: localhost
    port: 3306
    host_role: MASTER
    socket_timeout_seconds: -1
    schema_version_enabled: true
      fileName: mysql-bin-changelog.001234
      position: 4
      nextPosition: 4
      - test:users
      - test:places
      pool_size: 2
      buffer_size: 1000
  - name: mysql-db-2
    port: 3306
      - db1:table1
      buffer_size: 1000

Launch ZooKeeper Cluster

cd kafka
bin/ config/

Launch Kafka Broker

cd kafka
bin/ config/

Launch SpinalTap Standalone

java -jar build/libs/spinaltap-standalone-all.jar spinaltap_standalone.yaml