Transwarp Sample Code

This repository contains the example projects for componetnts on Transwarp Data Hub.

Project List

You can read the user-guide.txt for how to use the projects with a quick view.

BatchInsert. Example for batch insert data by Inceptor JDBC.

ComplexSearch. Example for fulltext search by Inceptor JDBC (index builded by Elasticsearch).

Elasticsearch. Example for usage of Elasticsearch Java API.

FlumeInterceptor. Example for Flume Interceptor, contanins row and column (regex) filter.

FlumeSink. Example for Flume Sink, synchronising folders and files.

FlumeSource. Example for Flume Source.

Hibernate. Example for Inceptor work with Hibernate 3.X.

Hibernate4. Example for Inceptor work with Hibernate 4.X.

Hibernate5. Example for Inceptor work with Hibernate 5.X.

InceptorJDBC. Example for connect Inceptor via jdbc.

InceptorUDAF. Example for Inceptor UDAF.

InceptorUDF. Example for Inceptor UDF.

InceptorUDTF. Example for Inceptor UDTF.

KafkaConsumer. Example for Kafka Consumer with multi-threading.

KafkaDecoder. Example for Kafka Decoder.

KafkaPartitioner. Example for Kafka Partitioner.

KafkaProducer. Example for Kafka Producer with multi-threading.

KeywordSearch. Example for keyword search by Inceptor JDBC (index builded by Elasticsearch), and contains PDF/PPT/Word/Excel to txt convert function.

MultiLocalMode. Example for multi-threading query with Hyperbase external table by Inceptor JDBC.

ObjectStore. Example for storing non-text (such as image, video, etc) files in Hyperbase or HDFS with with multi-threading.

StreamDataCheck. Example for black-list filter with StreamSQL UDF/UDTF.

WebHDFS. Example for Web HDFS Java API.

iBatis. Example for Inceptor work with iBatis.

rJava. Example for Spark Scala API.

Building and Running

To build the code, you will first need to have installed Maven and Java. Then import projects to your favourite IDE (If you use IntelliJ, every project isn't a IntelliJ Maven project's module, the whole project isn't IntelliJ project either. So you need import every project as existing module to your IntelliJ one by one).

The Transwarp Data Hub Maven repository, you can download here.