Coverage Status

Stratio Ingestion

Contents

Introduction

Stratio Ingestion started as a fork of Apache Flume (1.6), where you can find:

Custom sources and sinks, developed by Stratio

Several bug fixes

Several enhancements of Flume's sources & sinks

You can find more documentation about us here

Stratio Ingestion components

Details about Stratio Ingestion

Stratio Ingestion is based on Apache Flume so the first question is:

What is Apache Flume?

Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store.

Its use is not only designed for logs, in fact you can find a myriad of sources, sinks and transformations.

In addition, a sink could be a big data storage but also another real-time system (Apache Kafka, Spark Streaming).

Interesting facts about Stratio Ingestion

Compile & Package

$ mvn clean compile package -Ppackage

Distribution will be available at stratio-ingestion-dist/target/ folder. You will find .deb, .rpm and .tar.gz packages ready to use depending your environment. If you take a look at documentation you will find more details about how to install the product, and some useful examples to get a better understanding about Stratio Ingestion.

FAQ

Can I use Stratio Ingestion for aggregating data (time-based rollups, for example)?

*This is not a good idea from our experience, but you can use Stratio Sparkta for real-time aggregation.

Is Flume Ingestion multipersistence?

Yes, you can write data to JDBC sources, mongoDB, Apache Cassandra, ElasticSearch, Apache Kafka, among others.

Can I send data to decision-cep-engine?

Of course, we have developed a sink in order to send events from Flume to an existing stream in our CEP engine. The sink will create the stream if it does not exist in the engine.

Where can I find more details about Stratio Ingestion?

*You can take a look at our Documentation on Confluence

Changelog

See the changelog for changes.