varOne

Join the chat at https://gitter.im/SparkMonitor/varOne

This is an Apache Spark web monitoring tool, named varOne.

varOne provides a web UI for you to monitor the metrics of Spark applications more efficiently and easily. varOne ingests the spark event logs and metric data to summarizes them as rich charts. If you don't want to use this web UI, you can use the RESTful APIs provided by varOne and custom one by yourself.

Usage

Prerequisites

Tip: metrics.properties exists in $SPARK_HOME/conf.

Tip: varOne only support eventLog be storaged on HDFS currently, we will loose this restriction in near future.

a. Download

Click here to download varOne-0.1.0

b. Start varOne daemond

Deploy the varOne-0.1.0.tgz to each node in your cluster and untar all of it
Then pick one node to start all daemonds by following instructions:

After running, you can check whether VarOned process listed by jps
In addition, you can stop all varOne daemond as this command: ./bin/varOned-all.sh stop

c. Start varOne web server

Follow below steps to start varOne web server:

After running, open browser and go to http://localhost:8080/varOne-web/index.html

d. About varOne-site.xml

varOne.server.port

varOne.node.port

varOne.node.thread.number

varOne.server.context.path

varOne.war.tempdir

Development

Check this document