Cloudwatch Logback Appender

License

This project is licensed under the Apache License Version 2.0.

The copyright owner is Dieter Bogdoll.

Overview

This project provides a logback appender whichs target is AWS Cloudwatch Logs.

Maven

<dependency>
    <groupId>io.github.dibog</groupId>
    <artifactId>cloudwatch-logback-appender</artifactId>
    <version>1.0.6</version>
</dependency>

Configuration

With the following XML fragment you can configure your Cloudtwatch logback appender:

<appender name="cloud-watch" class="io.github.dibog.AwsLogAppender">

    <awsConfig>
        <credentials>
            <accessKeyId></accessKeyId>
            <secretAccessKey></secretAccessKey>
        </credentials>

        <profileName>awsProfile</profileName>

        <region></region>

        <clientConfig class="com.amazonaws.ClientConfiguration">
            <proxyHost></proxyHost>
            <proxyPort></proxyPort>
        </clientConfig>
    </awsConfig>

    <createLogGroup>false</createLogGroup>
    <queueLength>100</queueLength>
    <groupName>group-name</groupName>
    <streamName>stream-name</streamName>
    <dateFormat>yyyyMMdd_HHmm</dateFormat>

     <layout>
        <pattern>[%X{a} %X{b}] %-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
     </layout>

</appender>

To be able to use the appender the IAM profile under which the logging is running requires
at least the following AWS permissions:

The section <awsConfig> is optional and on an EC2 instance. It usually is not required as long as you have attached an IAM profile to your instance with the right permissions and/or have set the environment variables required to provide the AWS credentials.

But if that section is available in the configuration it will be used instead of the data from the environment.

To authenticate you can have currently three mechanism:

The sub section <region> should contain the AWS region into which the log information should be streamed, please find here the actual list of regions. You have to use the format you can find in the left column, e.g. like eu-central-1.

The <clientConfig> is again used mainly when your logging process is not run on an EC2 instance, but somewhere outside of AWS. Please lookup the ClientConfiguration within the AWS documentation.

And here now the remaining configuration elements:

Tips and Tricks

Unique log stream names

To make the log stream name unqiue across the same application and multiple ec2 instances, we can use the variable substitution mechanism of logback:

<appender name="cloud-watch" class="io.github.dibog.AwsLogAppender">

    <!-- just referencing the important settings -->
    <streamName>stream-name-${instance.id}</streamName>

</appender>

And set the variable (in our case) instance.id via either -D from the command line, or via calling System.setProperty("instance.id", uniqueId) as one of the first methods in your main.

Setting via -D is the recommended way.

Caveats

The pom.xml used for this project binds to a specific version of the AWS SDK. In case you are using in your project also the AWS SDK the changes are high that the version is different to that ot the cloudwatch-logback-appender, and this could lead to problems. If the version of your AWS SDK is smaller then the one of cloudwatch-logback-appender I advise you to upgrade to the latest AWS SDK version. If your AWS SDK is version is later then the one used by cloudwatch-logback-appender you could replace the dependecy to cloudwatch-logback-appender like this:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-logs</artifactId>
    <version>VERSION_OF_OUR_AWS_SDK</version>
</dependency>

<dependency>
    <groupId>io.github.dibog</groupId>
    <artifactId>cloudwatch-logback-appender</artifactId>
    <version>VERSION_OF_CLOUDWATCH_LOGBACK_APPENDER</version>
    <exclusions>
        <exclusion>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-logs</artifactId>
        </exclusion>
    </exclusions>
</dependency>