This reference architecture provides a set of YAML templates for deploying a reactive microservices architecture based on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate, AWS Lambda, Amazon Kinesis Data Streams, Amazon ElastiCache, and Amazon DynamoDB using Amazon CloudFormation.
You can launch this CloudFormation stack in the US East (North Virginia) Region in your account:
The Amazon CloudFormation-template is based on the reference architecture for deploying containerized microservices with Amazon ECS (Fargate launch type) and AWS CloudFormation (YAML).
The repository consists of a set of nested templates that deploy the following:
Using CloudFormation to deploy and manage services with ECS or Lambda has a number of nice benefits over more traditional methods (AWS CLI, scripting, etc.).
A template can be used repeatedly to create identical copies of the same stack (or to use as a foundation to start a new stack). Templates are simple YAML- or JSON-formatted text files that can be placed under your normal source control mechanisms, stored in private or public locations such as Amazon S3, and exchanged via email. With CloudFormation, you can see exactly which AWS resources make up a stack. You retain full control and have the ability to modify any of the AWS resources created as part of a stack.
Fed up with outdated documentation on your infrastructure or environments? Still keep manual documentation of IP ranges, security group rules, etc.?
With CloudFormation, your template becomes your documentation. Want to see exactly what you have deployed? Just look at your template. If you keep it in source control, then you can also look back at exactly which changes were made and by whom.
CloudFormation not only handles the initial deployment of your infrastructure and environments, but it can also manage the whole lifecycle, including future updates. During updates, you have fine-grained control and visibility over how changes are applied, using functionality such as change sets, rolling update policies and stack policies.
The templates below are included in this repository and reference architecture:
Template | Description |
---|---|
master.yaml | This is the master template - deploy it to CloudFormation and it includes all of the others automatically. |
infrastructure/vpc.yaml | This template deploys a VPC with a pair of public and private subnets spread across two Availability Zones. It deploys an Internet gateway, with a default route on the public subnets. It deploys a pair of NAT gateways (one in each zone), and default routes for them in the private subnets. |
infrastructure/security-groups.yaml | This template contains the security groups required by the entire stack. They are created in a separate nested template, so that they can be referenced by all of the other nested templates. |
infrastructure/load-balancers.yaml | This template deploys an ALB to public subnets, which exposes the ECS service. It is created in in a separate nested template, so that it can be referenced by all of the other nested templates and so that the ECS service can register with it. |
infrastructure/ecs-cluster.yaml | This template deploys an ECS cluster to the private subnets in Fargate mode. |
infrastructure/dynamodb.yaml | This template creates a DynamoDB table to persist event-data. |
infrastructure/elasticache | This template deploys an ElastiCache cluster with Redis 3-engine to the private subnets with a Replication Group for high availability. |
infrastructure/kinesis | This template deploys two Kinesis Streams to decouple services from each other and implement a buffer to temporarily store data. |
infrastructure/lambda | This template deploys two Lambda-functions with according EventSourceMapping to consume data from Kinesis Streams |
services/tracking-service/service.yaml | This is an example of a long-running ECS service that implements the basic logic for an AdTracking use-case. For the full source for the service, see services/tracking-service/src. |
After the CloudFormation templates have been deployed, the stack outputs contain a link to the load-balanced URL for the deployed microservice.
You can launch this CloudFormation stack in the US East 1 (North Virginia) Region in your account:
After the application has been deployed correctly, you can load test data into Redis by calling the following URL using e.g. curl:
curl http://<endpoint>/cache/fill
After the cache has been filled succesfully, you can call the tracking application with an existing program id e.g. 212313
curl http://<endpoint>/event/212313
This HTTP call returns a response like
{"userAgent":"curl/7.54.0","programId":"212313","programName":"program2","checksum":"124","customerId":9124,"customerName":"Customer2","messageId":"06bc2944-886c-4e56-907c-fa248c8af023","valid":true"}
All microservices a implemented using Java 8/11 and Golang and the Java microservices use Maven for dependency management.
The application has two different build-targets: standard build with Maven without any special paratemers and with the profile native-image-fargate
which builds a native binary using GraalVM. For this particular build target, it is necessary to also use a different Dockerfile (Dockerfile-native
) which uses a multistage build process to copy necessary files from the GraalVM-builder image to the target image (libsunec.so
and cacerts
), and to build the application inside the Docker container using GraalVM and Maven.
services/tracking-service/reactive-vertx
: cd services/tracking-service/reactive-vertx
mvn clean install -Dmaven.test.skip=true
docker build . -t <your_docker_repo>/reactive-vertx
-f DockerfileFor this build target, it is not necessary to install GraalVM, because it uses a multi-stage Docker image to build the application inside of a Docker container.
services/tracking-service/reactive-vertx
: cd services/tracking-service/reactive-vertx
docker build . -t <your_docker_repo>/reactive-vertx
-f Dockerfile-nativeservices/redis-updater
: cd services/redis-updater
make build
services/redis-updater
: cd services/redis-updater
make build
ContainerName
and Image
parameters to point to your container image instead of the example container.ListenerRule
priority number (no two services can have the same priority number - this is used to order the ALB path based routing rules).Path
at which you want the service exposed. By default, the containers in your ECS tasks/services are already configured to send log information to CloudWatch Logs and retain them for 7 days. Within each service's template (in services/*), a LogGroup is created that is named after the CloudFormation stack. All container logs are sent to that CloudWatch Logs log group.
You can view the logs by looking in your CloudWatch Logs console (make sure you are in the correct AWS region).
ECS also supports other logging drivers, including syslog
, journald
, splunk
, gelf
, json-file
, and fluentd
. To configure those instead, adjust the service template to use the alternative LogDriver
. You can also adjust the log retention period from the default 365 days by tweaking the RetentionInDays
parameter.
For more information, see the LogConfiguration API operation.
Deploy another CloudFormation stack from the same set of templates to create a new environment. The stack name provided when deploying the stack is prefixed to all taggable resources (e.g., EC2 instances, VPCs, etc.) so you can distinguish the different environment resources in the AWS Management Console.
This set of templates deploys the following network design:
Item | CIDR Range | Usable IPs | Description |
---|---|---|---|
VPC | 10.180.0.0/16 | 65,536 | The whole range used for the VPC and all subnets |
Public Subnet | 10.180.8.0/21 | 2,041 | The public subnet in the first Availability Zone |
Public Subnet | 10.180.16.0/21 | 2,041 | The public subnet in the second Availability Zone |
Private Subnet | 10.180.24.0/21 | 2,041 | The private subnet in the first Availability Zone |
Private Subnet | 10.180.32.0/21 | 2,041 | The private subnet in the second Availability Zone |
You can adjust the CIDR ranges used in this section of the master.yaml template:
VPC:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub ${TemplateLocation}/infrastructure/vpc.yaml
Parameters:
EnvironmentName: !Ref AWS::StackName
VpcCIDR: 10.180.0.0/16
PublicSubnet1CIDR: 10.180.8.0/21
PublicSubnet2CIDR: 10.180.16.0/21
PrivateSubnet1CIDR: 10.180.24.0/21
PrivateSubnet2CIDR: 10.180.32.0/21
ECS has the ability to perform rolling upgrades to your ECS services to minimize downtime during deployments. For more information, see Updating a Service.
To update one of your services to a new version, adjust the Image
parameter in the service template (in services/* to point to the new version of your container image. For example, if 1.0.0
was currently deployed and you wanted to update to 1.1.0
, you could update it as follows:
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
- Name: your-container
Image: registry.example.com/your-container:1.1.0
After you've updated the template, update the deployed CloudFormation stack; CloudFormation and ECS handle the rest.
To adjust the rollout parameters (min/max number of tasks/containers to keep in service at any time), you need to configure DeploymentConfiguration
for the ECS service.
For example:
Service:
Type: AWS::ECS::Service
Properties:
...
DesiredCount: 4
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 50
If you found yourself wishing this set of frequently asked questions had an answer for a particular problem, please submit a pull request. The chances are that others will also benefit from having the answer listed here.
Please create a new GitHub issue for any feature requests, bugs, or documentation improvements.
Where possible, please also submit a pull request for the change.