Heimdallr is a large-scale chat application server inspired by the LINE+ chatting service architecture and written in Scala language based on Akka's actor model. It provides fault-tolerant and reliable scaling-out options, based on Redis Pubsub to support expansion from proof-of-concept to enterprise-ready solutions. It has been tested to be at least 100x faster than socket.io for large-scale systems. Heimdallr is currently powered by real-world production and supporting heavy simultaneous connections. This project is licensed under the Apache License v2.0.
Heimdallr’s architecture consists of a Akka HTTP Server, Streams and ChatRoomActor, UserActor, AggregationActor. Each ChatRoom can be distributed across multiple servers. To synchronize the messages among servers, we uses Redis PubSub. UserActor is created per every websocket client.
TODO: not implemented yet.
The HTTP server test below was done using the wrk benchmarking tool with 6 threads and 10000 connections on a single m4.large instance.
| Node.js | Heimdallr | |
|---|---|---|
| Requests per sec | 14533.90 | 20675.89 |
| Avg. Latency | 68.94 ms | 13.35 ms |
| Summary | 873389 requests in 1.00m, 108.36MB read Socket errors: connect 8981, read 0, write 0, timeout 0 Requests/sec: 14533.90 Transfer/sec: 1.80MB |
1242244 requests in 1.00m, 181.26MB read Socket errors: connect 8981, read 0, write 0, timeout 0 Requests/sec: 20675.89 Transfer/sec: 3.02MB |
This table shows the performance of event broadcasting, the average latency for a message to arrive at a websocket client. This test was done on the same m4.large instance as the HTTP server test.
| Socket.io (single node) |
Heimdallr (single node) |
Heimdallr Cluster (2 nodes) |
Heimdallr Cluster (4 nodes) |
|
|---|---|---|---|---|
| 10 sub, 1 pub | 43 ms | 43 ms | 16 ms | 20 ms |
| 100 sub, 5 pub | 62 ms | 61 ms | 53 ms | 32 ms |
| 1000 sub, 10 pub | 496 ms | 390 ms | 129 ms | 47 ms |
| 1000 sub, 50 pub | 1304 ms | 554 ms | 141 ms | 109 ms |
| 1000 sub, 100 pub | 2242 ms | 605 ms | 202 ms | 114 ms |
Clone the repository and try to build with sbt:
% sbt run
To setup a websocket load balancer, you can use HAProxy or Apache2 HTTPD with proxy modules. If you need to support heavy concurrency, AWS ALB or HAProxy is recommended. If you want to use Apache2, enable modules with the following command: sudo a2enmod rewrite proxy proxy_http proxy_wstunnel proxy_balancer lbmethod_byrequests
And edit the virtual host file like below:
<VirtualHost *:80>
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
<IfModule proxy_module>
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/
RewriteRule ^(.*)$ /$1 [P]
ProxyRequests Off
ProxyPass "/" balancer://mycluster/
<Proxy "balancer://mycluster">
BalancerMember ws://{HEIMDALLR_SERVER_ADDRESS_1}:8080
BalancerMember ws://{HEIMDALLR_SERVER_ADDRESS_2}:8080
</Proxy>
</IfModule>
</VirtualHost>To enable the Redis PubSub, open the application.conf file and edit the Redis IP address and port like below:
redis-ip = "{REDIS_IP_ADDRESS}" and redis-port = 6379
In this example, Java-WebSocket library was used.
public class WSClientSample extends WebSocketClient {
public WSClientSample(URI serverURI) {
super(serverURI);
}
...
public static void main(String[] args) throws Exception {
String uri = ("ws://APACHE_WEBSERVERR/{CHATROOMID}");
WSClientSample c = new WSClientSample(new URI(uri));
c.connect();
}
}To see demo of our project, you can use our ready-made websocket connected html file in Heimdallr/src/main/web.
Basically follow the Scala style guide.