Evolving justbus-rs

Overview

Over the past year, justbus-rs has pretty much remained stagnant and there wasn't much changes to the codebase and internals of it. However, this year I would like to add more modularity to the project to enable people to choose between the caching strategies they want due to the fact that some caching strategies may work better with a different type of cpu.

Upgrading to actix-web 3.0

With the release of actix-web 3.0, I have decided that upgrading to the latest version will definitely benefit justbus-rs as there are many improvements within actix especially code safety. A lot of unsafe code blocks have been removed and this will also ensure that justbus-rs is less likely to run into weird safety issues. However, since a lot of unsafe blocks have been removed, that may have impacted performance in some way, BUT having a safer implementation is more desirable than a faster one with hidden bugs due to usage of unsafe.

Logging

For v0.3.0, you can enable logging by passing the feature flag logging to enable logging via env_logger crate. At the current moment I haven't really take a look at other logging alternatives but this shall suffice. Do note that logging has huge performance overhead since it's writing the logs to stdout. In the future I may try exploring other logging techniques which has lesser overhead.

TLS

If you want to enable TLS you can enable it via the tls flag. Internally it uses rustls as benchmarks suggests that they perform better than OpenSSL.

Different types of caching strategies

Currently I am looking different ways to implement the caching mechanism, however it is only limited to in-memory cache as I do not want any external communication which can be a bottleneck. The few ways I would like to cache the responses are

parking_lot::RwLock<HashMap<u32, String>> (More details below why I want to add this back)
cht lock-free hashmap
Dashmap

All these features can be toggled using feature flags during compilation if the user wishes to change the default caching strategy.

`RwLock<HashMap<u32>, String>`

Initially, in my previous post, I mentioned that I removed this method to cache the responses. However, ever since I upgraded my machine to a Ryzen 3600, the performace of RwLock<HashMap<u32, String>> managed to reach the same or even better performance than cht. Honestly, I am not sure what causes the increase in performance, but I suspect the huge CPU cache and higher memory bandwidth has something to do with it. Unfortunately, I am unable to test this hypothesis as I already sold my old DDR3 ram that sits on the older Intel Z77 motherboard.

You can enable this feature via the swisstable. If the user wishes to take advantage of hardware lock elision, they can simply switch to the nightly compiler and compile with the nightly feature to enable it.

`Cht` lock-free hashmap and `Dashmap`

The current implementation uses cht as that was one of the few concurrent hashmap implementations that I can find during the early stages of justbus-rs. Cht option will still be available via the cht feature. Dashmap has been advertised as one of the fastest concurrent hashmap in rust with really fantastic benchmarks. For the next v0.3.0 of justbus-rs, Dashmap has been included via the dashmap feature.

Performance

Disclaimer, this benchmark is naive and YMMV. You should do your own testing as there are a lot of other variables that is purposely ommited for the sake of brevity. For the benchmarks we will be using the following machine

Personal Computer

CPU: Ryzen 3600 @ stock frequency (6 core processor)
RAM: 2x8GB DDR4 CL16 3600Mhz
OS: Ubuntu 20.04

swisstable

Running 15s test @ http://localhost:8080/api/v1/timings/83139
  6 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.40ms    4.49ms 109.89ms   92.99%
    Req/Sec   126.60k    33.29k  172.64k    74.11%
  11357888 requests in 15.05s, 23.48GB read
  Non-2xx or 3xx responses: 59
Requests/sec:   754475.17
Transfer/sec:      1.56GB

Memory Usage @ Peak: 21MB

Closing words

0.3.0 has quite a number of changes with regards to compilation and the internal codebase might be more verbose, however this is done to ensure that the final binary does not contain any additional things that the user might not want to use. All in all, the functionality remains the same as there are 0 changes to the json representation and it should be as easy to change the old server to the new one without altering any client code.