aquatic: high-performance open BitTorrent tracker

High-performance open BitTorrent tracker, consisting of sub-implementations for different protocols:

Name	Protocol	OS requirements
aquatic_udp	BitTorrent over UDP	Unix-like (using mio)
aquatic_http	BitTorrent over HTTP with TLS (rustls)	Linux 5.8+ (using glommio)
aquatic_ws	WebTorrent over TLS (rustls, optional)	Linux 5.8+ (using glommio)

Features at a glance:

• Multithreaded design for handling large amounts of traffic
• All data is stored in-memory (no database needed)
• IPv4 and IPv6 support, with separate swarms
• Supports forbidding/allowing info hashes
• Built-in TLS support (no reverse proxy needed)
• Automated CI testing of full file transfers

Known users:

• explodie.org public tracker (udp://explodie.org:6969), typically serving ~80,000 requests per second

Usage

Compiling

• Install Rust with rustup (latest stable release is recommended)
• Install cmake with your package manager (e.g., apt-get install cmake)
• Clone this git repository and enter the directory
• Build the implementations that you are interested in:

# Tell Rust to enable support for all CPU extensions present on current CPU
# except for those relating to AVX-512. This is necessary for aquatic_ws and
# recommended for the other implementations.
. ./scripts/env-native-cpu-without-avx-512

cargo build --release -p aquatic_udp
cargo build --release -p aquatic_http
cargo build --release -p aquatic_ws

Configuring

Generate configuration files. They come with comments and differ between protocols.

./target/release/aquatic_udp -p > "aquatic-udp-config.toml"
./target/release/aquatic_http -p > "aquatic-http-config.toml"
./target/release/aquatic_ws -p > "aquatic-ws-config.toml"

Make adjustments to the files. You will likely want to adjust address (listening address) under the network section.

Note that both aquatic_http and aquatic_ws require configuring certificate and private key files to run over TLS (which is optional for aquatic_ws). More details are available in the respective configuration files.

Workers

To increase performance, number of worker threads can be increased. The sum of socket_workers and swarm_workers should equal the total number of CPU cores that you want to use. Recommended proportions:

	aquatic_udp				aquatic_ws
CPU cores	2-5	6-9	10-15	>=16	2-11	12-19	>=20
Swarm workers	1	2	3	4	1	2	3
Socket workers	1-4	4-7	7-12	>=12	1-10	10-17	>=17

Access control

Access control by info hash is supported for all protocols. The relevant part of configuration is:

[access_list]
# Access list mode. Available modes are allow, deny and off.
mode = "off"
# Path to access list file consisting of newline-separated hex-encoded info hashes.
path = ""

The file is read on start and when the program receives SIGUSR1. If initial parsing fails, the program exits. Later failures result in in emitting of an error-level log message, while successful updates of the access list result in emitting of an info-level log message.

Running

If you're running aquatic_http or aquatic_ws, please make sure locked memory limits are sufficient:

• If you're using a systemd service file, add LimitMEMLOCK=65536000 to it
• Otherwise, add the following lines to /etc/security/limits.conf, and then log out and back in:

*    hard    memlock    65536
*    soft    memlock    65536

Once done, start the application:

./target/release/aquatic_udp -c "aquatic-udp-config.toml"
./target/release/aquatic_http -c "aquatic-http-config.toml"
./target/release/aquatic_ws -c "aquatic-ws-config.toml"

If your server is pointed to by domain example.com and you configured the tracker to run on port 3000, people can now use it by adding its URL to their torrent files or magnet links:

Implementation	Announce URL
aquatic_udp	udp://example.com:3000
aquatic_http	https://example.com:3000/announce
aquatic_ws	wss://example.com:3000

Details on implementations

aquatic_udp: UDP BitTorrent tracker

Implements:

• BEP 015: UDP BitTorrent tracker protocol (more details). Exceptions:
  • Doesn't care about IP addresses sent in announce requests. The packet source IP is always used.
  • Doesn't track the number of torrent downloads (0 is always sent).

This is the most mature of the implementations. I consider it ready for production use.

Performance

More details are available here. Please note that request workers have been renamed to swarm workers.

Optimisation attempts that didn't work out

• Using glommio
• Using io-uring
• Using zerocopy + vectored sends for responses
• Using sendmmsg

aquatic_http: HTTP BitTorrent tracker

Implements:

• BEP 003: HTTP BitTorrent protocol (more details). Exceptions:
  • Only runs over TLS
  • Doesn't track the number of torrent downloads (0 is always sent)
  • Only compact responses are supported
• BEP 023: Compact HTTP responses
• BEP 007: IPv6 support
• BEP 048: HTTP scrape support. Notes:
  • Doesn't allow full scrapes, i.e. of all registered info hashes

aquatic_http has not been tested as much as aquatic_udp but likely works fine in production.

Running behind a reverse proxy is currently not supported due to the difficulties of determining the originating IP address without knowing the exact setup.

Performance

More details are available here. Please note that request workers have been renamed to swarm workers.

aquatic_ws: WebTorrent tracker

Aims for compatibility with WebTorrent clients. Notes:

• Doesn't track the number of torrent downloads (0 is always sent).
• Doesn't allow full scrapes, i.e. of all registered info hashes

aquatic_ws has not been tested as much as aquatic_udp but likely works fine in production.

Running behind a reverse proxy is supported, as long as IPv4 requests are proxied to IPv4 requests, and IPv6 requests to IPv6 requests.

Performance

More details are available here. Please note that request workers have been renamed to swarm workers.

Load testing

There are load test binaries for all protocols. They use a CLI structure similar to the trackers and support generation and loading of configuration files.

To run, first start the tracker that you want to test. Then run the corresponding load test binary:

./scripts/run-load-test-udp.sh
./scripts/run-load-test-http.sh
./scripts/run-load-test-ws.sh

To fairly compare HTTP performance to opentracker, set keep_alive to false in aquatic_http settings.

Architectural overview

Copyright and license

Copyright (c) 2020-2022 Joakim Frostegård

Distributed under Apache 2.0 license (details in LICENSE file.)

Trivia

The tracker is called aquatic because it thrives under a torrent of bits ;-)