aquatic/TODO.md
Joakim Frostegård beb8c52fe6 Update TODO
2023-02-25 22:32:34 +01:00

106 lines
4.2 KiB
Markdown

# TODO
## High priority
* udp prometheus metrics
* ws: wait for crates release of glommio with membarrier fix (PR #558)
* Release new version
* More non-CI integration tests?
* Remove aquatic_http_private?
## Medium priority
* Consider replacing unmaintained indexmap-amortized with plain indexmap
* Run cargo-fuzz on protocol crates
* udp: support link to arbitrary homepage as well as embedded tracker URL in statistics page
* Consider storing torrents in separate IndexMaps. The amount should be a power
of 2 and should be configurable. They could be stored in a Vec and the index
could be calculated by taking the first N bits of the info hash. Each such map
would also store when it was last cleaned. There would then be a small
configurable random chance that when an announce request is being processed,
the map will be cleaned. When doing the normal cleaning round, recently
cleaned maps would be skipped.
* quit whole program if any thread panics
* But it would be nice not to panic in workers, but to return errors instead.
Once JoinHandle::is_finished is available in stable Rust (#90470), an
option would be to
* Save JoinHandles
* When preparing to quit because of PanicSentinel sending SIGTERM, loop
through them, extract error and log it
* Run cargo-deny in CI
* udp: add IP blocklist, which would be more flexible than just adding option
for disallowing requests (claiming to be) from localhost
* stagger cleaning tasks?
* aquatic_ws
* Add cleaning task for ConnectionHandle.announced_info_hashes?
* RES memory still high after traffic stops, even if torrent maps and connection slabs go down to 0 len and capacity
* replacing indexmap_amortized / simd_json with equivalents doesn't help
* SinkExt::send maybe doesn't wake up properly?
* related to https://github.com/sdroege/async-tungstenite/blob/master/src/compat.rs#L18 ?
* aquatic_http_private
* Consider not setting Content-type: text/plain for responses and send vec as default octet stream instead
* stored procedure
* test ip format
* check user token length
* site will likely want num_seeders and num_leechers for all torrents..
* Performance hyperoptimization (receive interrupts on correct core)
* If there is no network card RSS support, do eBPF XDP CpuMap redirect based on packet info, to
cpus where socket workers run. Support is work in progress in the larger Rust eBPF
implementations, but exists in rebpf
* Pin socket workers
* Set SO_INCOMING_CPU (which should be fixed in very recent Linux?) to currently pinned thread
* How does this relate to (currently unused) so_attach_reuseport_cbpf code?
## Low priority
* aquatic_udp
* what poll event capacity is actually needed?
* load test
* move additional request sending to for each received response, maybe
with probability 0.2
* aquatic_ws
* large amount of temporary allocations in serialize_20_bytes, pretty many in deserialize_20_bytes
* extract response peers: extract "one extra" to compensate for removal,
of sender if present in selection?
# Not important
* aquatic_http:
* consider better error type for request parsing, so that better error
messages can be sent back (e.g., "full scrapes are not supported")
* test torrent transfer with real clients
* scrape: does it work (serialization etc), and with multiple hashes?
* 'left' optional in magnet requests? Probably not. Transmission sends huge
positive number.
* aquatic_ws
* write new version of extract_response_peers which checks for equality with
peer sending request???
# Don't do
* general: PGO didn't seem to help way back
## aquatic_http
* request from path:
* deserialize 20 bytes: possibly rewrite (just check length of underlying
bytes == 20 and then copy them), also maybe remove String from map for
these cases too. doesn't really improve performance
* crazy http parsing: check for newline with memchr, take slice until
there. then iter over space newlines/just take relevant data. Not faster
than httparse and a lot worse
## aquatic_udp_protocol
* Use `bytes` crate: seems to worsen performance somewhat
* Zerocopy (https://docs.rs/zerocopy/0.3.0/zerocopy/index.html) for requests
and responses. Doesn't improve performance