mirror of
https://github.com/YGGverse/aquatic.git
synced 2026-03-31 17:55:36 +00:00
2e67f11caf
* WIP: add udp uring support * WIP: fix udp uring address parsing * WIP: udp uring: resubmit recv when needed * WIP: udp uring: add OutMessageStorage, send swarm responses * WIP: udp uring: increase ring entries to 1024 * WIP: udp uring: add constants * WIP: udp uring: use sqpoll, avoid kernel calls * WIP: udp uring: disable sqpoll * WIP: udp uring: use VecDeque for local responses * udp uring: enable setup_coop_taskrun * udp uring: add RecvMsgStorage * udp: improve split of uring and mio implementations * udp uring: clean up * udp uring: initial ipv6 support * udp uring: improve helper structs * udp uring: clean up, use constants for important data * udp: share create_socket fn between implementations * udp uring: improve send buffer free index finding * udp uring: work on SendBuffers.try_add * udp uring: split into modules * udp uring: Rename RecvMsgMultiHelper to RecvHelper * udp uring: improve SendBuffers * udp uring: fix copyright attribution in buf_ring module * udp uring: stop always consuming 100% cpu * udp uring: clean up * udp uring: add handle_recv_cqe * udp uring: move local_responses into SocketWorker * udp uring: move timeout_timespec into SocketWorker * Update TODO * udp: make io-uring optional * Update TODO * udp uring: enqueue timeout before sends * udp uring: move likely empty buffer tracking logic into SendBuffers * udp uring: improve error handling and logging * udp uring: keep one timeout submitted at a time * udp uring: update pending_scrape_valid_until * udp uring: add second timeout for cleaning * Update TODO * udp uring: store resubmittable squeue entries in a Vec * udp uring: add comment, remove a log statement * Update TODO * Update TODO * udp: io_uring: fall back to mio if io_uring support not recent enough * udp: uring: add bytes_received statistics * udp: uring: add bytes_sent statistics * udp: uring: add more statistics * Update TODO * udp: uring: improve SendBuffers code * udp: uring: remove unneeded squeue sync calls * udp: uring: replace buf_ring impl with one from tokio-uring * udp: uring: store ring in TLS so it can be used in Drop impls * udp: uring: store BufRing in SocketWorker * udp: uring: silence buf_ring dead code warnings, improve comment * Update TODO * udp: uring: improve CurrentRing docs, use anonymous struct field * udp: uring: improve ring setup * udp: uring: get ipv6 working * udp: uring: make ring entry count configurable, use more send entries * udp: uring: log number of pending responses (info level) * udp: uring: improve comment on send_buffer_entries calculation * udp: improve config comments * udp: uring: add to responses stats when they are confirmed as sent * Update TODO * udp: uring: enable IoUring setup_submit_all * Update README
110 lines
4.2 KiB
Markdown
110 lines
4.2 KiB
Markdown
# TODO
|
|
|
|
## High priority
|
|
|
|
* udp uring
|
|
* uneven performance?
|
|
* thiserror?
|
|
* CI
|
|
* uring load test?
|
|
|
|
* ws: wait for crates release of glommio with membarrier fix (PR #558)
|
|
* Release new version
|
|
* More non-CI integration tests?
|
|
* Remove aquatic_http_private?
|
|
|
|
## Medium priority
|
|
|
|
* Run cargo-fuzz on protocol crates
|
|
* udp: support link to arbitrary homepage as well as embedded tracker URL in statistics page
|
|
|
|
* Consider storing torrents in separate IndexMaps. The amount should be a power
|
|
of 2 and should be configurable. They could be stored in a Vec and the index
|
|
could be calculated by taking the first N bits of the info hash. Each such map
|
|
would also store when it was last cleaned. There would then be a small
|
|
configurable random chance that when an announce request is being processed,
|
|
the map will be cleaned. When doing the normal cleaning round, recently
|
|
cleaned maps would be skipped.
|
|
|
|
* quit whole program if any thread panics
|
|
* But it would be nice not to panic in workers, but to return errors instead.
|
|
Once JoinHandle::is_finished is available in stable Rust (#90470), an
|
|
option would be to
|
|
* Save JoinHandles
|
|
* When preparing to quit because of PanicSentinel sending SIGTERM, loop
|
|
through them, extract error and log it
|
|
|
|
* Run cargo-deny in CI
|
|
|
|
* udp: add IP blocklist, which would be more flexible than just adding option
|
|
for disallowing requests (claiming to be) from localhost
|
|
|
|
* stagger cleaning tasks?
|
|
|
|
* aquatic_ws
|
|
* Add cleaning task for ConnectionHandle.announced_info_hashes?
|
|
* RES memory still high after traffic stops, even if torrent maps and connection slabs go down to 0 len and capacity
|
|
* replacing indexmap_amortized / simd_json with equivalents doesn't help
|
|
* SinkExt::send maybe doesn't wake up properly?
|
|
* related to https://github.com/sdroege/async-tungstenite/blob/master/src/compat.rs#L18 ?
|
|
|
|
* aquatic_http_private
|
|
* Consider not setting Content-type: text/plain for responses and send vec as default octet stream instead
|
|
* stored procedure
|
|
* test ip format
|
|
* check user token length
|
|
* site will likely want num_seeders and num_leechers for all torrents..
|
|
|
|
* Performance hyperoptimization (receive interrupts on correct core)
|
|
* If there is no network card RSS support, do eBPF XDP CpuMap redirect based on packet info, to
|
|
cpus where socket workers run. Support is work in progress in the larger Rust eBPF
|
|
implementations, but exists in rebpf
|
|
* Pin socket workers
|
|
* Set SO_INCOMING_CPU (which should be fixed in very recent Linux?) to currently pinned thread
|
|
* How does this relate to (currently unused) so_attach_reuseport_cbpf code?
|
|
|
|
## Low priority
|
|
|
|
* aquatic_udp
|
|
* what poll event capacity is actually needed?
|
|
* load test
|
|
* move additional request sending to for each received response, maybe
|
|
with probability 0.2
|
|
|
|
* aquatic_ws
|
|
* large amount of temporary allocations in serialize_20_bytes, pretty many in deserialize_20_bytes
|
|
|
|
* extract response peers: extract "one extra" to compensate for removal,
|
|
of sender if present in selection?
|
|
|
|
# Not important
|
|
|
|
* aquatic_http:
|
|
* consider better error type for request parsing, so that better error
|
|
messages can be sent back (e.g., "full scrapes are not supported")
|
|
* test torrent transfer with real clients
|
|
* scrape: does it work (serialization etc), and with multiple hashes?
|
|
* 'left' optional in magnet requests? Probably not. Transmission sends huge
|
|
positive number.
|
|
|
|
* aquatic_ws
|
|
* write new version of extract_response_peers which checks for equality with
|
|
peer sending request???
|
|
|
|
# Don't do
|
|
|
|
* general: PGO didn't seem to help way back
|
|
|
|
## aquatic_http
|
|
* request from path:
|
|
* deserialize 20 bytes: possibly rewrite (just check length of underlying
|
|
bytes == 20 and then copy them), also maybe remove String from map for
|
|
these cases too. doesn't really improve performance
|
|
* crazy http parsing: check for newline with memchr, take slice until
|
|
there. then iter over space newlines/just take relevant data. Not faster
|
|
than httparse and a lot worse
|
|
|
|
## aquatic_udp_protocol
|
|
* Use `bytes` crate: seems to worsen performance somewhat
|
|
* Zerocopy (https://docs.rs/zerocopy/0.3.0/zerocopy/index.html) for requests
|
|
and responses. Doesn't improve performance
|