aquatic_ws: mio: replace native_tls with rustls, rewrite connection state logic completely (#38)

* ws: mio: use rustls, rewrite Connection logic

* ws: mio: improve poll register/deregister handling

* ws: mio: work on type-level poll registry safety

* ws: mio: use stronger poll registry type-level guarantees

* ws: mio: fix stream reading

* ws: clean up, run fmt

* ws: mio: don't require registered connection for writing, improve docs

* ws: mio: add Connection::get_meta(), make Connection::meta private

* ws: mio: add ConnectionMap struct; remove utils.rs

* ws: mio: move token counter into ConnectionMap, improve docs

* ws: mio: connection: move Connection struct above state structs

* Update TODO

* ws: fix build errors

* ws: upgrade to tungstenite 0.16

* ws load test: don't panic on Close message; print shorter errors

* ws: fix socket worker bugs, add log statements

* ws: mio: wait for write availability if would block for ws messages

* Update README

* ws: mio: limit channels & queues; read 1 message only; other fixes

* ws: mio: send local responses each event; decrease channel size

* Update TODO

* ws: mio: limit ws send queue, fixing memory leak; limit pending messages

Also change some log output levels and run rustfmt

* Update TODO

* Update TODO

TODO.md

@@ -40,6 +40,15 @@
     messages can be sent back (e.g., "full scrapes are not supported")
 
 * aquatic_ws
+  * mio
+    * shard torrent state. this could decrease dropped messages too, since
+      request handlers won't send large batches of them
+    * connection cleaning interval
+    * use access list cache
+    * use write event interest for handshakes too
+    * deregistering before closing is required by mio, but it hurts performance
+      * blocked on https://github.com/snapview/tungstenite-rs/issues/51
+    * connection closing: send tls close message etc?
   * glommio
     * proper cpu set pinning
     * RES memory still high after traffic stops, even if torrent maps and connection slabs go down to 0 len and capacity