Welcome to the aquatic-crawler wiki!
Conception
torrent client > aquatic_udp > infohash.bin < aquatic-crawler > * /preload/info-hash.torrent
torrent client <-----------------------| * /preload/info-hash/data
<-------------------------------------| * /preload/.info-hash/tmp
/preload/.info-hash/tmp is data pending for preload (auto-removed on complete)
How to use
- Run the aquatic_udp tracker with binary export enabled.
- Add this tracker to your BitTorrent client (and share with others).
- Run the aquatic-crawler connected to the
aquatic_udp binary API.
- Export the resolved aggregation results to any supported format, such as RSS.
- Subscribe to RSS feed updates in your favorite client.
Goals
SSD-friendly
The crawler is designed to use an in-memory index; it writes data to disk only when specified options are requested and only when the data needs to be updated (through additional read-only filters).
Memory-safe
This crawler also includes limitations on index length and member values, based on the custom argument options or its defaults.
Privacy-oriented
Exported data values, such as RSS content, are sanitized for BOM characters, HTML entities, and XSS injections to ensure safe use in clients without additional setup. However, this project is still in development, so please use it with caution, or make your revision and report any issues found.