mirror of
https://github.com/YGGverse/aquatic-crawler.git
synced 2026-03-31 17:15:35 +00:00
89 lines
No EOL
2.5 KiB
Markdown
89 lines
No EOL
2.5 KiB
Markdown
# aquatic-crawler
|
|
|
|
|
|
[](https://deps.rs/repo/github/YGGverse/aquatic-crawler)
|
|
[](https://crates.io/crates/aquatic-crawler)
|
|
|
|
Crawler/aggregation tool for the [Aquatic](https://github.com/greatest-ape/aquatic) BitTorrent tracker API.
|
|
|
|
> [!NOTE]
|
|
> Project in development!
|
|
|
|
## Roadmap
|
|
|
|
* Targets supported
|
|
* [x] IPv4/IPv6 info-hash JSON/API (requires [PR#233](https://github.com/greatest-ape/aquatic/pull/233))
|
|
* [x] local file path
|
|
* [ ] remote URL
|
|
* Storage
|
|
* [x] File system (dump as `.torrent`)
|
|
* [x] V1
|
|
* [ ] V2
|
|
* [ ] [Manticore](https://github.com/manticoresoftware/manticoresearch-rust) full text search
|
|
* [ ] SQLite
|
|
|
|
## Install
|
|
|
|
1. `git clone https://github.com/YGGverse/aquatic-crawler.git && cd aquatic-crawler`
|
|
2. `cargo build --release`
|
|
3. `sudo install target/release/aquatic-crawler /usr/local/bin/aquatic-crawler`
|
|
|
|
## Usage
|
|
|
|
``` bash
|
|
aquatic-crawler --infohash-source /path/to/info-hash-ipv4.json\
|
|
--infohash-source /path/to/info-hash-ipv6.json\
|
|
--infohash-source /path/to/another-source.json\
|
|
--torrent-tracker udp://host:port\
|
|
--torrents-path /path/to/storage
|
|
```
|
|
* all arguments are optional, to support multiple source and target drivers
|
|
* running without arguments does nothing!
|
|
|
|
### Options
|
|
|
|
``` bash
|
|
-d, --debug <DEBUG>
|
|
Debug level
|
|
|
|
* `e` - error * `i` - info
|
|
|
|
[default: ei]
|
|
|
|
-c, --clear
|
|
Clear previous index collected on crawl session start
|
|
|
|
-i, --infohash-source <INFOHASH_SOURCE>
|
|
Filepath(s) to the Aquatic tracker info-hash JSON/API
|
|
|
|
* PR#233 feature
|
|
|
|
--torrents-path <TORRENTS_PATH>
|
|
Directory path to store the `.torrent` files
|
|
|
|
--torrent-tracker <TORRENT_TRACKER>
|
|
Define custom tracker(s) to preload the `.torrent` files info
|
|
|
|
--disable-dht
|
|
Disable DHT resolver (useful with `torrent_tracker`)
|
|
|
|
--enable-upnp-port-forwarding
|
|
Enable UPnP
|
|
|
|
--enable-upload
|
|
Enable upload
|
|
|
|
--socks-proxy-url <SOCKS_PROXY_URL>
|
|
Use `socks5://[username:password@]host:port`
|
|
|
|
-s <SLEEP>
|
|
Crawl loop delay in seconds
|
|
|
|
[default: 300]
|
|
|
|
-h, --help
|
|
Print help (see a summary with '-h')
|
|
|
|
-V, --version
|
|
Print version
|
|
``` |