# aquatic-crawler

![Build](https://github.com/YGGverse/aquatic-crawler/actions/workflows/build.yml/badge.svg)
[![Dependencies](https://deps.rs/repo/github/YGGverse/aquatic-crawler/status.svg)](https://deps.rs/repo/github/YGGverse/aquatic-crawler)
[![crates.io](https://img.shields.io/crates/v/aquatic-crawler.svg)](https://crates.io/crates/aquatic-crawler)

SSD-friendly FS crawler for the [Aquatic](https://github.com/greatest-ape/aquatic) BitTorrent tracker, based on the [librqbit](https://github.com/ikatson/rqbit/tree/main/crates/librqbit) API

> [!NOTE]
> * requires PR#233 or the [info-hash-api](https://github.com/YGGverse/aquatic/tree/info-hash-api) branch implementation
> * compatible with any other `--infohash` source in `hash1hash2...` binary format
> * use [βtracker](https://github.com/YGGverse/btracker/) for the frontend
> * see the project [Wiki](https://github.com/YGGverse/aquatic-crawler/wiki) for details

## Conception

```
torrent client > aquatic_udp > infohash.bin < aquatic-crawler > * /preload/info-hash.torrent
torrent client               <-----------------------|          * /preload/info-hash/data
               <-------------------------------------|          * /preload/.info-hash/tmp
```

## Install

> [!TIP]
> You may want to install some [system dependencies](https://github.com/YGGverse/aquatic-crawler/wiki/Dependencies)

1. `git clone https://github.com/YGGverse/aquatic-crawler.git && cd aquatic-crawler`
2. `cargo build --release`
3. `sudo install target/release/aquatic-crawler /usr/local/bin/aquatic-crawler`

## Usage

> [!TIP]
> * prepend `RUST_LOG=debug` to debug, append `NO_COLOR=1` to disable fmt
> * use `--preload-*` arguments group to avoid preloading everything (by default)
> * as the crawler is running in a single thread, use `--peer-read-write-timeout` (in seconds) to skip long-lasting connections. Content preload will be restored in the next crawl cycle after resolving other torrents from active peers in the queue.

``` bash
aquatic-crawler --infohash /path/to/info-hash-ipv4.bin\
                --infohash /path/to/info-hash-ipv6.bin\
                --infohash /path/to/another-source.bin\
                --tracker  udp://host1:port\
                --tracker  udp://host2:port\
                --preload  /path/to/directory
```

### Options

``` bash
aquatic-crawler --help
```