From 9269ec2a9e354b9b7dc40121b065e2e392f96788 Mon Sep 17 00:00:00 2001 From: yggverse Date: Wed, 7 Jan 2026 20:53:37 +0200 Subject: [PATCH] add readme --- README.md | 7 ++++- crates/crawler/README.md | 58 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+), 1 deletion(-) create mode 100644 crates/crawler/README.md diff --git a/README.md b/README.md index f114474..9b14499 100644 --- a/README.md +++ b/README.md @@ -7,4 +7,9 @@ Convert RSS feeds into multiple formats > [!NOTE] -> Branch in development! \ No newline at end of file +> Branch in development! + +## Components + +* `rssto-mysql` - shared database library +* `rssto-crawler` - RSS feed reader and data scrapper daemon \ No newline at end of file diff --git a/crates/crawler/README.md b/crates/crawler/README.md new file mode 100644 index 0000000..9ea940f --- /dev/null +++ b/crates/crawler/README.md @@ -0,0 +1,58 @@ +# rssto-crawler + +## Install + +``` bash +git clone https://github.com/YGGverse/rssto.git +cd rssto +cargo build --release +``` + +## Launch + +``` bash +rssto-crawler -c config/example.toml +``` +> [!TIP] +> * prepend `RUST_LOG=rssto_crawler=trace` to print worker details (supported [levels](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.LevelFilter.html)) +> * or just `RUST_LOG=trace` to debug all components in use +> * append `-u TIME` to run as the daemon with `TIME` interval update +> * see `rssto-crawler --help` to print all available options + +### Systemd + +1. Install `rssto-crawler` by copy the binary compiled into the native system apps destination: + * Linux: `sudo install target/release/rssto-crawler /usr/local/bin/rssto-crawler` +2. Create `systemd` configuration file at `/etc/systemd/system/rssto-crawler.service`: + +``` rssto-crawler.service +[Unit] +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple + +User=rssto +Group=rssto + +# Uncomment for debug +# Environment="RUST_LOG=DEBUG" +# Environment="NO_COLOR=1" + +ExecStart=/usr/local/bin/rssto-crawler -c /path/to/config.toml + +StandardOutput=file:///home/rssto/crawler-debug.log +StandardError=file:///home/rssto/crawler-error.log + +[Install] +WantedBy=multi-user.target +``` +* example above requires new system user (`useradd -m rssto`) + +3. Run in priority: + +* `systemctl daemon-reload` - reload systemd configuration +* `systemctl enable rssto-crawler` - enable new service +* `systemctl start rssto-crawler` - start the process +* `systemctl status rssto-crawler` - check process launched \ No newline at end of file