23 commits

Author	SHA1	Message	Date
yggverse	1bee7daf77	apply missed db updates	2026-01-11 21:02:35 +02:00
yggverse	2b804d8915	normalize db tables, optionally persist channel descriptions, remove entries logic from the crawler, update config options	2026-01-11 20:36:00 +02:00
yggverse	b34d7cdcdd	implement allowed_tags config option, format config, update documentation comments	2026-01-10 22:48:45 +02:00
yggverse	5570049588	implement html sanitizing	2026-01-10 21:25:46 +02:00
yggverse	e86b241ee6	implement local image features	2026-01-10 17:53:48 +02:00
yggverse	3e94399ccb	use config file instead of argument options	2026-01-10 16:37:28 +02:00
yggverse	ec0cca64f3	implement persist_images_selector, minimize codebase by using bail, change image table structure to use sha256 hash as the unique image identity	2026-01-10 14:38:01 +02:00
yggverse	bc61b5c09c	move config.toml example into crate root	2026-01-10 11:53:22 +02:00
yggverse	ee083dfc45	optimize db api	2026-01-10 01:41:20 +02:00
yggverse	f48e256fad	separate Pollable and Transactional features, separate table members, use single-connection transactions method in the crawler and llm crates, minor crawler optimizations such as disconnect from db server on each queue iteration complete	2026-01-09 22:35:06 +02:00
yggverse	de3fda435a	implement initial llm features	2026-01-08 14:05:16 +02:00
yggverse	aaad4fd49d	update systemd option example	2026-01-08 00:57:25 +02:00
yggverse	e26236a591	rename table	2026-01-08 00:42:26 +02:00
yggverse	eedc9c06fc	remove self mutable dependency as pool	2026-01-07 22:22:08 +02:00
yggverse	9269ec2a9e	add readme	2026-01-07 20:53:37 +02:00
yggverse	feb3a2d519	add comment	2026-01-07 19:30:12 +02:00
yggverse	776de04c1d	skip processed channel items	2026-01-07 19:27:36 +02:00
yggverse	259ac118dc	fix channel item links with its data type	2026-01-07 19:17:47 +02:00
yggverse	c0734731cb	disallow nullable title/description values for the content table, implement contents_by_channel_item_id_source_id, return last insert id for insert_content, fix content_id data type, implement initial content version save on crawl	2026-01-07 19:02:00 +02:00
yggverse	d8f2d723f5	handle some 3'th party errors, init scraper features	2026-01-07 18:25:21 +02:00
yggverse	63373609da	fix default port	2026-01-07 16:13:57 +02:00
yggverse	d669f1ba78	add comments	2026-01-07 15:56:25 +02:00
yggverse	e070db316c	initial commit	2026-01-07 15:00:55 +02:00