68 commits

Author	SHA1	Message	Date
ghost	38fbc32151	fix document fields update	2023-11-27 20:55:10 +02:00
ghost	08995e6199	randomize new pages queue	2023-11-27 20:24:46 +02:00
ghost	6a9117757b	reset http code to 404 on page index initiation	2023-11-27 19:44:14 +02:00
ghost	015221eafb	fix semaphore condition #5	2023-11-27 19:34:14 +02:00
ghost	a499c363f6	prevent multi-thread execution #5	2023-11-27 19:31:03 +02:00
ghost	2961045c76	implement index cleaner tool #5	2023-11-27 19:29:17 +02:00
ghost	02dd3649a7	add CURL options that prevent crawl queue stuck	2023-11-27 16:54:26 +02:00
ghost	349f26f5ea	update option name	2023-11-26 21:33:34 +02:00
ghost	133548a98c	fix url check conditions	2023-11-26 20:53:31 +02:00
ghost	dfb2c06738	add crc32url filter	2023-11-25 18:10:23 +02:00
ghost	192e45103d	add index settings support	2023-11-25 16:01:46 +02:00
ghost	875382c56e	implement FTP snaps	2023-11-25 03:19:54 +02:00
ghost	72f2fdaeca	change config location	2023-11-25 00:16:08 +02:00
ghost	c6e9ba9d09	implement local storage feature with tar.gz compression	2023-11-24 19:51:43 +02:00
ghost	dc807fe4d5	add url trim	2023-11-24 18:37:25 +02:00
ghost	01753b0557	add crawl queue delay support	2023-11-20 00:06:17 +02:00
ghost	13cf61b42c	fix debug output	2023-11-19 23:34:13 +02:00
ghost	7dfc800a67	initial commit	2023-11-19 23:00:51 +02:00