Viktor Lofgren 0ca43f0c9c (live-crawler) Improve live crawler short-circuit logic ... We should not wait until we've fetched robots.txt to decide whether we have any data to fetch! This makes the live crawler very slow and leads to unnecessary requests.		2024-12-27 20:54:42 +01:00
..
java/nu/marginalia/livecrawler	(live-crawler) Improve live crawler short-circuit logic	2024-12-27 20:54:42 +01:00
test/nu/marginalia/livecrawler	(live-crawler) Improve live crawler short-circuit logic	2024-12-27 20:54:42 +01:00
build.gradle	(refac) Move export tasks to a process and clean up process initialization for all ProcessMainClass descendents	2024-11-21 16:00:09 +01:00