Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-24 05:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
eb65daaa88
MarginaliaSearch/code/processes/crawling-process/java/nu/marginalia/crawl
History
Viktor Lofgren fb673de370 (crawler) Change the header 'User-agent' to 'User-Agent'
2025-01-28 15:34:16 +01:00
..
fetcher (crawler) Change the header 'User-agent' to 'User-Agent' 2025-01-28 15:34:16 +01:00
logic (crawler) Clean up the crawler code a bit, removing vestigial abstractions and historical debris 2024-10-15 17:27:59 +02:00
retreival (crawler) Reduce long retention of CrawlDataReference objects and their associated SerializableCrawlDataStreams 2025-01-26 15:40:17 +01:00
warc (refac) Move export tasks to a process and clean up process initialization for all ProcessMainClass descendents 2024-11-21 16:00:09 +01:00
CrawlerMain.java (crawler) Reduce long retention of CrawlDataReference objects and their associated SerializableCrawlDataStreams 2025-01-26 15:40:17 +01:00
CrawlerModule.java (chore) Remove lombok 2024-11-11 21:14:38 +01:00
DomainStateDb.java (crawler) Add default CT when it's missing for icons 2025-01-22 13:55:47 +01:00