Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-24 21:29:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
47e58a21c6
MarginaliaSearch/code/processes/crawling-process/model/java/nu/marginalia/io
History
Viktor Lofgren 3714104976 Add loader for slop data in converter. 
Also alter CrawledDocument to not require String parsing of the underlying byte[] data.  This should reduce the number of large memory allocations quite significantly, hopefully reducing the GC churn a bit.
2024-12-17 15:40:24 +01:00
..
crawldata/format Add loader for slop data in converter. 2024-12-17 15:40:24 +01:00
CrawledDomainReader.java Switch to new Slop format for crawl data storage and processing. 2024-12-15 19:34:03 +01:00
CrawlerOutputFile.java Switch to new Slop format for crawl data storage and processing. 2024-12-15 19:34:03 +01:00
SerializableCrawlDataStream.java Add loader for slop data in converter. 2024-12-17 15:40:24 +01:00