Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-24 13:19:02 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
b510b7feb8
MarginaliaSearch/code/processes/crawling-process/model/java/nu/marginalia/slop
History
Viktor Lofgren b510b7feb8 Spike for storing crawl data in slop instead of parquet 
This seems to reduce RAM overhead to 100s of MB (from ~2 GB), as well as roughly double the read speeds.  On disk size is virtually identical.
2024-12-15 15:49:47 +01:00
..
SlopCrawlDataRecord.java Spike for storing crawl data in slop instead of parquet 2024-12-15 15:49:47 +01:00