Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-23 13:09:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4c2810720a
MarginaliaSearch/code/index/index-journal
History
Viktor Lofgren 9f47ce8d15 (chore) Remove lombok 
There are likely some instances of delombok gore with this commit.
2024-11-11 21:14:38 +01:00
..
java/nu/marginalia/index/journal (chore) Remove lombok 2024-11-11 21:14:38 +01:00
build.gradle (slop) Break slop out into its own repository 2024-08-13 09:50:05 +02:00
readme.md (doc) Correct dead links and stale information in the docs 2024-09-13 11:01:05 +02:00

readme.md

Index Journal

The index journal contains a list of entries with keywords and keyword metadata per document.

This journal is written by processes/loading-process and read when constructing the forward and reverse indices.

The journal uses the Slop library to store data in a columnar fashion.

The journal will may be split into multiple files to help index construction, as a merge strategy is used to reduce the amount of RAM required during index construction.

Unlike most slop data stores, the index journal allows direct access to the underlying columns, as the needs of the index construction processes are fairly varied.