Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-24 05:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4a8afa6b9f
MarginaliaSearch/code/index/index-journal
History
Viktor Lofgren 4a8afa6b9f (index, WIP) Position data partially integrated with forward and reverse indexes. 
There's no graceful way of doing this in small commits, pushing to avoid the risk of data loss.
2024-06-06 12:54:52 +02:00
..
java/nu/marginalia/index/journal (index, WIP) Position data partially integrated with forward and reverse indexes. 2024-06-06 12:54:52 +02:00
test/nu/marginalia/index/journal (index, WIP) Position data partially integrated with forward and reverse indexes. 2024-06-06 12:54:52 +02:00
build.gradle (index, WIP) Position data partially integrated with forward and reverse indexes. 2024-06-06 12:54:52 +02:00
readme.md (docs) Begin un-fucking the docs after refactoring 2024-02-27 21:22:21 +01:00

readme.md

Index Journal

The index journal contains a list of entries with keywords and keyword metadata per document.

This journal is written by processes/loading-process and read when constructing the forward and reverse indices.

The journal format is a file header, followed by a zstd-compressed list of entries, each containing a header with document-level data, and a data section with keyword-level data.

The journal data may be split into multiple files, and the journal writers and readers are designed to handle this transparently via their Paging implementation.

Central Classes

Model

I/O