4 Commits

Author	SHA1	Message	Date
Viktor Lofgren	6efc0f21fe	(index) Clean up data model ... The change set cleans up the data model for the term-level data. This used to contain a bunch of fields with document-level metadata. This data-duplication means a larger memory footprint and worse memory locality. The ranking code is also modified to not accept SearchResultKeywordScores, but rather CompiledQueryLong and CqDataInts containing only the term metadata and the frequency information needed for ranking. This is again an effort to improve memory locality.	2024-04-24 14:44:39 +02:00
Viktor Lofgren	150ee21f3c	(ngram) Clean up ngram lexicon code ... This is both an optimization that removes some GC churn, as well as a clean-up of the code that removes references to outdated concepts.	2024-04-24 14:44:38 +02:00
Viktor Lofgren	7dd8c78c6b	(ngrams) Remove the vestigial logic for capturing permutations of n-grams ... The change also reduces the object churn in NGramLexicon, as this is a very hot method in the converter.	2024-04-24 14:44:38 +02:00
Viktor Lofgren	6a7a7009c7	(convert) Initial integration of segmentation data into the converter's keyword extraction logic	2024-04-24 14:44:17 +02:00