mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-23 21:18:58 +00:00
a18edad04c
We want to index all words in the document, stopword handling is moved to the index where we change the semantics to elide inclusion checks in query construction for a very short list of words tentatively hard-coded in SearchTerms. |
||
|---|---|---|
| .. | ||
| java/nu/marginalia/language | ||
| resources/dictionary | ||
| test/nu/marginalia/language | ||
| test-resources/html | ||
| build.gradle | ||
| readme.md | ||
Language Processing
This library contains various tools used in language processing.
Central Classes
- SentenceExtractor - Creates a DocumentLanguageData from a text, containing its words, how they stem, POS tags, and so on.
See Also
features-convert/keyword-extraction uses this code to identify which keywords are important.