mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-23 13:09:00 +00:00
3101b74580
This is a system-wide change. The index used to have a lexicon, mapping words to wordIds using a large in-memory hash table. This made index-construction easier, but it also added a fairly significant RAM penalty to both the index service and the loader. The new design moves to 64 bit word identifiers calculated using the murmur hash of the keyword, and an index construction based on merging smaller indices. It also became necessary half-way through to upgrade guice as its error reporting wasn't *quite* compatible with JDK20. |
||
|---|---|---|
| .. | ||
| api | ||
| common | ||
| features-convert | ||
| features-crawl | ||
| features-index | ||
| features-search | ||
| libraries | ||
| process-models | ||
| processes | ||
| services-core | ||
| services-satellite | ||
| tools | ||
| readme.md | ||
Code
This is a pretty large and diverse project with many moving parts.
You'll find a short description in each module of what it does and how it relates to other modules. The modules each have names like "library" or "process" or "feature". These have specific meanings. See doc/module-taxonomy.md.
Overview
A map of the most important components and how they relate can be found below.
Services
- core services "macroservices", stateful, memory hungry doing heavy lifting.
- satellite services "microservices", stateless providing additional functionality.
-
- api - public API
- an internal API
Processes
Processes are batch jobs that deal with data retrieval, processing and loading.
Tools
Features
Features are relatively stand-alone components that serve some part of the domain. They aren't domain-independent, but isolated.
Libraries and primitives
Libraries are stand-alone code that is independent of the domain logic.