mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-24 05:18:58 +00:00
c73e43f5c9
In the scenario where an operator * Performs a new crawl from spec * Doesn't load the data into the index * Recrawls the data The recrawl will not find the domains in the database, and the crawl log will be overwritten with an empty file, irrecoverably losing the crawl log making it impossible to load! To mitigate the impact similar problems, the change saves a backup of the old crawl log, as well as complains about this happening. More specifically to this exact scenario however, the parquet-loaded domains are also preemptively inserted into the domain database at the start of the crawl. This should help the DbCrawlSpecProvider to find them regardless of loaded state. This may seem a bit redundant, but losing crawl data is arguably the worst type of disaster scenario for this software, so it's arguably merited. |
||
|---|---|---|
| .. | ||
| assistant-service | ||
| control-service | ||
| executor-service | ||
| index-service | ||
| query-service | ||
| readme.md | ||
Core Services
The cores services constitute the main functionality of the search engine, relatively agnostic to the Marginalia application.
-
The index-service contains the indexes, it answers questions about which documents contain which terms.
-
The query-service Interprets queries and delegates work to index-service.
-
The control-service provides an operator's user interface, and is responsible for orchestrating the various processes of the system.
-
The assistant-service helps the search service with spelling suggestions other peripheral functionality.