Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-24 05:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
0caef1b307
MarginaliaSearch/code/services-core
History
Viktor Lofgren 0caef1b307 (warc) Toggle for saving WARC data 
Add a toggle for saving the WARC data generated by the search engine's crawler.  Normally this is discarded, but for debugging or archival purposes, retaining it may be of interest.

The warc files are concatenated into larger archives, up to about 1 GB each.
An index is also created containing filenames, domain names, offsets and sizes
to help navigate these larger archives.

The warc data is saved in a directory warc/ under the crawl data storage.
2024-01-12 13:45:14 +01:00
..
assistant-service (*) install script for deploying Marginalia outside the codebase 2024-01-11 12:40:03 +01:00
control-service (warc) Toggle for saving WARC data 2024-01-12 13:45:14 +01:00
executor-service (control) UX-improvements for control service 2024-01-12 12:33:05 +01:00
index-service (*) install script for deploying Marginalia outside the codebase 2024-01-11 12:40:03 +01:00
query-service (*) install script for deploying Marginalia outside the codebase 2024-01-11 12:40:03 +01:00
readme.md (refactor) Move search service into services-satellite 2023-10-09 13:40:01 +02:00

readme.md

Core Services

The cores services constitute the main functionality of the search engine, relatively agnostic to the Marginalia application.

  • The index-service contains the indexes, it answers questions about which documents contain which terms.

  • The query-service Interprets queries and delegates work to index-service.

  • The control-service provides an operator's user interface, and is responsible for orchestrating the various processes of the system.

  • The assistant-service helps the search service with spelling suggestions other peripheral functionality.