mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-24 05:18:58 +00:00
aebb2652e8
Refactoring keyword extraction to extract spans information. Modifying the intermediate storage of converted data to use the new slop library, which is allows for easier storage of ad-hoc binary data like spans and positions. This is a bit of a katamari damacy commit that ended up dragging along a bunch of other fairly tangentially related changes that are hard to break out into separate commits after the fact. Will push as-is to get back to being able to do more isolated work. |
||
|---|---|---|
| .. | ||
| api | ||
| java/nu/marginalia | ||
| test/nu/marginalia/svc | ||
| build.gradle | ||
| readme.md | ||
The execution subsystem is responsible for the execution of long running tasks on each index node. It lives in the executor-service module.
It accomplishes this using the message queue and actor library, which permits program state to survive crashes and reboots.
The subsystem exposes four APIs:
- Execution API - for starting and stopping tasks, also contains miscellaneous commands
- Crawl API - for managing the crawl workflow
- Sideload API - for sideloading data
- Export API - for exporting data