mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-23 21:18:58 +00:00
51e46ad2b0
Since some of the export tasks have been memory hungry, sometimes killing the executor-services, they've been moved to a separate process that can be given a larger Xmx. While doing this, the ProcessMainClass was given utilities for the boilerplate surrounding receiving mq requests and responding to them, some effort was also put toward making the process boot process a bit more uniform. It's still a bit heterogeneous between different processes, but a bit less so for now. |
||
|---|---|---|
| .. | ||
| api | ||
| java/nu/marginalia | ||
| test/nu/marginalia/svc | ||
| build.gradle | ||
| readme.md | ||
The execution subsystem is responsible for the execution of long running tasks on each index node. It lives in the executor-service module.
It accomplishes this using the message queue and actor library, which permits program state to survive crashes and reboots.
The subsystem exposes four APIs:
- Execution API - for starting and stopping tasks, also contains miscellaneous commands
- Crawl API - for managing the crawl workflow
- Sideload API - for sideloading data
- Export API - for exporting data