Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-23 21:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2ef66ce0ca
MarginaliaSearch/code/execution
History
Viktor Lofgren 2ef66ce0ca (actor) Reset NEW flag earlier when auto-deletion is disabled 
Don't wait until the loader step is finished to reset the NEW flag, as this leaves manually processed (but not yet loaded) crawl data stuck in "CREATING" in the GUI.
2024-07-31 10:31:03 +02:00
..
api (crawl) Add new functionality for re-crawling a single domain 2024-07-05 15:31:55 +02:00
data-extractors (restructure) Clean up repo by moving stray features into converter-process and crawler-process 2024-07-30 10:14:00 +02:00
java/nu/marginalia (actor) Reset NEW flag earlier when auto-deletion is disabled 2024-07-31 10:31:03 +02:00
test/nu/marginalia/svc (refac) Remove src/main from all source code paths. 2024-02-23 16:13:40 +01:00
build.gradle (restructure) Clean up repo by moving stray features into converter-process and crawler-process 2024-07-30 10:14:00 +02:00
readme.md Clean up documentation and rename domain-links to link-graph 2024-02-28 11:40:39 +01:00

readme.md

The execution subsystem is responsible for the execution of long running tasks on each index node. It lives in the executor-service module.

It accomplishes this using the message queue and actor library, which permits program state to survive crashes and reboots.

The subsystem exposes four APIs:

  • Execution API - for starting and stopping tasks, also contains miscellaneous commands
  • Crawl API - for managing the crawl workflow
  • Sideload API - for sideloading data
  • Export API - for exporting data