MarginaliaSearch/code/functions/live-capture
Viktor Lofgren a91ab4c203 (live-crawler) Crude first-try process for live crawling #WIP
Some refactoring is still needed, but an dummy actor is in place and a process that crawls URLs from the livecapture service's RSS endpoints; that makes it all the way to being indexable.
2024-11-19 19:35:01 +01:00
..
api (live-crawler) Crude first-try process for live crawling #WIP 2024-11-19 19:35:01 +01:00
java/nu/marginalia (rss) Add endpoint for extracting URLs changed withing a timespan. 2024-11-18 14:59:32 +01:00
test/nu/marginalia (rss) Add endpoint for extracting URLs changed withing a timespan. 2024-11-18 14:59:32 +01:00
build.gradle (feed) Use the message queue to permit the feeds service to tell the calling actor when it's finished 2024-11-10 17:45:20 +01:00
readme.md (live-capture) Add readme to live-capture function 2024-09-28 11:35:46 +02:00

This is a subsystem that allows on-demand screenshot capture of a website.

It uses the local browserless API to capture data. To use this module, you must have a browserless docker container running on machine, and then set the live-capture.browserless-uri system property to the address of the browserless container (e.g http://my-container:3000/).

When disabled, the subsystem will acknowledge the request, but will not act on it.

The module will only enable on the primary node of a service to simplify dealing with race conditions and duplicate requests.