MarginaliaSearch/crawl/crawl-job-extractor-process/readme.md
2023-03-06 18:32:13 +01:00

4 lines
178 B
Markdown

# Crawl Job Extractor
The crawl job extractor creates a file containing a list of domains
along with known URLs. This is consumed by the [crawling-process](../crawling-process).