mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-24 13:19:02 +00:00
c41e68aaab
This commit also refactors the executor a bit, and introduces a new converter-feature called data-extractors for this class of jobs.
7 lines
496 B
Markdown
7 lines
496 B
Markdown
Contains converter-*like* extraction jobs that operate on crawled data to produce export files.
|
|
|
|
## Important classes
|
|
|
|
* [AtagExporter](src/main/java/nu/marginalia/extractor/AtagExporter.java) - extracts anchor texts from the crawled data.
|
|
* [FeedExporter](src/main/java/nu/marginalia/extractor/FeedExporter.java) - tries to find RSS/Atom feeds within the crawled data.
|
|
* [TermFrequencyExporter](src/main/java/nu/marginalia/extractor/TermFrequencyExporter.java) - exports the 'TF' part of TF-IDF. |