Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-24 05:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
78f21dd19a
MarginaliaSearch/code/processes/crawling-process
History
Viktor Lofgren 24dce8c03b Remove link filtering for mediawiki, it's too strict and not every site uses the /wiki/-pattern.
2023-07-01 19:32:25 +02:00
..
src Remove link filtering for mediawiki, it's too strict and not every site uses the /wiki/-pattern. 2023-07-01 19:32:25 +02:00
build.gradle Tests for crawler specialization + testdata 2023-06-27 10:57:54 +02:00
readme.md More restructuring, big bug fixes in keyword extraction. 2023-03-13 17:39:53 +01:00

readme.md

Crawling Process

The crawling process downloads HTML and saves them into per-domain snapshots.

Central Classes

See Also