Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-23 21:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
6c3abff664
MarginaliaSearch/code/features-crawl
History
Viktor Lofgren a6b03a66dc (crawl) Reduce Charset.forName() object churn 
Cache the Charset object returned from Charset.forName() for future use, since we're likely to see the same charset again and Charset.forName(...) can be surprisingly expensive and its built-in caching strategy, which just caches the 2 last values seen doesn't cope well with how we're hitting it with a wide array of random charsets
2024-07-04 20:49:07 +02:00
..
content-type (crawl) Reduce Charset.forName() object churn 2024-07-04 20:49:07 +02:00
crawl-blocklist (*) Lift jetty and guava-dependencies 2024-05-23 14:20:01 +02:00
link-parser (*) Lift jetty and guava-dependencies 2024-05-23 14:20:01 +02:00
readme.md (doc) Update docs 2024-02-06 16:29:55 +01:00

readme.md

Crawl Features

These are bits of search-engine related code that are relatively isolated pieces of business logic, that benefit from the clarity of being kept separate from the rest of the crawling code.