MarginaliaSearch/code/processes/live-crawling-process
Viktor Lofgren 927bc0b63c (live-crawler) Add Accept-Encoding: gzip to outbound requests
This change adds `Accept-Encoding: gzip` to all outbound requests from the live crawler and feed fetcher, and the corresponding decoding logic for the compressed response data.

The change addresses issue #136, save for making the fetcher's requests conditional.
2024-12-27 03:59:34 +01:00
..
java/nu/marginalia/livecrawler (live-crawler) Add Accept-Encoding: gzip to outbound requests 2024-12-27 03:59:34 +01:00
test/nu/marginalia/livecrawler (live-crawler) Add Accept-Encoding: gzip to outbound requests 2024-12-27 03:59:34 +01:00
build.gradle (refac) Move export tasks to a process and clean up process initialization for all ProcessMainClass descendents 2024-11-21 16:00:09 +01:00