Mirror/MarginaliaSearch
1
0
Fork 0
mirror of https://github.com/MarginaliaSearch/MarginaliaSearch.git synced 2025-02-23 21:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
6a1bfd6270
MarginaliaSearch/third-party/openzim
History
Viktor Lofgren 27ffb8fa8a (converter) Integrate zim->db conversion into automatic encyclopedia processing workflow 
Previously, in order to load encyclopedia data into the search engine, it was necessary to use the encyclopedia.marginalia.nu converter to first create a .db-file.  This isn't very ergonomic, so parts of that code-base was lifted in as a 3rd party library, and conversion from .zim to .db is now done automatically.

The output file name is based on the original filename, plus a crc32 hash and a .db-ending, to ensure we can recycle the data on repeat loads.
2024-01-19 13:59:03 +01:00
..
src/main/java/org/openzim (converter) Integrate zim->db conversion into automatic encyclopedia processing workflow 2024-01-19 13:59:03 +01:00
build.gradle (*) Upgrade to JDK21 with preview enabled. 2023-09-24 10:38:59 +02:00
readme.md The refactoring will continue until morale improves. 2023-03-12 10:04:48 +01:00

readme.md

OpenZIM

OpenZIM - GPL-2.0

OpenZIM is a ZIM file reader. This code has been modified in a fairly crude manner to be much faster than the original code base which seems quite antique. It also supports XZ compression.

Important Note the license is incompatible with AGPL 3, so we can't link Marginalia directly to this. It's still very useful for building tools that deal with wikipedia data which would be stand-alone.