MarginaliaSearch/code/libraries
2024-08-03 10:32:46 +02:00
..
array (wip) Extract and encode spans data 2024-07-27 11:44:13 +02:00
blocking-thread-pool (build) Java 22 and its consequences has been a disaster for Marginalia Search 2024-04-24 13:54:04 +02:00
braille-block-punch-cards (build) Java 22 and its consequences has been a disaster for Marginalia Search 2024-04-24 13:54:04 +02:00
btree (cleanup) Remove next-prime library only used in tests 2024-07-17 13:48:03 +02:00
coded-sequence (index) Experimental ranking signals 2024-08-03 10:32:46 +02:00
easy-lsh (build) Java 22 and its consequences has been a disaster for Marginalia Search 2024-04-24 13:54:04 +02:00
geo-ip (build) Java 22 and its consequences has been a disaster for Marginalia Search 2024-04-24 13:54:04 +02:00
guarded-regex (build) Java 22 and its consequences has been a disaster for Marginalia Search 2024-04-24 13:54:04 +02:00
language-processing (index) Add span information for anchor tags, tweak ranking params 2024-08-01 11:46:30 +02:00
message-queue (*) Lift jetty and guava-dependencies 2024-05-23 14:20:01 +02:00
random-write-funnel (build) Java 22 and its consequences has been a disaster for Marginalia Search 2024-04-24 13:54:04 +02:00
slop (restructure) Clean up repo by moving stray features into converter-process and crawler-process 2024-07-30 10:14:00 +02:00
term-frequency-dict (*) Lift jetty and guava-dependencies 2024-05-23 14:20:01 +02:00
test-helpers (wip) Extract and encode spans data 2024-07-27 11:44:13 +02:00
LICENSE.txt The refactoring will continue until morale improves. 2023-03-12 10:50:31 +01:00
readme.md (big-string) Remove the unused bigstring library 2024-05-18 13:40:03 +02:00

Libraries

These are libraries that are not strongly coupled to the search engine's business logic. These libraries may not depend on features, services, processes, models, etc.

NOTE: These libraries are co-licensed under the MIT license.

Libraries

  • The array library is for memory mapping large memory-areas, which Java has bad support for. It's designed to be able to easily replaced when Java's Foreign Function And Memory API is released.
  • The btree library offers a static BTree implementation based on the array library.
  • language-processing contains primitives for sentence extraction and POS-tagging.
  • The message-queue library.

Micro libraries

  • easy-lsh is a simple locality-sensitive hash for document deduplication
  • guarded-regex makes predicated regular expressions clearer
  • random-write-funnel is a tool for reducing write amplification when constructing large files out of order.
  • next-prime naive brute force prime sieve.
  • braille-block-punch-cards renders bit masks into human-readable dot matrices using the braille block.