Commit Graph

2087 Commits

Author SHA1 Message Date
Viktor Lofgren
65b74f9cab (registry) Fix broken test 2024-05-23 14:15:01 +02:00
Viktor Lofgren
59ec70eb73 (*) Clean up code related to crawl parquet inspection 2024-05-22 12:55:08 +02:00
Viktor Lofgren
365229991b (control) Improve pagination for crawl data inspector 2024-05-21 19:44:48 +02:00
Viktor Lofgren
959a8e29ee (control) Improve pagination for crawl data inspector 2024-05-21 19:27:25 +02:00
Viktor Lofgren
197c82acd4 (control) Add filter functionality for crawl data inspector 2024-05-21 19:05:44 +02:00
Viktor Lofgren
9539fdb53c (control) Clean up UX for crawl data inspector 2024-05-21 18:27:24 +02:00
Viktor Lofgren
24bf29d369 (*) Upgrade opennlp and deprecate the monkey patched version of the code as it's no longer needed 2024-05-20 18:03:21 +02:00
Viktor Lofgren
17dc00d05f (control) Partial implementation of inspection utility for crawl data
Uses duckdb and range queries to read the parquet files directly from the index partitions.

UX is a bit rough but is in working order.
2024-05-20 18:02:46 +02:00
Viktor Lofgren
4fcd4a8197 (index) Refactor to reduce the level of indirection 2024-05-19 12:40:33 +02:00
Viktor Lofgren
daf2a8df54 (btree) Roll back optimization of queryDataWithIndex
It had been previously assumed that re-writing this function in the style of retain() would make it faster, but it had the opposite effect.

The reason why retain is so fast due to properties of the data that hold true when intersecting document lists, where long runs of adjacent documents are expected, but not when looking up the data associated with the already intersected documents, where the data is more sparse.
2024-05-19 11:29:28 +02:00
Viktor Lofgren
88997a1c4f (btree) Clean up code 2024-05-18 18:38:46 +02:00
Viktor Lofgren
d12c77305c (btree) Clean up code 2024-05-18 18:03:17 +02:00
Viktor Lofgren
ab4e2b222e (array) Fix broken benchmarks 2024-05-18 13:41:24 +02:00
Viktor Lofgren
b867eadbef (big-string) Remove the unused bigstring library 2024-05-18 13:40:03 +02:00
Viktor Lofgren
19163fa883 (array) Clean up the Array library
IntArray gets the YAGNI axe.   The array library had two implementations, one for longs which was used, and one for ints, which only ever saw bit rot.   Removing the latter, as all it ever did was clutter up the codebase and add technical debt.  If we need int arrays, we fork LongArray again (or add int capabilities to it)

Also cleaning up the interfaces, removing layers of redundant abstractions and adding javadocs.

Finally adding sz=2 specializations to the quick- and insertion sort algorithms.  It seems the JIT isn't optimizing these particularly well, this is an attempt to help it out a bit.
2024-05-18 13:23:06 +02:00
Viktor Lofgren
650f3843bb (array) Clean up search function jungle
Retire search functions that weren't used, including the native implementations.  Drop confusing suffixes on search function names.  Search functions no longer encode search misses as negative values.

Replaced binary search function with a branchless version that is much faster.

Cleaned up benchmark code.
2024-05-17 14:31:02 +02:00
Viktor Lofgren
9e766bc056 (array) Clean up search function jungle
Retire search functions that weren't used, including the native implementations.  Drop confusing suffixes on search function names.  Search functions no longer encode search misses as negative values.

Replaced binary search function with a branchless version that is much faster.

Cleaned up benchmark code.
2024-05-17 14:30:06 +02:00
Viktor Lofgren
48aff52e00 (array) Increase LongArray on-heap alignment to 16 bytes
This primarily affects benchmarks, making performance more consistent for the 128 bit operations, as the system mostly works with memory mapped data.
2024-05-16 19:12:36 +02:00
Viktor Lofgren
9d7616317e (array) Clean up native code a bit 2024-05-16 14:47:10 +02:00
Viktor Lofgren
d227a09fb1 (search) Extend paperdoll service mock with site info data and screenshots
It's a bit of a hack job but will do, random exploration is available but only through a "browse:random"-style query
2024-05-15 12:40:55 +02:00
Viktor Lofgren
f48cf77c4d (array, experimental) Add benchmark results for quicksort 2024-05-14 18:15:30 +02:00
Viktor Lofgren
3549be216f (array, experimental) Documentation for native algos 2024-05-14 17:43:05 +02:00
Viktor Lofgren
c3e3a3dbc5 (search) Fix problem list in clustered search results 2024-05-14 13:05:52 +02:00
Viktor Lofgren
55a7c1db00 (array, experimental) Call C++ helper methods to do some low level stuff a bit faster than is possible with Java 2024-05-14 12:54:14 +02:00
Viktor Lofgren
c837321df1 (search) Provide a notification when no search results are found. 2024-05-06 20:11:39 +02:00
Viktor Lofgren
af7f6b89ec (search) Delete vestigial stylesheet from the old design. 2024-05-06 19:52:29 +02:00
Viktor Lofgren
29a4d3df23 (search) Imrpove search-service paperdoll by mocking suggestions and news 2024-05-06 19:52:13 +02:00
Viktor
bcbb9afac0
Merge pull request #93 from MarginaliaSearch/accessibility-improvements
Accessibility improvements
2024-05-04 15:45:26 +02:00
Viktor Lofgren
7d1cafc070 (control) Add skip link for navigation in control GUI 2024-05-04 12:36:44 +02:00
Viktor Lofgren
5951c67a8b (search) Center the search results page 2024-05-04 12:23:21 +02:00
Viktor Lofgren
c454007730 (search) Increase contrast for some UI elements 2024-05-04 12:02:52 +02:00
Viktor Lofgren
4e49cca43d (search) Clean up SCSS code a bit 2024-05-04 11:58:54 +02:00
Viktor Lofgren
49a8c06095 (search) Improve contrast for text on random button 2024-05-04 11:51:19 +02:00
Viktor Lofgren
d01d9fa670 (search) Add screenreader-specific notification remark about when search results start. 2024-05-04 11:41:06 +02:00
Viktor Lofgren
a53a32f006 (search) Spell out website problems with "atomic elements" instead of having a hover that's inaccessible with keyboard navigation 2024-05-04 11:41:05 +02:00
Viktor Lofgren
3548d54cf6 (search) Add a screenreader-only alert when the search filters are updated to make it easier to understand what happens. 2024-05-04 11:41:04 +02:00
Viktor Lofgren
01f242ac7e (search) Add stylesheet class for screenreader-only items 2024-05-04 11:41:03 +02:00
Viktor Lofgren
2840d9d403 (search) Add screenreader-only positions count text to search results 2024-05-04 11:41:03 +02:00
Viktor Lofgren
9fecfc5025 (search) Add autocomplete attribute to search-form 2024-05-04 11:41:02 +02:00
Viktor Lofgren
1b901e01f2 (search) Add bypass link that skips navigation 2024-05-04 11:41:01 +02:00
Viktor Lofgren
974aa35558 (search) Add proper alt-text to random exploration mode 2024-05-04 11:41:00 +02:00
Viktor Lofgren
4021a0ae98 (search) Add en-US language tags to all templates 2024-05-04 11:40:59 +02:00
Viktor Lofgren
b7a95be731 (search) Create a small mocking framework for running the search service in isolation. 2024-05-04 11:40:59 +02:00
Viktor Lofgren
616649f040 (logs) Fix logdir location 2024-05-04 11:40:59 +02:00
Viktor
ac3c692b5f
Merge pull request #92 from MarginaliaSearch/no-docker-v2
(WIP) Changes to make the system runnable outside of docker
2024-05-01 13:00:56 +02:00
Viktor Lofgren
6087f9635c (qs) Move index.html out of public directory
It was put there to simulate the /public interface paradigm that is now deprecated.
2024-05-01 12:56:12 +02:00
Viktor Lofgren
2ad0bfda1e (*) Fix boot orchestration for the services
This corrects an annoying bug that had the system crash and burn on first start-up due to a race condition in service initialization, where the services were attempting to access the database before it was properly migrated.

A fix was in principle already in place, but it was running too late and did not prevent attempts to access the as-yet uninitialized database.  Move the first boot check into the MainClass instead of the Service constructor.

The change also adds more appropriate docker dependencies to the services to fix rare errors resolving the hostname of the database.
2024-05-01 12:39:48 +02:00
Viktor Lofgren
cf8b12bcdc Update install.sh with refined service descriptions 2024-05-01 12:07:30 +02:00
Viktor Lofgren
08f8b6e022 (system) Log loaded properties to the console 2024-04-30 18:29:11 +02:00
Viktor Lofgren
800ed6b1e9 (zk) Terminately immediately if zookeeper isn't found
This makes debugging easier
2024-04-30 18:28:49 +02:00