Viktor Lofgren
8c559c8121
(conf) Add additional logic for discovering system root
2024-04-16 12:37:18 +02:00
Viktor Lofgren
448a941de2
(encyclopedia) Fix memory issue in preconversion step
...
Use SimpleBlockingThreadPool pool instead of Java's Workstealing Pool as the latter causes runaway memory consumption in some circumstances, while SimpleBlockingThreadPool uses a bounded queue and always pushes back against the supplier if it can't hold any more tasks.
2024-04-05 16:57:53 +02:00
Viktor Lofgren
e1151ecf2a
(gradle) Upgrade to Gradle 8.7
...
This will reduce the hassle of juggling JDK versions for JDK 22, which was not supported by Gradle 8.5.
2024-04-05 15:12:38 +02:00
Viktor
3890c413a3
Merge pull request #88 from jmholla/patch-1
...
Update keywords docs use of explore to browse
2024-04-01 09:14:02 +02:00
Joshua Holland
8e02f567d7
Update keywords docs use of explore to browse
...
I can't tell when this happened, but the proper keyword now seems to be browse and not explore.
2024-04-01 00:04:12 -05:00
Viktor Lofgren
de8e753fc8
(run) Fix traefik middleware configuration
2024-03-28 13:03:12 +01:00
Viktor
cd8f33f830
Merge pull request #86 from MarginaliaSearch/jdk-22
...
Lift JDK version to 22
2024-03-21 14:29:41 +01:00
Viktor Lofgren
824765b1ee
(*) Fix JDK22 migration issues
...
A few bizarre build errors cropped up when migrating to JDK22. Not at all sure what caused them, but they were easy to mitigate.
2024-03-21 14:27:13 +01:00
Viktor Lofgren
9e8138f853
(docs) Upgrade to JDK22
2024-03-21 14:27:13 +01:00
Viktor Lofgren
fe8d583fdd
(sys) Upgrade to JDK22
...
This also entails upgrading JIB to 3.4.1 and Lombok to 1.18.32.
2024-03-21 14:27:13 +01:00
Viktor
258a344810
Merge pull request #85 from patrickbreen/master
...
(run/install.sh) fix docker compose file
2024-03-18 13:09:30 +01:00
Your Name
2a03014652
(run/install.sh) fix docker compose file
...
I was following the release demo video for v2024.01.0
https://www.youtube.com/watch?v=PNwMkenQQ24 and when I did 'docker
compose up' the containers couldn't resolve the DNS name for 'zookeeper'
I realized this was because the zookeeper container was using the
default docker network, so I specified the wmsa network explicitly.
2024-03-17 15:33:19 -04:00
Viktor Lofgren
57e6a12d08
(registry) Correct registerMonitor() behavior
...
The previous behavior would listen to too many changes, and based on zookeeper and not curator assumptions about behavior, add an additional monitor on each invocation of each monitor, (which always trigger on service state changes), leading to each monitor re-registering and effectively doubling monitors in numbers whenever a service stopped or started, which in turn meant a lot of bizarre thrashing behavior even on changes in services that don't explicitly talk to each other.
This re-registering behavior is no longer done.
2024-03-06 12:22:15 +01:00
Viktor Lofgren
46423612e3
(refac) Merge service-discovery and service modules
...
Also adds a few tests to the server/client code.
2024-03-03 10:49:23 +01:00
Viktor Lofgren
29bf473d74
(encyclopedia) Add URLencoding to path element
...
This prevents corruption of the links to the sideloaded encyclopedia data when the article path contains characters that are not valid in a URL.
2024-03-01 17:28:09 +01:00
Viktor Lofgren
9689f3faee
(domain-info) Fix incorrect array indexing
2024-02-29 18:56:09 +01:00
Viktor Lofgren
93fa58c93d
(domain-info) Fix incorrect array indexing
...
Using the id instead of idx when addressing the ranksArray caused exceptions.
2024-02-29 17:54:23 +01:00
Viktor Lofgren
186a98cc99
(doc) Fix wonky bullet lists
2024-02-28 17:43:05 +01:00
Viktor Lofgren
9993f265ca
(doc) Remove irrelevant text
2024-02-28 17:40:05 +01:00
Viktor Lofgren
144f967dbf
(misc) Tweak pool sizes
2024-02-28 16:23:02 +01:00
Viktor Lofgren
b31c9bb726
(docs) Update process docs
2024-02-28 15:21:33 +01:00
Viktor Lofgren
c0820b5e5c
(docs) Update service docs
2024-02-28 15:19:31 +01:00
Viktor Lofgren
65b8a1d5d9
(grpc) Reduce error spam
2024-02-28 14:44:48 +01:00
Viktor Lofgren
a0648844fb
(grpc) Reduce error spam
2024-02-28 14:35:29 +01:00
Viktor Lofgren
c4a27003c6
(docs) Fix formatting
2024-02-28 14:22:57 +01:00
Viktor Lofgren
41abd8982f
(math) Clean up error handling
2024-02-28 14:19:50 +01:00
Viktor Lofgren
86bbc1043e
(service) Clean up thread pool creation
2024-02-28 14:06:32 +01:00
Viktor Lofgren
9a045a0588
(index) Clean up index code
2024-02-28 13:09:47 +01:00
Viktor Lofgren
9415539b38
(docs) Update docs
2024-02-28 12:25:19 +01:00
Viktor Lofgren
84bab2783d
(docs) Fix fake news in docs
2024-02-28 12:16:45 +01:00
Viktor
0d6e7673e4
Merge pull request #81 from MarginaliaSearch/service-discovery
...
Zookeeper for service-discovery, kill service-client lib, refactor everything
2024-02-28 12:15:25 +01:00
Viktor Lofgren
d78e9e715f
(misc) Fix broken tests
2024-02-28 12:12:43 +01:00
Viktor Lofgren
a8ec59eb75
(conf) Add migration warning when ZOOKEEPER_HOSTS is not set.
2024-02-28 12:09:38 +01:00
Viktor Lofgren
20fc0ef13c
(gradle) Add task alias 'docker' for 'jibDockerBuild'
...
The change also moves the jib boilerplate to an include.
2024-02-28 11:59:15 +01:00
Viktor Lofgren
37ae8cb33c
Migrate the docker compose files
2024-02-28 11:48:16 +01:00
Viktor Lofgren
9f1649636e
Clean up documentation and rename domain-links to link-graph
2024-02-28 11:40:39 +01:00
Viktor Lofgren
3a65fe8917
Add offload executor to GrpcChannelPoolFactory
2024-02-27 22:08:39 +01:00
Viktor Lofgren
99a6e56e99
(index-client) Increase thread count in index client
...
This should be a fair bit larger than the number of index nodes
2024-02-27 22:00:29 +01:00
Viktor Lofgren
e696fd9e92
(docs) Begin un-fucking the docs after refactoring
2024-02-27 21:22:21 +01:00
Viktor Lofgren
c943954bb4
(domain-info) Reduce memory usage
2024-02-27 21:22:21 +01:00
Viktor Lofgren
eaf836dc66
(service/grpc) Reduce thread count
...
Netty and GRPC by default spawns an incredible number of threads on high-core CPUs, which amount to a fair bit of RAM usage.
Add custom executors that throttle this behavior.
2024-02-27 21:22:21 +01:00
Viktor Lofgren
dbf64b0987
(logs) Add the option for json logging
2024-02-27 21:22:20 +01:00
Viktor Lofgren
8d0af9548b
(search) Bot mitigation
...
Add the ability to indicate to the search service that a request is malicious, and to poison the results by providing randomly reorered old results instead.
2024-02-27 21:22:19 +01:00
Viktor Lofgren
67aa20ea2c
(array) Attempting to debug strange errors
2024-02-27 21:22:18 +01:00
Viktor Lofgren
5604e9f531
(query) Bump query length, see what happens :P
2024-02-27 21:22:17 +01:00
Viktor Lofgren
1a51ec2d69
(index) Index optimization
2024-02-27 21:22:17 +01:00
Viktor Lofgren
3eb0800742
(index) Improve granularity of candidate queue polling
2024-02-27 21:22:17 +01:00
Viktor Lofgren
427f3e922f
(index) Retire count operation, clean up index code.
2024-02-27 21:22:17 +01:00
Viktor Lofgren
823ca73a3f
(domain-ranking) Fix a crash during ranking the edges of the similarity graph doesn't quite match the vertices of the link graph.
2024-02-27 21:22:17 +01:00
Viktor Lofgren
7fc0d4d786
(index) Observability for query execution queues
2024-02-27 21:22:17 +01:00
