mirror of
https://github.com/MarginaliaSearch/MarginaliaSearch.git
synced 2025-02-23 21:18:58 +00:00
0307c55f9f
To avoid having to either hard-code or manually configure service addresses (possibly several dozen), and to reduce the project's dependency on docker to deal with routing and discovery, the option to use [Zookeeper](https://zookeeper.apache.org/) to manage services and discovery has been added. A service registry interface was added, with a Zookeeper implementation and a basic implementation that only works on docker and hard-codes everything. The last remaining REST service, the assistant-service, has been migrated to gRPC. This also proved a good time to clear out primordial technical debt from the root of the codebase. The 'service-client' library has been taken behind the barn and given a last farewell. It's replaced by a small library for managing gRPC channels. Since it's no longer used by anything, RxJava has been removed as a dependency from the project. Although the current state seems reasonably stable, this is a work-in-progress commit. |
||
|---|---|---|
| .. | ||
| src | ||
| build.gradle | ||
| readme.md | ||
BTree
This package contains a small library for creating and reading a static b-tree in as implicit pointer-less datastructure. Both binary indices (i.e. sets) are supported, as well as arbitrary multiple-of-keysize key-value mappings where the data is interlaced with the keys in the leaf nodes. This is a fairly low-level datastructure.
The b-trees are specified through a BTreeContext which contains information about the data and index layout.
The b-trees are written through a BTreeWriter and read with a BTreeReader.
Demo
BTreeContext ctx = new BTreeContext(
4, // num layers max
1, // entry size, 1 = the leaf node has just just the key
BTreeBlockSize.BS_4096); // page size
// Allocate a memory area to work in, see the array library for how to do this with files
LongArray array = LongArray.allocate(8192);
// Write a btree at offset 123 in the area
long[] items = new long[400];
BTreeWriter writer = new BTreeWriter(array, ctx);
final int offsetInFile = 123;
long btreeSize = writer.write(offsetInFile, items.length, slice -> {
// here we *must* write items.length * entry.size words in slice
// these items must be sorted!!
for (int i = 0; i < items.length; i++) {
slice.set(i, items[i]);
}
});
// Read the BTree
BTreeReader reader = new BTreeReader(array, ctx, offsetInFile);
reader.findEntry(items[0]);
Useful Resources
Youtube: Abdul Bari, 10.2 B Trees and B+ Trees. How they are useful in Databases. This isn't exactly the design implemented in this library, but very well presented and a good refresher.