Viktor Lofgren
a09c84e1b8
(query) Modify tokenizer to match the behavior of the sentence extractor
...
This must match, otherwise a query like "plato's republic" won't match the indexed keywords, since they would strip the possessive.
2024-04-24 14:44:39 +02:00
Viktor Lofgren
c583a538b1
(search) Add implicit coherence constraints based on segmentation
2024-04-24 14:44:39 +02:00
Viktor Lofgren
5f6a3ef9d0
(ngram) Correct |s|^|s|-normalization to use length and not count
2024-04-24 14:44:39 +02:00
Viktor Lofgren
e23359bae9
(query, minor) Remove debug statement
2024-04-24 14:44:39 +02:00
Viktor Lofgren
5531ed632a
(query, minor) Remove debug statement
2024-04-24 14:44:39 +02:00
Viktor Lofgren
c96da0ce1e
(segmentation) Pick best segmentation using |s|^|s|-style normalization
...
This is better than doing all segmentations possible at the same time.
2024-04-24 14:44:38 +02:00
Viktor Lofgren
7dd8c78c6b
(ngrams) Remove the vestigial logic for capturing permutations of n-grams
...
The change also reduces the object churn in NGramLexicon, as this is a very hot method in the converter.
2024-04-24 14:44:38 +02:00
Viktor Lofgren
8bf7d090fd
(qs) Clean up parsing code using new record matching
2024-04-24 14:44:38 +02:00
Viktor Lofgren
a3a6d6292b
(qs, index) New query model integrated with index service.
...
Seems to work, tests are green and initial testing finds no errors. Still a bit untested, committing WIP as-is because it would suck to lose weeks of work due to a drive failure or something.
2024-04-24 14:44:38 +02:00
Viktor Lofgren
8cb9455c32
(qs, WIP) Fix edge cases in query compilation
...
This addresses the relatively common case where the graph consists of two segments, such as x y, z w; in this case we want an output like (x_y) (z w | z_w) | x y (z_w). The generated output does somewhat pessimize a few other cases, but this one is arguably more important.
2024-04-24 14:44:38 +02:00
Viktor Lofgren
dc65b2ee01
(qs, WIP) Clean up dead code
2024-04-24 14:44:38 +02:00
Viktor Lofgren
98a1adbf81
(qs, WIP) Tidy it up a bit
2024-04-24 14:44:38 +02:00
Viktor Lofgren
0bd1e15cce
(qs, WIP) Tidy it up a bit
2024-04-24 14:44:38 +02:00
Viktor Lofgren
eda926767e
(qs, WIP) Tidy it up a bit
2024-04-24 14:44:38 +02:00
Viktor Lofgren
cd1a18c045
(qs, WIP) Break up code and tidy it up a bit
2024-04-24 14:44:38 +02:00
Viktor Lofgren
6f567fbea8
(qs, WIP) Fix output determinism, fix tests
2024-04-24 14:44:38 +02:00
Viktor Lofgren
0ebadd03a5
(WIP) Query rendering finally beginning to look like it works
2024-04-24 14:44:38 +02:00
Viktor Lofgren
2253b556b2
WIP
2024-04-24 14:44:17 +02:00
Viktor Lofgren
6a7a7009c7
(convert) Initial integration of segmentation data into the converter's keyword extraction logic
2024-04-24 14:44:17 +02:00
Viktor Lofgren
3c75057dcd
(qs) Retire NGramBloomFilter, integrate new segmentation model instead
2024-04-24 14:44:17 +02:00
Viktor Lofgren
760b80659d
(WIP) Partial integration of new query expansion code into the query-serivice
2024-04-24 14:44:17 +02:00
Viktor Lofgren
04879c005d
(WIP) Improve data extraction from wikipedia data
2024-04-24 14:44:17 +02:00
Viktor Lofgren
cb82927756
(WIP) Implement first take of new query segmentation algorithm
2024-04-24 14:44:17 +02:00
Viktor Lofgren
9f1649636e
Clean up documentation and rename domain-links to link-graph
2024-02-28 11:40:39 +01:00
Viktor Lofgren
5604e9f531
(query) Bump query length, see what happens :P
2024-02-27 21:22:17 +01:00
Viktor Lofgren
427f3e922f
(index) Retire count operation, clean up index code.
2024-02-27 21:22:17 +01:00
Viktor Lofgren
1d34224416
(refac) Remove src/main from all source code paths.
...
Look, this will make the git history look funny, but trimming unnecessary depth from the source tree is a very necessary sanity-preserving measure when dealing with a super-modularized codebase like this one.
While it makes the project configuration a bit less conventional, it will save you several clicks every time you jump between modules. Which you'll do a lot, because it's *modul*ar. The src/main/java convention makes a lot of sense for a non-modular project though. This ain't that.
2024-02-23 16:13:40 +01:00
