Viktor Lofgren
a1fb92468f
(refac) Remove ResultRankingParameters, QueryLimits class and use protobuf classes directly instead
...
This is primarily to make the code a bit easier to reason about, and will reduce the level of indirection and data copying in the search-servi->query-service->index-service communication chain.
2025-01-08 16:15:57 +01:00
Viktor Lofgren
b7f0a2a98e
(search-service) Fix metrics for errors and request times
...
This was previously in place, but broke during the jooby migration.
2025-01-08 14:10:43 +01:00
Viktor Lofgren
5fb76b2e79
(search-service) Fix metrics for errors and request times
...
This was previously in place, but broke during the jooby migration.
2025-01-08 14:06:03 +01:00
Viktor Lofgren
ad8c97f342
(search-service) Begin replacement of the crawl queue mechanism with node_affinity flagging
...
Previously a special db table was used to hold domains slated for crawling, but this is deprecated, and instead now each domain has a node_affinity flag that decides its indexing state, where a value of -1 indicates it shouldn't be crawled, a value of 0 means it's slated for crawling by the next index partition to be crawled, and a positive value means it's assigned to an index partition.
The change set also adds a test case validating the modified behavior.
2025-01-08 13:25:56 +01:00
Viktor Lofgren
dc1b6373eb
(search-service) Clean up readme
2025-01-08 13:04:39 +01:00
Viktor Lofgren
983d6d067c
(search-service) Add indexing indicator to sibling domains listing
2025-01-08 12:58:34 +01:00
Viktor Lofgren
a84a06975c
(ranking-params) Add disable penalties flag to ranking params
...
This will help debugging ranking issues. Later it may be added to some filters.
2025-01-08 00:16:49 +01:00
Viktor Lofgren
d2864c13ec
(query-params) Add additional permitted query params
2025-01-07 20:21:44 +01:00
Viktor Lofgren
03ba53ce51
(legacy-search) Update nav bar with correct links
2025-01-07 17:44:52 +01:00
Viktor Lofgren
d4a6684931
(specialization) Soften length requirements for wiki-specialized documents (incl. cppreference)
2025-01-07 15:53:25 +01:00
Viktor
6f0485287a
Merge pull request #145 from MarginaliaSearch/cppreference_fixes
...
Cppreference fixes
2025-01-07 15:43:19 +01:00
Viktor Lofgren
59e2dd4c26
(specialization) Soften length requirements for wiki-specialized documents (incl. cppreference)
2025-01-07 15:41:30 +01:00
Viktor Lofgren
ca1807caae
(specialization) Add new specialization for cppreference.com
...
Give this reference website some synthetically generated tokens to improve the likelihood of a good match.
2025-01-07 15:41:05 +01:00
Viktor Lofgren
26c20e18ac
(keyword-extraction) Soften constraints on keyword patterns, allowing for longer segmented words
2025-01-07 15:20:50 +01:00
Viktor Lofgren
7c90b6b414
(query) Don't blindly make tokens containing a colon into a non-ranking advice term
2025-01-07 15:18:05 +01:00
Viktor Lofgren
b63c54c4ce
(search) Update opensearch.xml to point to non-redirecting domains.
2025-01-07 00:23:09 +01:00
Viktor Lofgren
fecd2f4ec3
(deploy) Add legacy search service to deploy script
2025-01-07 00:21:13 +01:00
Viktor Lofgren
39e420de88
(search) Add wayback machine link to siteinfo
2025-01-06 20:33:10 +01:00
Viktor Lofgren
dc83619861
(rssreader) Further suppress logging
2025-01-06 20:20:37 +01:00
Viktor Lofgren
87d1c89701
(search) Add listing of sibling subdomains to site overview
2025-01-06 20:17:36 +01:00
Viktor Lofgren
a42a7769e2
(leagacy-search) Remove legacy paperdoll class
2025-01-06 20:17:36 +01:00
Viktor
202bda884f
Update readme.md
...
Add note about installing tailwindcss via npm
2025-01-06 18:35:13 +01:00
Viktor Lofgren
2315fdc731
(search) Vendor rssreader and modify it to be able to consume the nlnet atom feed
...
Also dial down the logging a bit for the rssreader package.
2025-01-06 17:58:50 +01:00
Viktor Lofgren
b5469bd8a1
(search) Turn relative feed URLs absolute when dealing with RSS/Atom item URLs
2025-01-06 16:56:24 +01:00
Viktor Lofgren
6a6318d04c
(search) Add separate websiteUrl property to legacy service
2025-01-06 16:26:08 +01:00
Viktor Lofgren
55933f8d40
(search) Ensure we respect old URL contracts
...
/explore/random should be equivalent to /explore
2025-01-06 16:20:53 +01:00
Viktor
be6382e0d0
Merge pull request #127 from MarginaliaSearch/serp-redesign
...
Web UI redesign
2025-01-06 16:08:14 +01:00
Viktor Lofgren
45e771f96b
(api) Update the / API redirect to the new documentation stub.
2025-01-06 16:07:32 +01:00
Viktor Lofgren
8dde502cc9
Merge branch 'master' into serp-redesign
2025-01-05 23:33:35 +01:00
Viktor Lofgren
3e66767af3
(search) Adjust query parsing to trim tokens in quoted search terms
...
Quoted search queries that contained keywords with possessive 's endings were not returning any results, as the index does not retain that suffix, and the query parser was not stripping it away in this code path.
This solves issue #143 .
2025-01-05 23:33:09 +01:00
Viktor Lofgren
9ec9d1b338
Merge branch 'master' into serp-redesign
2025-01-05 21:10:20 +01:00
Viktor Lofgren
dcad0d7863
(search) Tweak token formation.
2025-01-05 21:01:09 +01:00
Viktor Lofgren
94e1aa0baf
(search) Tweak token formation to still break apart emails in brackets.
2025-01-05 20:55:44 +01:00
Viktor Lofgren
b62f043910
(search) Adjust token formation rules to be more lenient to C++ and PHP code.
...
This addresses Issue #142
2025-01-05 20:50:27 +01:00
Viktor Lofgren
6ea22d0d21
(search) Update front page with work-in-progress note
2025-01-05 19:08:02 +01:00
Viktor Lofgren
8c69dc31b8
Merge branch 'master' into serp-redesign
2025-01-05 18:52:51 +01:00
Viktor Lofgren
00734ea87f
(search) Add hover text for matchogram
2025-01-05 18:50:44 +01:00
Viktor Lofgren
3009713db4
(search) Fix broken tests
2025-01-05 18:50:27 +01:00
Viktor
9b2ceaf37c
Merge pull request #141 from MarginaliaSearch/vlofgren-patch-1
...
Update FUNDING.yml
2025-01-05 18:40:20 +01:00
Viktor
8019c2ce18
Update FUNDING.yml
2025-01-05 18:40:06 +01:00
Viktor Lofgren
a9e312b8b1
(service) Add links to marginalia-search.com where appropriate
2025-01-05 16:56:38 +01:00
Viktor Lofgren
4da3563d8a
(service) Clean up exceptions when requestScreengrab is not available
2025-01-04 14:45:51 +01:00
Viktor Lofgren
48d0a3089a
(service) Improve logging around grpc
...
This change adds a marker for the gRPC-specific logging, as well as improves the clarity and meaningfulness of the log messages.
2025-01-02 20:40:53 +01:00
Viktor Lofgren
594df64b20
(domain-info) Use appropriate sqlite database when fetching feed status
2025-01-02 20:20:36 +01:00
Viktor Lofgren
06efb5abfc
Merge branch 'master' into serp-redesign
2025-01-02 18:42:12 +01:00
Viktor Lofgren
78eb1417a7
(service) Only block on SingleNodeChannelPool creation in QueryClient
...
The code was always blocking for up to 5s while waiting for the remote end to become available, meaning some services would stall for several seconds on start-up for no sensible reason.
This should make most services start faster as a result.
2025-01-02 18:42:01 +01:00
Viktor Lofgren
8c8f2ad5ee
(search) Add an indicator when a link has a feed in the similar/linked domains views
2025-01-02 18:11:57 +01:00
Viktor Lofgren
f71e79d10f
(search) Add a copy of the old UI as a separate service, search-service-legacy
2025-01-02 18:03:42 +01:00
Viktor Lofgren
1b27c5cf06
(search) Add a copy of the old UI as a separate service, search-service-legacy
2025-01-02 18:02:17 +01:00
Viktor Lofgren
67edc8f90d
(domain-info) Only flag domains with rss feed items as having a feed
2025-01-02 17:41:52 +01:00
