(refactor) Remove features-search and update documentation

code/api/readme.md

@@ -3,7 +3,7 @@
 ## Core Services
 
 * [assistant-api](assistant-api/)
-* [search-api](search-api/)
+* [query-api](query-api/)
 * [index-api](index-api/)
 
 These are clients for the [core services](../services-core/), along with what models
@@ -20,4 +20,4 @@ requests can be sent.
 [process-mqapi](process-mqapi/) defines requests and inboxes for the message queue based API used 
 for interacting with processes.   
 
-See [libraries/message-queue](../libraries/message-queue) and [services-application/control-service](../services-application/control-service). 
+See [libraries/message-queue](../libraries/message-queue) and [services-application/control-service](../services-core/control-service). 

code/common/linkdb/readme.md

@@ -8,4 +8,4 @@ this information take effect in production immediately, even before
 the information was searchable.
 
 It is constructed by the [loading-process](../../processes/loading-process), and consumed 
-by the [search-service](../../services-core/search-service).
+by the [index-service](../../services-core/index-service).

code/common/model/readme.md

@@ -6,7 +6,6 @@ This package contains common models to the search engine
 
 * [EdgeDomain](src/main/java/nu/marginalia/model/EdgeDomain.java)
 * [EdgeUrl](src/main/java/nu/marginalia/model/EdgeUrl.java)
-* [EdgeId](src/main/java/nu/marginalia/model/id/EdgeId.java)
 * [DocumentMetadata](src/main/java/nu/marginalia/model/idx/DocumentMetadata.java)
 * [DocumentFlags](src/main/java/nu/marginalia/model/idx/DocumentFlags.java)
 * [WordMetadata](src/main/java/nu/marginalia/model/idx/WordMetadata.java)

code/features-index/domain-ranking/readme.md

@@ -17,7 +17,7 @@ Contains domain ranking algorithms.
 
 ## See Also
 
-* [features-search/result-ranking](../../features-search/result-ranking) - Ranks search results
+* [result-ranking](../result-ranking) - Ranks search results
 
 ## Useful Resources
 

code/features-index/readme.md

@@ -14,11 +14,11 @@ These indices rely heavily on the [libraries/btree](../libraries/btree) and [lib
 
 ## Algorithms
 
-* [domain-ranking](domain-ranking/) contains ranking algorithms.
+* [domain-ranking](domain-ranking/) contains domain ranking algorithms.
+* [result-ranking](result-ranking/) contains logic for ranking search results by relevance.
 
 # Libraries
 
 * [index-query](index-query/) contains structures for evaluating search queries.
 * [index-journal](index-journal/) contains tools for writing and reading index data.
-* [lexicon](lexicon/) contains a mapping between words' string representation and an unique integer identifier.
 

code/features-index/result-ranking/readme.md

@@ -11,4 +11,4 @@ results higher.
 
 ## See Also
 
-* [features-index/domain-ranking](../../features-index/domain-ranking) - Ranks domains
+* [features-index/domain-ranking](../domain-ranking) - Ranks domains

code/features-qs/query-parser/readme.md

@@ -1,6 +1,6 @@
 # Query Parser
 
-End-user search query parsing tools used by the [search-service](../../services-core/search-service).
+End-user search query parsing tools used by the [query-service](../../services-core/query-service).
 
 ## Central Classes
 

code/features-qs/readme.md

@@ -0,0 +1,7 @@
+**Note**: This package is called `features-qs` rather than `features-query` because the latter,
+though more consistent with other packages like features-index, would be very confusing
+as there are other packages elsewhere with the 'query' name (e.g. features-index/index-query).
+
+## Contents
+
+* [query-parser](query-parser/) contains code for parsing the user-facing query grammar.

code/features-search/readme.md

@@ -3,8 +3,6 @@
 These are bits of search-engine related code that are relatively isolated pieces of business logic,
 that benefit from the clarity of being kept separate from the rest of the
 search engine code.
-
-* [query-parser](query-parser/) contains code for parsing the user-facing query grammar.
-* [result-ranking](result-ranking/) contains logic for ranking search results by relevance.
+* 
 * [screenshots](screenshots/) and [random-websites](random-websites/) contains SQL queries random
   exploration mode.

code/libraries/language-processing/readme.md

@@ -13,4 +13,4 @@ its words, how they stem, POS tags, and so on.
 [features-convert/keyword-extraction](../../features-convert/keyword-extraction) uses this code to identify which keywords
 are important.
 
-[features-search/query-parser](../../features-search/query-parser) also does some language processing.
+[features-qs/query-parser](../../features-qs/query-parser) also does some language processing.

code/process-models/crawling-model/readme.md

@@ -7,7 +7,6 @@ Contains models shared by the [crawling-process](../../processes/crawling-proces
 
 * [CrawledDocument](src/main/java/nu/marginalia/crawling/model/CrawledDocument.java)
 * [CrawledDomain](src/main/java/nu/marginalia/crawling/model/CrawledDomain.java)
-* [CrawlingSpecification](src/main/java/nu/marginalia/crawling/model/spec/CrawlingSpecification.java)
 
 ### Serialization
 * [CrawledDomainReader](src/main/java/nu/marginalia/crawling/io/CrawledDomainReader.java)

code/processes/crawling-process/readme.md

@@ -8,7 +8,7 @@ into per-domain snapshots.
 * [CrawlerMain](src/main/java/nu/marginalia/crawl/CrawlerMain.java) orchestrates the crawling.
 * [CrawlerRetreiver](src/main/java/nu/marginalia/crawl/retreival/CrawlerRetreiver.java)
   visits known addresses from a domain and downloads each document.
-* [HttpFetcher](src/main/java/nu/marginalia/crawl/retreival/HttpFetcher.java)
+* [HttpFetcher](src/main/java/nu/marginalia/crawl/retreival/fetcher/HttpFetcherImpl.java)
   fetches a URL.
 
 ## See Also

code/processes/loading-process/readme.md

@@ -6,5 +6,4 @@ the index-service.
 
 ## Central Classes
 
-* [LoaderMain](src/main/java/nu/marginalia/loading/LoaderMain.java) main class.
-* [Loader](src/main/java/nu/marginalia/loading/loader/Loader.java) evaluates loading instructions. 
+* [LoaderMain](src/main/java/nu/marginalia/loading/LoaderMain.java) main class.

code/processes/readme.md

@@ -5,8 +5,7 @@
 The [crawling-process](crawling-process/) fetches website contents and saves them
 as compressed JSON models described in [crawling-model](../process-models/crawling-model/).
 
-The operation is specified by a crawl job specification. This is generated by [tools/crawl-job-extractor](../tools/crawl-job-extractor/)
-based on the content in the database.
+The operation is specified by a [crawl specification](../process-models/crawl-spec), which can be created in the control GUI.
 
 ## 2. Converting Process
 

code/readme.md

@@ -20,7 +20,7 @@ A map of the most important components and how they relate can be found below.
 * * [assistant](services-core/assistant-service)
 * [application services](services-application/) "microservices", stateless providing additional functionality and making an application out of the search engine.
 * * [api](services-application/api-service)  - public API
-* * [search](services-core/search-service) - marginalia search application
+* * [search](services-application/search-service) - marginalia search application
 * * [dating](services-application/dating-service)  - [https://explore.marginalia.nu/](https://explore.marginalia.nu/)
 * * [explorer](services-application/explorer-service)  - [https://explore2.marginalia.nu/](https://explore2.marginalia.nu/)
 * an [internal API](api/)

code/services-application/search-service/build.gradle

@@ -41,8 +41,8 @@ dependencies {
 
     implementation project(':code:features-search:screenshots')
     implementation project(':code:features-search:random-websites')
-    implementation project(':code:features-search:query-parser')
-    implementation project(':code:features-search:result-ranking')
+    implementation project(':code:features-qs:query-parser')
+    implementation project(':code:features-index:result-ranking')
 
     implementation libs.bundles.slf4j
 

code/services-application/search-service/readme.md

@@ -20,8 +20,6 @@ appropriate services.
 * [CommandEvaluator](src/main/java/nu/marginalia/search/command/CommandEvaluator.java) interprets a user query and acts
 upon it, dealing with special operations like `browse:` or `site:`.
 
-* [QueryFactory](src/main/java/nu/marginalia/search/query/QueryFactory.java) parses a search query into a machine readable query specification.
-
 * [SearchQueryIndexService](src/main/java/nu/marginalia/search/svc/SearchQueryIndexService.java) passes a parsed search query to the index service, and
 then decorates the search results so that they can be rendered.
 

code/services-core/index-service/build.gradle

@@ -37,7 +37,7 @@ dependencies {
     implementation project(':code:features-index:index-forward')
     implementation project(':code:features-index:index-reverse')
     implementation project(':code:features-index:domain-ranking')
-    implementation project(':code:features-search:result-ranking')
+    implementation project(':code:features-index:result-ranking')
     implementation project(':third-party:commons-codec')
     testImplementation project(path: ':code:services-core:control-service')
     testImplementation project(':code:common:process')

code/services-core/query-service/build.gradle

@@ -30,7 +30,7 @@ dependencies {
     implementation project(':code:api:index-api')
     implementation project(':code:api:query-api')
     implementation project(':code:common:service-discovery')
-    implementation project(':code:features-search:query-parser')
+    implementation project(':code:features-qs:query-parser')
     implementation project(':code:features-index:index-query')
     implementation project(':code:libraries:language-processing')
     implementation project(':code:libraries:term-frequency-dict')

code/services-core/query-service/readme.md

@@ -11,5 +11,5 @@ to that format.
 ## See Also
 
 * [api/query-api](../../api/query-api)
-* [features-search/query-parser](../../features-search/query-parser)
+* [features-qs/query-parser](../../features-qs/query-parser)
 * [features-index/index-query](../../features-index/index-query)

settings.gradle

@@ -26,8 +26,8 @@ include 'code:libraries:message-queue'
 
 include 'code:features-search:screenshots'
 include 'code:features-search:random-websites'
-include 'code:features-search:query-parser'
-include 'code:features-search:result-ranking'
+include 'code:features-qs:query-parser'
+include 'code:features-index:result-ranking'
 
 include 'code:features-convert:adblock'
 include 'code:features-convert:stackexchange-xml'