Viktor Lofgren
dd9406d0ac
(control) Make storage type tabs consistent
...
This had fallen off in the Create New Specification view, it lacked Exports.
2023-11-17 11:26:45 +01:00
Viktor Lofgren
e9a01caa5c
(index) Fix broken metrics
2023-11-11 12:53:47 +01:00
Viktor Lofgren
858357a246
(metrics) Get prometheus up out of disrepair
...
* Fix bad labels
* Add nodeId where appropriate
* Hopefully fix histogram buckets for index query times
2023-11-08 14:01:28 +01:00
Viktor Lofgren
0152004c42
Initial Commit Anchor Tags
...
* Added new (optional) model file in $WMSA_HOME/data/atags.parquet
* Converter gets a component for creating a projection of its domains onto the full atags parquet file
* New WordFlag ExternalLink
* These terms are also for now flagged as title words
* Fixed a bug where Title words aliased with UrlDomain words
* Fixed a bug in the encyclopedia sideloader that gave everything too high topology ranking
2023-11-04 14:24:17 +01:00
Viktor Lofgren
8e9698c9a0
(control/search) Add ability to suggest removing a site from random exploration
...
This is what most complaints have been about.
2023-11-02 15:29:49 +01:00
Viktor Lofgren
3047e2dd7c
(screenshot-capture-tool) Make screenshot-capture-tool cooperate with docker
2023-11-01 16:38:55 +01:00
Viktor Lofgren
a8b9d21f2d
(executor) Refine atag export logic
...
* Remove obviously uninteresting tags
* Omit URL schema for more sensible sorting
* Change the column order to put the source domain last
2023-11-01 13:23:14 +01:00
Viktor Lofgren
c77a5b7cb6
(control) GUI for atags export
2023-10-31 17:55:47 +01:00
Viktor Lofgren
23f2068e33
(executor) Actor for exporting anchor tag data from crawl data
2023-10-31 17:32:34 +01:00
Viktor Lofgren
ffadfb4149
(control) Use a partial template for the storage types tabs.
2023-10-31 17:12:14 +01:00
Viktor Lofgren
b7e38cfbae
(control) Add exports view
2023-10-31 17:08:48 +01:00
Viktor Lofgren
659743b39c
(executor) Export Data actor allocates its own storage
2023-10-31 17:04:07 +01:00
Viktor Lofgren
69758c5859
(control) Nicer redirects acknowledging actions
2023-10-31 16:26:29 +01:00
Viktor Lofgren
2871a326e6
(ctrl/exe) Clean up UX and code
2023-10-29 14:09:39 +01:00
Viktor Lofgren
abb42f0f36
(crawler) Fix bug in SQL statement
...
Arguments were in the wrong order in inserting fetching sites submitted to be crawled
2023-10-29 13:19:17 +01:00
Viktor Lofgren
88f49834fd
(docs) Update documentation
2023-10-27 12:45:39 +02:00
Viktor Lofgren
c7cb6664b4
(control) Indicate missing services with danger-color instead of having a distracting and constantly updating last-seen number
2023-10-26 18:05:22 +02:00
Viktor Lofgren
79adba9284
(index) Fix bug in dealing with quoted search terms
2023-10-26 16:28:23 +02:00
Viktor Lofgren
f613f4f2df
(array) Fix spurious search results
...
This was caused by a bug in the binary search algorithm causing it to sometimes return positive values when encoding a search miss.
It was also necessary to get rid of the vestiges of the old LongArray and IntArray classes to make this fix doable.
2023-10-26 15:27:02 +02:00
Viktor Lofgren
abbadc92a0
(exdecutor) Prevent TriggerAdjacencyCalculationActor from showing up in the actions tab when it isn't running
2023-10-25 21:25:07 +02:00
Viktor Lofgren
97fcbdd6d9
(control) Move storage actions into the actions tab
...
* Also disable annoying CSS animations
2023-10-25 21:23:56 +02:00
Viktor Lofgren
d7686b665e
Refactoring
...
* Encyclopedia sideloader; permit providing base URL.
* Storage base shows node id in GUI
* ProcessLivenessMonitorActor restarts automatically
* Clean-up of outbox code
2023-10-25 18:51:02 +02:00
Viktor Lofgren
84cdac83d6
(control) Move message queue monitor to control
2023-10-24 16:44:28 +02:00
Viktor Lofgren
313cc2965c
(index-creation) Print whether full or prio is created
...
Previous state of saying reverse index for both was pretty confusing.
2023-10-24 16:23:10 +02:00
Viktor Lofgren
95f74c5ea7
(control) Filter out heartbeats that are stopped
2023-10-24 16:09:28 +02:00
Viktor Lofgren
0406e76889
(api) Remove logging cruft
2023-10-24 13:39:05 +02:00
Viktor Lofgren
c2b28c0f8d
(api) Trial streaming API
2023-10-24 13:26:46 +02:00
Viktor Lofgren
a860f8f1a8
(index/qs) GRPC API for better query peformance
2023-10-24 11:38:07 +02:00
Viktor Lofgren
2ed2f35a9b
(actor) Rewrite of the actor prototype class using record pattern matching
2023-10-23 10:18:20 +02:00
Viktor Lofgren
119151cad3
(converter) Separtion of concerns
2023-10-22 14:35:33 +02:00
Viktor Lofgren
758f9b5aa5
(converter) Get UUID pips out of the models
...
Rendering concerns shouldn't be in the models, it's poor separation of concerns and very difficult to follow.
2023-10-22 14:24:52 +02:00
Viktor Lofgren
eb4158df0b
(control) Fix start/stop FSM endpoints
2023-10-22 14:03:09 +02:00
Viktor Lofgren
12fda1a36b
(control) Temporarily re-writing the data balancer to get it to work in prod
...
Need to clean this up later.
2023-10-22 14:03:09 +02:00
Viktor Lofgren
e927f99777
(control) JSON serializes Map<Integer> to Map<Double> and Java gets confused
2023-10-21 16:24:20 +02:00
Viktor Lofgren
044bcf55bd
(control) Fix SQL in rebalance actor
2023-10-21 16:13:37 +02:00
Viktor Lofgren
e475af9f49
(control) Initialize controlActorService
2023-10-21 16:06:53 +02:00
Viktor Lofgren
c6abcd91fa
(control) Better use of FS states, fix bug with start/stop actors
2023-10-20 16:37:49 +02:00
Viktor Lofgren
d76d926c38
(control/executor) Add new configuration options for node
...
It's now possible to configure prod instance to not retain processed data.
2023-10-20 14:05:19 +02:00
Viktor Lofgren
2b3c167845
(controller) Additional configuration options for node
2023-10-20 13:13:36 +02:00
Viktor Lofgren
584bb3a648
(fs) interface cleanup
2023-10-20 12:24:18 +02:00
Viktor Lofgren
7b5ec6b98f
(executor-service) Embed dist/ in executor-service's docker image
2023-10-19 17:48:34 +02:00
Viktor Lofgren
23526f6d1a
(executor) Executor service now pulls DomainType list for CRAWL on "recrawl"
...
This is an automatic integration with the submit-site repo on github and also
crawl-queue.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
809b3ee023
(control) Update GUI for crawl specs. They are now less important than they were before.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
23f0c79fba
(control) GUI for data sets/domain types.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
81dd3809e9
(*) WIP Add node affinity to EC_DOMAIN
...
Very messy commit due to fractalline yak shaving
2023-10-19 17:48:34 +02:00
Viktor Lofgren
978550f809
(executor-service) Retire features-convert and move the corresponding packages into the executor service.
2023-10-16 15:43:46 +02:00
Viktor Lofgren
84fea0fd05
(node) Nodes auto-start their monitor actors.
2023-10-16 15:33:22 +02:00
Viktor Lofgren
2df3e0f881
(node) Nodes auto-configure on start-up instead of requiring manual configuration.
2023-10-16 14:46:35 +02:00
Viktor Lofgren
ede5d1f890
(actor) Give process spawners more easily recognizable names.
2023-10-16 14:19:00 +02:00
Viktor Lofgren
39911e3acd
(control) Fix incorrect storage base and clean up GUI for data
2023-10-16 13:30:26 +02:00
