From 19402772fc8b2823c818f389f90202531e714977 Mon Sep 17 00:00:00 2001 From: Viktor Lofgren Date: Tue, 1 Aug 2023 22:50:05 +0200 Subject: [PATCH] (scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows. --- doc/crawling.md | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/doc/crawling.md b/doc/crawling.md index cfb38f9e..679f65c6 100644 --- a/doc/crawling.md +++ b/doc/crawling.md @@ -30,22 +30,9 @@ This can be done by editing the file `${WMSA_HOME}/conf/user-agent`. ## Setup -To operate the crawler, you need to set up a filesystem structure. +Ensure that the system is running and go to https://localhost:8081. See the documentation in [run/](../run/) for more information. +By default the system is configured to store data in `run/samples`. (!!!FIXME: How do you change this now?!!!) -You need - -* a directory for crawl data -* a directory for processed data -* a crawl specification file -* a crawl plan file - -Assuming we want to keep our crawl and processed data in -`/data`, then we would create the following directories: - -```bash -$ mkdir /data/crawl -$ mkdir /data/processed -``` ### Specifications