Commit Graph

340 Commits

Author SHA1 Message Date
Bobby Wibowo
81cf940160
feat: added audios count to uploads stats
As always, this uses hardcoded whitelisted audio extensions (the ones
that are also internally used for "is:audio" filter).
2021-01-08 11:22:31 +07:00
Bobby Wibowo
991d743ef0
feat: add "files in albums" to albums stats 2021-01-08 11:18:50 +07:00
Bobby Wibowo
defa3f2a8c
perf: skip checking zips on disk for album stats
Closes #353.
2021-01-08 11:03:26 +07:00
Bobby Wibowo
452542ca36
refactor: Client/ServerError on tokenController 2021-01-08 10:56:09 +07:00
Bobby Wibowo
a5607c00f6
refactor: make use of improved utils.authorize() 2021-01-08 10:50:25 +07:00
Bobby Wibowo
c1562e11f4
refactor: ClientError on utilsController
Improved utils.authorize().

Various fixes to thumbnailer and tags stripper.
2021-01-08 10:48:08 +07:00
Bobby Wibowo
59efef2d18
refactor: Client/ServerError on authController 2021-01-08 10:11:56 +07:00
Bobby Wibowo
1142b64e3c
refactor: no throw literal in utilsController 2021-01-08 09:56:01 +07:00
Bobby Wibowo
b5af733dc2
refactor: Client/ServerError on uploadController 2021-01-08 09:44:04 +07:00
Bobby Wibowo
a816aac1f1
fix: await instead of return in try-catch block
Otherwise uncaught exceptions on errors.
2021-01-08 09:27:38 +07:00
Bobby Wibowo
721c1c84a9
fix: proper status codes in some album apis 2021-01-08 09:22:31 +07:00
Bobby Wibowo
e2143b4d80
refactor: UserError -> ClientError, ServerError
ClientError will default to 400 HTTP error code.
ServerError will default to 500 HTTP error code.

Following the previous commit, these for now are only being used in
albumsController. More will soon follow.

Additionally fixed existing album names can sometimes be re-used when
editing an album.
2021-01-08 08:44:28 +07:00
Bobby Wibowo
ae31033c0c
refactor: init UserError, a custom Error object
This will be used for errors that are to be delivered to users, AND not
to be logged into the server (as in it stacktraces and all).
This will eventually remove the need to throw string literals.

In this commit, this has only been implemented on albumsController.js,
but more will soon to come.
2021-01-08 07:29:14 +07:00
Bobby Wibowo
0dfdccb25e
refactor: move custom multer storage
controllers dir: multerStorageController.js to utils/multerStorage.js
2021-01-08 07:25:27 +07:00
Bobby Wibowo
0a19b025a0
removed utils.parallelLimit
well, that was a pointless endeavor.
i wasn't thinking clearly.

also updated all scripts that previously used them, to use a combo of
for-loop and setInterval to print progress.
2020-12-27 19:44:10 +07:00
Bobby Wibowo
18db55ba24
Added scripts/rebuild-hashes.js
Closes #345
2020-12-27 18:16:01 +07:00
Bobby Wibowo
e85e8e886d
added lolisafe upstream compat to /api/album/:id
it will re-map body of /api/album/get/:id into upstream-compatible body.
prep for lolisafe albums support for magane plugin.

/api/album/:id/:page will stil respond with the old format as that's
what the dashboard use and expect.

list views of uploads, users and albums in dashboard will now show
total items count on the table's top right corner.
2020-12-26 19:54:41 +07:00
Bobby Wibowo
c5647cb8bf
albums sidebar in dashboard is now collapsible
they'll also be collapsed on initial page load

this uses a new client-side dependency, bulma-collapsible
https://github.com/creativebulma/bulma-collapsible

/api/albums to fetch albums list now support simple reply, where only
their ids and names will be returned.
this simple reply will also return all of the user's albums, instead of
being limited to only 9 or 25 entires like before.

fixed add to album in dashboard, and album selector in homepage uploader
being limited to only 25 albums.
2020-12-26 18:49:51 +07:00
Bobby Wibowo
d7b11e7e56
try cf purge up to 3 times
with preset delay in between (60 secs if rate limited, 5 secs for
unexpected errors)

an alternative to global queue mechanism in #342
not perfect, but easier to implement

closes #342
2020-12-26 16:52:07 +07:00
Bobby Wibowo
3a0810b0be
updated albumsController.js 2020-12-26 15:32:44 +07:00
Bobby Wibowo
771a3f65d2
query amount of temporary uploads for statistics 2020-12-25 21:18:45 +07:00
Bobby Wibowo
346d9864f3
improved codes for statistics
much more expandable, and should be easier to understand overall.

make more statistics operations run concurrently to speed them up.

make linuxDiskStats config key obsolete by using systeminformation
package to also query for any mounted file systems.
2020-12-25 21:06:21 +07:00
Bobby Wibowo
51ab9a6fc5
fs.copyFile() for chunks data on non-default path
Closes #314
2020-11-21 06:31:36 +07:00
Bobby Wibowo
896f74e0ca
allow overriding location of chunks
Closes #302
2020-11-10 01:07:10 +07:00
Bobby Wibowo
9c7241d145
Allow filtering audio files with is:audio
For now only support FLAC, MP3, WAV and WMA
More extensions will come at a later date
2020-11-03 22:51:29 +07:00
Bobby Wibowo
db2897fbe6
statistics: always refetch ClamAV version
also better display when not using ClamAV
2020-11-02 18:23:23 +07:00
Bobby Wibowo
f41e325273
fixed mods editing own albums also disable them 2020-11-01 16:38:36 +07:00
Bobby Wibowo
afdbc7e9a1
Added ClamAV version to Statistics 2020-11-01 07:10:43 +07:00
Bobby Wibowo
88f852584c
!! REPLACED ClamAV BACKEND: clamdjs -> clamscan !!
Update your config file!
2020-11-01 06:35:56 +07:00
Bobby Wibowo
47dd512910
Removed custom ESLint curly rule
Sigh, why did you do this, past me..?

Also fixed "Delete uploads by names".
2020-10-31 01:12:09 +07:00
Bobby Wibowo
22e6c2c3c6
Updated
Replaced all Array.concat() with spread operator and/or push().
Fixed some faulty search logics when using exclusion filters.
Fixed capitalization of mac font in _variables.scss. Stylelint somehow
fricked up again in my previous commit.
2020-10-11 17:32:22 +07:00
Bobby Wibowo
50dc747984
Bug fixed the new collision check
Geez.
2020-09-27 08:04:41 +07:00
Bobby Wibowo
ea15b145b1
Added alt stricter file name collision checks 2020-09-27 05:18:42 +07:00
Bobby Wibowo
67a69bf4e2
Cleaned multerStorageController.js 2020-09-27 03:20:32 +07:00
Bobby Wibowo
c113184385
Improved disk usage entry in statistics
Now will calculate usage as (total - avail).
In Linux, ext filesystems by default reserves 5% of the space to be
usable by root, making them essentially already "used" space.
Originally we didn't take that into account.

Disk usage percentage will now round down: Math.round() -> Math.floor().
The general behavior in other tools such as "df".
2020-09-08 19:04:12 +07:00
Bobby Wibowo
65847232c4
Added "time taken" info for Statistics menu 2020-08-24 08:08:21 +07:00
Bobby Wibowo
0b27babb31
Refactored "hhmmss" -> "uptime" 2020-08-22 01:39:04 +07:00
Bobby Wibowo
41fb14e865
Added system & node uptime to Statistics menu
src/js/misc/utils.js: Added page.getPrettyUptime().

Updated dependency:
helmet: 3.23.3 -> 4.1.0

lolisafe.js: Disabled CSP by default.
Since helmet 4, CSP would instead be enabled by default.
2020-08-22 01:35:30 +07:00
Bobby Wibowo
08db3e55a8
Allow sorting by original names 2020-07-25 16:39:04 +07:00
Bobby Wibowo
a93e34bf87
Init cf-api-token 2020-06-26 13:48:15 +07:00
Bobby Wibowo
49ec795941
Disabled file delete GET API
40dd66297612b2a8429c8fcad2b07170a3ba582b#commitcomment-40060391
2020-06-21 22:21:38 +07:00
Bobby Wibowo
40dd662976
Removed delete URL from upload response
I wasn't aware ShareX wouldn't do a basic GET request to the delete URL,
but would instead open your browser to it.
Unfortunately, due to the structure of lolisafe auth setting token
through HTTP header, which had always been done by JS as it has to get
the value from local storage, just visiting the API from browser will
merely respond with "No token provided".
We'd need cookie-based auth, which I do not want to add.
The old me that added deletion API long ago might have been aware of
this fact, but the me of a few hours ago already forgot..
I'll give this further thoughts at a later date, as ShareX isn't really
a priority to me.

Bumped v1 version string and rebuilt client assets.
2020-06-20 04:01:16 +07:00
Bobby Wibowo
51c8df71bc
Added deletion URL for ShareX or derivatives
For registered users only!
This requires adding a basic GET API for file deletion, so that I did.
Configs which guests download will not include pattern for delete URL,
so they won't get notified of unusable delete URL or anything like that.

dev: Improved logger.debug() to support specifying options for node's
Util.inspect() if an object is set as its last param
(assuming >1 params).
Default options now also includes enabling colors.

src/js/utils.js: Simplified dynamic ShareX config generator.
Among other things, it will now use JSON.stringify().
I don't even remember why we didn't use that in the first place..

Some logic improvements in src/js/home.js.

Bumped v1 version string and rebuilt client assets.
2020-06-20 01:28:23 +07:00
Bobby Wibowo
a4d33c67a2
Updated uploadController.js 2020-06-16 03:01:32 +07:00
Bobby Wibowo
585331c6e5
Added timeout on chunked uploads
The service will now automatically clean up any leftover data from any
inactive & unfinished chunked upload attempts.

Updated config.sample.js for updated description and new sub-option
for chunkSize option.
2020-06-15 23:48:43 +07:00
Bobby Wibowo
b4c8b1d90e
BLAZING FAST CHUNKED UPLOADS 🚀
Inspired by our recent switch to using blake3 for file hashing, chunks
will now be written to a tmp file directly as they're uploaded.
So no more waiting so long for "rebuilding chunks".
There will still be some delay on every following attempts of uploading
each chunks. I'm not sure the specifics, as we're already reusing the
write stream.
2020-06-15 23:14:33 +07:00
Bobby Wibowo
14b97ecbf1
Updated uploadController.js
Make sure .generateThumbs() is always on force mode.
There may be stray thumbnails that weren't stored in DB due to
unexpected errors and whatnot.
2020-06-15 21:10:40 +07:00
Bobby Wibowo
0b8b1ed026
More fixes to thumbnailers
Use fluent-ffmpeg's .screenshots() function instead,
with some countermeasures for weird situations.

No more selective error suppressions.
2020-06-15 21:04:30 +07:00
Bobby Wibowo
0851d71688
Fix SQLite not understanding escape char
I honestly expected Knex.js would've already handled this
2020-06-07 20:51:59 +07:00
Bobby Wibowo
8bcee712ac
A toggle button to show original file names
For admins, preference for your personal uploads list and Manage uploads
aren't shared, just like thumbs/lists toggle.

Non-keyed keywords for filtering will now apply to original names too.

Added a new fontello icon for this button.

Various other things I'm too lazy to write.
2020-06-07 12:29:17 +07:00