Commit Graph

132 Commits

Author SHA1 Message Date
Bobby
6788dc2094
fix: align with clamscan v2 2022-04-15 15:36:50 +07:00
Bobby
ffc82f6a2a
feat: logger.debug() outputs only on dev mode 2022-04-15 15:30:32 +07:00
Bobby
e31af2d267
fix: round down system uptime
invalidate system info cache quicker (1s to 0.5s)
2021-09-08 00:46:08 +07:00
Bobby Wibowo
ae6d7936c7
feat: blacklist extensions for strip tags
GIFs are known to not work without custom globally-installed libvips
with ImageMagick or GraphicsMagick support.
https://sharp.pixelplumbing.com/api-output#gif
https://sharp.pixelplumbing.com/install#custom-libvips

It's highly recommended to update your config following the changes to
the sample config file.

This also addressed a bug where images would still get recorded to DB
despite them not existing physically due to strip tags errors.
2021-02-01 06:13:37 +07:00
Bobby Wibowo
5a74776978
fix: thumbnails not generated with mixed case
e.g. .jpG, .JPG, .PnG, etc.

Also simplified some codes.
2021-02-01 05:23:53 +07:00
Bobby Wibowo
c921db7ee2
docs: better extension names parsing 2021-01-29 23:17:56 +07:00
Bobby Wibowo
631d14d887
feat: better extension names parsing
It will now preserve upper/lower/mixed case.

It will now properly preserve all tarballs (#370).
2021-01-29 23:15:24 +07:00
Bobby Wibowo
a752bb89e8
fix: added failsafe to available disk stats 2021-01-29 22:44:58 +07:00
Bobby Wibowo
968494bb37
fix: systeminformation v5 breaking changes
Made the codes for stats generation a bit more readable.

Usage percentage for file systems will now properly reflect "non-root"
usage percentage in ext2/3/4 file systems.
2021-01-27 23:50:45 +07:00
Bobby Wibowo
81cf940160
feat: added audios count to uploads stats
As always, this uses hardcoded whitelisted audio extensions (the ones
that are also internally used for "is:audio" filter).
2021-01-08 11:22:31 +07:00
Bobby Wibowo
991d743ef0
feat: add "files in albums" to albums stats 2021-01-08 11:18:50 +07:00
Bobby Wibowo
defa3f2a8c
perf: skip checking zips on disk for album stats
Closes #353.
2021-01-08 11:03:26 +07:00
Bobby Wibowo
c1562e11f4
refactor: ClientError on utilsController
Improved utils.authorize().

Various fixes to thumbnailer and tags stripper.
2021-01-08 10:48:08 +07:00
Bobby Wibowo
1142b64e3c
refactor: no throw literal in utilsController 2021-01-08 09:56:01 +07:00
Bobby Wibowo
0a19b025a0
removed utils.parallelLimit
well, that was a pointless endeavor.
i wasn't thinking clearly.

also updated all scripts that previously used them, to use a combo of
for-loop and setInterval to print progress.
2020-12-27 19:44:10 +07:00
Bobby Wibowo
18db55ba24
Added scripts/rebuild-hashes.js
Closes #345
2020-12-27 18:16:01 +07:00
Bobby Wibowo
d7b11e7e56
try cf purge up to 3 times
with preset delay in between (60 secs if rate limited, 5 secs for
unexpected errors)

an alternative to global queue mechanism in #342
not perfect, but easier to implement

closes #342
2020-12-26 16:52:07 +07:00
Bobby Wibowo
771a3f65d2
query amount of temporary uploads for statistics 2020-12-25 21:18:45 +07:00
Bobby Wibowo
346d9864f3
improved codes for statistics
much more expandable, and should be easier to understand overall.

make more statistics operations run concurrently to speed them up.

make linuxDiskStats config key obsolete by using systeminformation
package to also query for any mounted file systems.
2020-12-25 21:06:21 +07:00
Bobby Wibowo
9c7241d145
Allow filtering audio files with is:audio
For now only support FLAC, MP3, WAV and WMA
More extensions will come at a later date
2020-11-03 22:51:29 +07:00
Bobby Wibowo
db2897fbe6
statistics: always refetch ClamAV version
also better display when not using ClamAV
2020-11-02 18:23:23 +07:00
Bobby Wibowo
afdbc7e9a1
Added ClamAV version to Statistics 2020-11-01 07:10:43 +07:00
Bobby Wibowo
88f852584c
!! REPLACED ClamAV BACKEND: clamdjs -> clamscan !!
Update your config file!
2020-11-01 06:35:56 +07:00
Bobby Wibowo
47dd512910
Removed custom ESLint curly rule
Sigh, why did you do this, past me..?

Also fixed "Delete uploads by names".
2020-10-31 01:12:09 +07:00
Bobby Wibowo
22e6c2c3c6
Updated
Replaced all Array.concat() with spread operator and/or push().
Fixed some faulty search logics when using exclusion filters.
Fixed capitalization of mac font in _variables.scss. Stylelint somehow
fricked up again in my previous commit.
2020-10-11 17:32:22 +07:00
Bobby Wibowo
c113184385
Improved disk usage entry in statistics
Now will calculate usage as (total - avail).
In Linux, ext filesystems by default reserves 5% of the space to be
usable by root, making them essentially already "used" space.
Originally we didn't take that into account.

Disk usage percentage will now round down: Math.round() -> Math.floor().
The general behavior in other tools such as "df".
2020-09-08 19:04:12 +07:00
Bobby Wibowo
65847232c4
Added "time taken" info for Statistics menu 2020-08-24 08:08:21 +07:00
Bobby Wibowo
0b27babb31
Refactored "hhmmss" -> "uptime" 2020-08-22 01:39:04 +07:00
Bobby Wibowo
41fb14e865
Added system & node uptime to Statistics menu
src/js/misc/utils.js: Added page.getPrettyUptime().

Updated dependency:
helmet: 3.23.3 -> 4.1.0

lolisafe.js: Disabled CSP by default.
Since helmet 4, CSP would instead be enabled by default.
2020-08-22 01:35:30 +07:00
Bobby Wibowo
a93e34bf87
Init cf-api-token 2020-06-26 13:48:15 +07:00
Bobby Wibowo
0b8b1ed026
More fixes to thumbnailers
Use fluent-ffmpeg's .screenshots() function instead,
with some countermeasures for weird situations.

No more selective error suppressions.
2020-06-15 21:04:30 +07:00
Bobby Wibowo
6d5ee7dd63
Thumbs size 2020-06-06 19:43:20 +07:00
Bobby Wibowo
46c84db64f
Merge branch 'patch-3' of https://github.com/camjac251/lolisafe into camjac251-patch-3 2020-06-06 19:34:46 +07:00
Bobby Wibowo
28c64d531a
Potentially much better video thumbnails
Notable changes, use fluent-ffmpeg's functions instead of passing raw
ffmpeg switches with inputOptions() or outputOptions().

It seems fluent-ffmpeg may be doing some janks with outputOptions().
I can't get select="eq(pict_type\\,I)" to work with it, but it works
with videoFilters(). Weird stuff.
2020-06-06 19:28:42 +07:00
camjac251
cf2593465a Use top variable over config 2020-06-06 02:39:53 -05:00
Bobby Wibowo
8b4b0e79c5
Improved albums public page cache and more
Removed its dependency towards albums' editedAt property.
Editing album's metas (name, description, etc) will no longer update
its editedAt property.
Instead it will now ONLY be updated when adding/removing files to/from
it. Just like how it was meant to be, which was to be used to check
whether it's necessary to re-generate their downloadable ZIPs.

Albums public page cache will still be properly invalidated when
adding/removing files to/from it, as well as after editing their metas.

Added views/album-notice.njk to be used to render okay-ish notice when
an album's public page is still being generated.
I was originally thinking of using it for disabled albums as well, but
I refrained from it to reduce the possibility of disabled album IDs from
being easily scanned (as it just returns 404 now).

Removed invalidatedAt property from stats cache. Instead their caches
will immediately be nullified as they should (thus frees up memory
slightly as well).

Stats cache for albums will now only be cleared when truly necessary.
As in, adding/removing files to/from albums will no longer clear them.

Updated Nunjucks files to properly use h1, h2, h3 tags in actual
hierarchical orders.
Elements that don't need to use hX tags will now use P instead.
Nothing changes visually, only structurally.

Fixed some elements in Nunjucks using single quotes instead of
double quotes. They'd have worked the same, but consistency.

Added h1 title in FAQ page.

Make text for no JS warning a bit bigger, and improved the phrasing
a little bit.
2020-06-03 10:44:24 +07:00
camjac251
9a4c0d5cea Update utilsController.js 2020-06-01 04:27:57 -05:00
camjac251
776414814d add user config option for thumbnail size 2020-06-01 04:17:23 -05:00
camjac251
de70b93cc6
thumbnail generation fix and scale
ffmpeg can sometimes generate broken thumbnails (sometimes a full grey image) when it seeks in the input instead of the output. When it's added after the input, it is a bit slower but it is more stable and will fixes the issue for various formats like HEVC or Prores.

The resolution could benefit from being increased 3x for users who scale up the webpage in their browser or tablet views. It becomes scaled up 200% or 300% when viewed on a smaller screen.
2020-05-29 05:51:38 -05:00
camjac251
1efa8c4ebc
more video extensions for thumbs and reorder
added additional extensions to the video list for more support in thumbnails on files (all are common filetypes so ffmpeg should support the various codecs needed)
Also reordered the image and video list alphabetically
2020-05-25 22:47:24 -05:00
Bobby Wibowo
968fd74676
Updated
No longer initiate expired uploads check if
config.uploads.temporaryUploadsInterval is not set.

"yarn delete-expired" will now list names of the expired uploads too.
2020-05-16 21:45:14 +07:00
Bobby Wibowo
19b4a5e217
Updated Linux-only disk stats
Restore disk usage stats even when config.linuxDiskStats is off

The said config will now only toggle the 'extended' stats which are
disk usage of each directories within the uploads directory
2020-05-02 19:28:13 +07:00
Bobby Wibowo
dc59476592
Updated utilsController.js
Suppress more ffmpeg/ffprobe error messages
2020-04-19 21:52:34 +07:00
Bobby Wibowo
a6156da798
Added .m4v and .m2ts video extensions
Their thumbs can be made with ffmpeg
Run "yarn thumbs 2" to create thumbs for existing uploads with those
extensions

Rebuilt client assets and bumped v1 version string
2020-04-19 21:41:17 +07:00
Bobby Wibowo
3e3878b93c
Massively overhauled uploads filtering endpoint
Please consult the Help? button again to learn all the syntax changes!
The prompt will now also have its width expanded!

Updated dependency, knex: 0.20.13 -> 0.20.15.

Added new dependency: search-query-parser.

Updated all sub-dependencies.

Critical? Admins-only API /users/edit will no longer return NEW password
salt of the user when randomizing their password.

Added page.escape() function to js/misc/utils.js.
This will be used to escape input in upload filters input box.
The same function used in utilsController.js.

Pretty dates will now use / instead of - for date separator.
This is due to the fact that date range key for filtering uploads
can not accepts dates with - separator.
To avoid inconsistency, we will now use / separator.

Caching system of album public pages will now be disabled during
development (yarn develop).

Cleaned up domClick() function in js/dashboard.js.

If using date or expiry range keys when filtering uploads, attach
client's timezone offset to the API requets.
This will be used by the server to calculate timezone differences.

Success prompt when changing token will now auto-close.

Removed ID column from Manage Users.

Improved success prompt when editing users.
This will properly list all of the edited fields at once,
excluding user group change.
Success message for user group change will require a bit more changes
on the API endpoint, which is a bit annoying.

Rebuilt client-side assets and bumped v1 version string.
2020-04-19 02:52:11 +07:00
Bobby Wibowo
325ccfac81
Added config option to enable linux disk stats
The option is disabled by default.
Meaning you need to explicitly set it to true to restore
the old behavior.

The one that uses both "df" and "du" binaries.
I found them to be very slow with 100k+ uploads
on my ancient potato server.
2020-04-08 07:08:18 +07:00
Bobby Wibowo
46fa9677ff
Extended configuration for virus scanning
Added whitelist extensions
Added max size limit
2020-04-04 21:20:01 +07:00
Bobby Wibowo
d9ddfe8e9a
Implemented stripping tags from images
... and optionally videos using ffmpeg (still experimental).

Users can choose whether to strip tags of their uploads or not from
the home uploader's Config tab (safe.fiery.me will have it disabled
by default).

The behavior will also be applied to the downloadable ShareX config.

Server owners can choose to force either behavior.

Make sure to add the new config from config.sample.js.

---

Fixed all instances of "e.i." to "e.g.".
My English sucks okay.

Bumped v1 version string.
2019-11-29 20:42:53 +07:00
Bobby Wibowo
ee920f942f
Cache system & disk stats longer
System stats will be cached for only 1000 ms (1 s)

Disk stats will be cached for 60000 ms (60 s)
2019-11-14 15:08:56 +07:00
Bobby Wibowo
2a81e899b4
Fixed stats not re-generating after errors 2019-11-14 14:53:33 +07:00