Commit Graph

270 Commits

Author SHA1 Message Date
Bobby Wibowo
968fd74676
Updated
No longer initiate expired uploads check if
config.uploads.temporaryUploadsInterval is not set.

"yarn delete-expired" will now list names of the expired uploads too.
2020-05-16 21:45:14 +07:00
Bobby Wibowo
417adbde93
Fixed type-is and keyword exclusion logic 2020-05-03 04:51:22 +07:00
Bobby Wibowo
f189d86abc
Updated uploadController.js
Fixed filtering null albumid
2020-05-03 04:40:04 +07:00
Bobby Wibowo
126cfe0e15
Added type-is filter keys (is:image and is:video)
Fixed text queries threshold being applied to moderators

Bumped v1 version string and rebuilt client assets
2020-05-03 04:32:45 +07:00
Bobby Wibowo
d201b03f59
Added filtering uploads by albumid
This works when listing all uploads as well, but Album column will only
be shown when albumid key is used in the filters.
I plan to someday add Manage Albums menu, which will use "View uploads"
buttons, just like the ones in Manage Users.
2020-05-03 03:30:50 +07:00
Bobby Wibowo
1980d536db
Initialize upload filters for regular users (WIP)
Updated ESLint's ECMA version to 9 (2018).
I'll need to use some lookbehind regex directives from now on.
It's supported since Node 10, which is the oldest version I'll support.

Refactored "can not" -> "cannot".

Filtering for regular users is still work in progress.
Some features aren't working as expected yet.
2020-05-03 02:39:24 +07:00
Bobby Wibowo
41d52d947b
Added support for negative page num
e.g. -1 means last page, -2 means the 2nd from last, and so on
will only accept up to -N where N is the amount of pages
anything lower will alwasy return the first page
this works for both list uploads and list users APIs

fixed some Object.assign in dashboard.js

added bottom control buttons in manage users
2020-05-02 22:42:23 +07:00
Bobby Wibowo
19b4a5e217
Updated Linux-only disk stats
Restore disk usage stats even when config.linuxDiskStats is off

The said config will now only toggle the 'extended' stats which are
disk usage of each directories within the uploads directory
2020-05-02 19:28:13 +07:00
Bobby Wibowo
ff54f088bd
Added client assets update detection
/api/check and /api/token/verify will now pass along v1 version string
if used (when config.cacheControl is used) for both home.js and
dashboard.js to use to compare version strings.
This should help notify users using browsers with a bit more overbearing
caching (generally mobile browsers).

Rebuilt client assets and bumped v1 version string

Slight update to yarn.lock (a small syntax change for node-gyp, idk why)
2020-04-30 05:56:28 +07:00
Bobby Wibowo
922269181c
More improvements to uploads filtering!
Renamed "orderby" key to "sort" (for sorting uploads).

Fixed non-keyed keyword exclusions not working as expected when
more than one are used at the same time.

Support not specifying "from" date when filtering with range keys
(date and expiry).

Proper logic for NULL values inclusion/exclusion when filtering with
user and/or ip keys.

Improved Help? prompt again!!
Also clarify about timezone differences.

Added logger.debug() function.
Basically a shorthand for console.log(require('util').inspect()).

Rebuilt client asssets and bumped v1 version string.
2020-04-20 01:19:20 +07:00
Bobby Wibowo
dc59476592
Updated utilsController.js
Suppress more ffmpeg/ffprobe error messages
2020-04-19 21:52:34 +07:00
Bobby Wibowo
a6156da798
Added .m4v and .m2ts video extensions
Their thumbs can be made with ffmpeg
Run "yarn thumbs 2" to create thumbs for existing uploads with those
extensions

Rebuilt client assets and bumped v1 version string
2020-04-19 21:41:17 +07:00
Bobby Wibowo
3e3878b93c
Massively overhauled uploads filtering endpoint
Please consult the Help? button again to learn all the syntax changes!
The prompt will now also have its width expanded!

Updated dependency, knex: 0.20.13 -> 0.20.15.

Added new dependency: search-query-parser.

Updated all sub-dependencies.

Critical? Admins-only API /users/edit will no longer return NEW password
salt of the user when randomizing their password.

Added page.escape() function to js/misc/utils.js.
This will be used to escape input in upload filters input box.
The same function used in utilsController.js.

Pretty dates will now use / instead of - for date separator.
This is due to the fact that date range key for filtering uploads
can not accepts dates with - separator.
To avoid inconsistency, we will now use / separator.

Caching system of album public pages will now be disabled during
development (yarn develop).

Cleaned up domClick() function in js/dashboard.js.

If using date or expiry range keys when filtering uploads, attach
client's timezone offset to the API requets.
This will be used by the server to calculate timezone differences.

Success prompt when changing token will now auto-close.

Removed ID column from Manage Users.

Improved success prompt when editing users.
This will properly list all of the edited fields at once,
excluding user group change.
Success message for user group change will require a bit more changes
on the API endpoint, which is a bit annoying.

Rebuilt client-side assets and bumped v1 version string.
2020-04-19 02:52:11 +07:00
Bobby Wibowo
1c260c87b0
Added feature to create new user from Manage Users
New admins-only API route: /api/users/create

Restored checkboxes and bulk buttons in Manage Users
Currently useless, as bulk operators are still WIP

Added filter input in Manage Users, currently WIP

Rebuilt client-side assets and bumped v1 version string
2020-04-17 14:25:18 +07:00
Bobby Wibowo
7389ac6c07
Init support for filtering with keywords (WIP) 2020-04-17 13:36:57 +07:00
Bobby Wibowo
940be1ad99
Updated uploadController.js
Filters input in Manage Uploads will now display error if it's filled
with invalid filter/sort keys.
SQLITE_ERROR will also now be forwarded to users if they're moderators.
The expectation is to only display it when used through Manage Uploads.
2020-04-12 16:30:33 +07:00
Bobby Wibowo
325ccfac81
Added config option to enable linux disk stats
The option is disabled by default.
Meaning you need to explicitly set it to true to restore
the old behavior.

The one that uses both "df" and "du" binaries.
I found them to be very slow with 100k+ uploads
on my ancient potato server.
2020-04-08 07:08:18 +07:00
Bobby Wibowo
8f50ba49c0
Added sorting feature for Manage Uploads
This adds support for a new key named 'orderby'.
The key needs to be used with internal column names used in the db.
The key can be used more than once.
Help message in Dashboard's Manage Uploads have been updated to include
examples using the said key.
2020-04-04 23:36:43 +07:00
Bobby Wibowo
24b9a8ac78
Updated pathsController.js 2020-04-04 21:20:17 +07:00
Bobby Wibowo
46fa9677ff
Extended configuration for virus scanning
Added whitelist extensions
Added max size limit
2020-04-04 21:20:01 +07:00
Bobby Wibowo
edce59243b
Better cache control when not using CDN 2020-02-11 17:18:04 +07:00
Bobby Wibowo
d9ddfe8e9a
Implemented stripping tags from images
... and optionally videos using ffmpeg (still experimental).

Users can choose whether to strip tags of their uploads or not from
the home uploader's Config tab (safe.fiery.me will have it disabled
by default).

The behavior will also be applied to the downloadable ShareX config.

Server owners can choose to force either behavior.

Make sure to add the new config from config.sample.js.

---

Fixed all instances of "e.i." to "e.g.".
My English sucks okay.

Bumped v1 version string.
2019-11-29 20:42:53 +07:00
Bobby Wibowo
337a0a61ff
Implemented parallel URL uploads
This doesn't use the server's built-in ability to accept multiple URLs
per API request.
It behaves the same as regular uploads, in that it executes one API call
per file, simultaneously.

I figured this is a better implementation to shift queues faster.

---

Fetch error from URL uploads due to exceeding size limit will no longer
be logged in server's console.

Clients will also see better formatted error message for URL uploads'
file size limit errors.

---

Bumped dependencies:
knex: 0.20.2 -> 0.20.3
systeminformation: 4.15.3 -> 4.16.0

Bumped v1 version string
2019-11-29 17:42:29 +07:00
Bobby Wibowo
ee920f942f
Cache system & disk stats longer
System stats will be cached for only 1000 ms (1 s)

Disk stats will be cached for 60000 ms (60 s)
2019-11-14 15:08:56 +07:00
Bobby Wibowo
2a81e899b4
Fixed stats not re-generating after errors 2019-11-14 14:53:33 +07:00
Bobby Wibowo
7aaf913166
Actually parse error messages of du & df to String 2019-11-14 14:46:49 +07:00
Bobby Wibowo
98d38d3402
Fixed broken disk stats in linux 2019-11-14 14:42:09 +07:00
Bobby Wibowo
3d09df501d
Updated
Better 'df' handling (check the TODO entry for more details).

Simplified a few lines in dashboard.js.

Bumped v1 version string.
2019-11-14 05:06:59 +07:00
Bobby Wibowo
b646a4a82f
Updated uploadController.js
String updates.
2019-11-10 03:41:54 +07:00
Bobby Wibowo
a28d862c14
Added group bypass to virus scanning
Also better-ish scan results handling again, I guess

Updated dependency knex: 0.20.0 -> 0.20.1
2019-11-06 03:35:04 +07:00
Bobby Wibowo
36763c2a77
Security fix
Replaced all instances of DB .whereRaw with their much safer equivalent
methods.

All previous usages of .whereRaw were vulnerable to SQL injections,
cause we were passing the data directly.

Fortunately, they were only used in API routes that required staff
(moderators included) accounts.

---

Updated dependency:
helmet: 3.21.1 -> 3.21.2
2019-10-22 10:52:52 +07:00
Bobby Wibowo
2443390199
Updated utilsController.js
Thumbnails generator: Skip video files that does not have valid duration
metadata, for some reason.
2019-10-15 17:53:23 +07:00
Bobby Wibowo
4f04225ba0
Updated
Added delete user feature.
API: /api/users/delete
json: id<number>, purge[boolean]
By default will not purge out files, but will still clear userid
attribute from the files.
All associated albums will also be marked, and have their ZIP archives
be unliked, if applicable.

Fixed purging albums not properly reporting amount of associated files
that could not be removed, if any.

Fixed moderators being able to disable users by manually sending API
requests, if they at least know of the user IDs.
They could only disable regular users however.
2019-10-07 06:11:07 +07:00
Bobby Wibowo
411d17e1fb
Updated
* Changed colorscheme to black (experimental).

* Fixed ClamAV failing to report names of dirty files.

* Removed built-in support for Google site verification (globals.njk).
Just use HTML verification with public directory,
or manually edit home.njk.

* Bumped v1 version string.
2019-10-06 05:20:59 +07:00
Bobby Wibowo
84a3de0d8d
Updated
Fixed statistics columns width in browsers except Firefox.

And a few other things.
2019-09-28 16:42:49 +07:00
Bobby Wibowo
98a8d03a7f
Updated
Updated controllers to use Promise.all (concurrent processing) wherever
applicable.

Added 2 new entries to todo.md.

Don't check "Select all" checkbox in dashboard when there are no
uploads.

Bumped v1 version string.
2019-09-23 15:09:15 +07:00
Bobby Wibowo
ea37e0b7d3
Updated
Reduced album title max length from 280 to 70.
Existing albums with longer titles will have their titles truncated in
their public pages, but the original titles will still remain in db.

"Load images for preview" will now properly display its saved value.

Increased max parallel uploads to 10.

"yarn develop" will now also restart safe if some Nunjuck templates are
edited (_globals.njk, _layout.njk, and album.njk).

Better meta tags generation.

Bumped v1 version string.
2019-09-19 08:27:19 +07:00
Bobby Wibowo
9e9b0d4439
Updated
Updated some dev dependencies.

---

Gulp will now build CSS/JS files during development into dist-dev
directory, to prevent IDE's Git from unnecessarily building diff's.

Added dist-dev to ignore files.

---

The entire config fille will now be passed to Nunjuck templates for ease
of access of config values.

Root domain for use in Nunjuck templates will now be parsed from config.

Better page titles.

Updated help message for "Uploads history order" option in
homepage's config tab.

Added "Load images for preview" option to homepage's config tab.
Setting this to false will now prevent image uploads from loading
themselves for previews.

Uploads' original names in homepage's uploads history are now
selectable.

Min/max length for user/pass are now enforced in auth's front-end.

Improved performance of album public pages.
Their generated HTML pages will now be cached into memory.
Unfortunately, No-JS version of their pages will be cached separately,
so each album may take up to double the memory space.

File names in thumbnails no longer have their full URLs as tooltips.
I saw no point in that behavior.

Added video icons.
Homepage's uploads history will now display video icons for videos.

"View thumbnail" button in Dashboard is now renamed to "Show preview".
Their icons will also be changed depending on their file types.

Added max length for albums' title & description.
These will be enforced both in front-end and back-end.
Existing albums that have surpassed the limits will not be enforced.

A few other small improvements.
2019-09-17 11:13:41 +07:00
Bobby Wibowo
c9ba16e1d6
Updates (very important to read)
Client-side CSS & JS files will now be processed with Gulp.
Gulp tasks are configured in gulpfile.js file.

CSS files will be optimized with postcss-preset-env, which will
auto-add vendor prefixes and convert any parts necessary for browsers
compatibility.
Afterwards they will be minified with cssnano.

JS files will be optimized with bublé,
likewise for browsers compatibility.
Afterwards they will be minified with terser.

Unprocessed CSS & JS files will now be located at src directory, while
the processed results will be located at dist directory.

Due to bublé, the JS files should now be compatible up to IE 11
at the minimum.
Previously the safe would not work in IE 11 due to extensive usage of
template literals.
Due to that as well, JS files in src directory will now extensively use
arrow functions for my personal comfort (as they will be converted too).

The server will use the processed files at dist directory by default.
If you want to rebuild the files by your own, you can run "yarn build".
Gulp is a development dependency, so make sure you have installed all
development dependencies (e.i. NOT using "yarn install --production").

---

yarn lint -> gulp lint

yarn build -> gulp default

yarn watch -> gulp watch

yarn develop -> env NODE_ENV=development yarn watch

---

Fixed not being able to demote staff into normal users.

/api/token/verify will no longer respond with 401 HTTP error code,
unless an error occurred (which will be 500 HTTP error code).

Fixed /nojs route not displaying file's original name when a duplicate
is found on the server.

Removed is-breeze CSS class name, in favor of Bulma's is-info.

Removed custom styling from auth page, in favor of global styling.

Removed all usage of style HTML attribute in favor of CSS classes.

Renamed js/s/ to js/misc/.

Use loading spinners on dashboard's sidebar menus.

Disable all other sidebar menus when something is loading.

Changed title HTML attribute of disabled control buttons in
uploads & users list.

Hid checkboxes and WIP controls from users list.

Better error messages handling.
Especially homepage will now support CF's HTTP error codes.

Updated various icons.
Also, added fontello config file at public/libs/fontello/config.json.
This should let you edit them more easily with fontello.

Use Gatsby icon for my blog's link in homepage's footer.

A bunch of other improvements here & there.
2019-09-15 13:20:11 +07:00
Bobby Wibowo
4c741f811a
Add todo file, cause otherwise I'll always forget 2019-09-12 18:34:04 +07:00
Bobby Wibowo
2251f7b154
Stoppin' disk stats goin' to deep 😏 2019-09-10 23:47:34 +07:00
Bobby Wibowo
264bd88e88
Updated
Improved performance of /api/users/:id (admin's manage users).

Promisify fs.writeFile.

Improved performance of /api/stats.
By a lot in Linux, cause uploads size will be deferred to "du" binary.
In addition, total usage of whichever disk uploads path resides on will
also be queried using "df" binary.
Non-Linux will have to rely on manual calculation by querying DB
for each upload's size.
But logics related to uploads stats were also improved to be almost
twice as fast as before.

Improved parsing of /api/stats results on dashboard.js.
This allows ease of extending server's response by not having to update
dashboard.js by much, if at all.

Improved codes relating to item menus in dashboard's sidebar.
Finally much cleaner now 👍

No longer use /api/upload/delete API route from dashboard.
Single file deletion and bulk files deletion, both from uploads list or
by names, will now properly use a single function that will use
/api/upload/bulkdelete API route.

/api/upload/delete will still be kept indefinitely for backward support.

Fixed oddities with Select all checkbox.

Replaced all instances of modifying HTML element's style attribute with
adding/removing is-hidden CSS helper class.

Rephrased all instances of "files" to "uploads" in any display strings.

Fixed notice message when server is on private mode.

A few other improvements.
2019-09-10 23:31:27 +07:00
Bobby Wibowo
df8cac0f0b
Fixed some critical bugs 2019-09-09 01:33:07 +07:00
Bobby Wibowo
02e2e402c3
!!! MASSIVE OVERHAUL !!!
As the title says, this commit is a massive overhaul.
I've rewritten/restrucuted almost everything in the controller scripts.
Because of that, there's a considerable possibility that I've broken
something somewhere.

Notable changes:

Added temporary uploads.

Removed file name length changer from dashboard,
in favor of an equivalent in homepage config tab.
This allows non-registered users to also set file name length.

A bunch of other undocmented stuff.
I don't know, I'm too tired to remember them all.
2019-09-08 08:56:29 +07:00
Bobby Wibowo
05b905bc9b
Updated
Upgraded dependencies.

Stop adding cache-control header to album zip files unless
config.cacheControl is enabled.

Updated CSS files.
Moved thumbnail-related styling to thumbs.css.
Various other fixes & improvements.

Moved render.js from public/js to public/js/s.

Removed sharex.js in favor of public/js/s/utils.js.
Moved getPrettyDate() & getPrettyBytes() to that JS file as well.

Added lsKeys global variable wherever applicable.
Previously the idea was only used in dashboard.js.

Added No-JS version of album public pages.
You'll only have to add ?nojs to the URL.
Viewing the regular version with JS disabled will show a notice with
a link to the No-JS version of the particular album.
Overall page size of the regular version will now be lower as well,
since there'll be no need to add noscript tag for each thumbs.

No longer show Administrator section to non-admin in the dashboard.
Moderators will ONLY be able to see manage users menu as well.

Simplified FAQ wherever applicable.
Added a new FAQ about bug report or feature request.

Updated link for Firefox extension.
Also pushed Firefox link before Chrome, cause I like it more.

Added browser settings menu to dashboard.
This allows you to choose file size unit (kilobyte vs kibibyte) for that
specific browser.
The preference will be used on homepage, dashboard and album pages.
This also allows you to set chunk size and maximum parallel uploads
for the homepage uploader.

All menu links in the dashboard will now scroll to the content once
loaded.
Previously it would only do so with manage uploads/users when
switching pages.

Refactored all instances of for-in & for-of loop from browser JS files.
For the sake of uniformity, for now.
2019-09-02 02:23:16 +07:00
Bobby Wibowo
dcc393e7e0
Updated
Trim user & pass upon login / registration, on both client & server.
Users that might have already had whitespace prefix/suffix will need to
have their usernames manually updated in the database.

Warp various text inputs in the dashboard into HTML form.
This will make them be submittable when pressing Enter on the keyboard.

Switching page using the prev/next buttons, pagination, and jump to page
input, will now scroll the view to the top of the page element.

Bumped v1 version string.
2019-08-27 05:00:57 +07:00
Bobby Wibowo
7e3d177d00
Updated
Added logger.js to format console logs (adding timestamps).

Re-ordered modules loading in lolisafe.js, and a few other minor edits.

Updated dev dependencies.

A few other minor edits.
2019-08-27 00:02:06 +07:00
Bobby Wibowo
51f12ad900
Updated utilsController.js
Used .some() instead of .find().
Probably a tiny bit more efficient.
2019-08-23 20:35:28 +07:00
Bobby Wibowo
f4fa6b6a96
Updated
On video thumbnails generation, skip video files that do not have any
video streams/channels.
This will prevent unnecessary spawning of ffmpeg for those files.

Also suppress error logging when video streams/channels are not found.
2019-08-23 20:31:44 +07:00
Bobby Wibowo
e45d854c09
Updated
* Updated path resolving for upload folder option.
This SHOULD now allow using an absolute path, even those outside of
the lolisafe installation.

* Added a config option at uploads -> generateThumbs, named placeholder.
It's a string option that lets you set path of the placeholder image
for files whose thumbnail could not be generated.
It defaults to the old hard-coded path when set to falsy value.

* Fixed thumbs script not using upload folder from config file.

* Added filters for thumb generation error handling.
This is used to ignore some common error messages,
such as file formats not being supported.
2019-08-23 16:49:53 +07:00