Commit Graph

146 Commits

Author SHA1 Message Date
camjac251
9a4c0d5cea Update utilsController.js 2020-06-01 04:27:57 -05:00
camjac251
776414814d add user config option for thumbnail size 2020-06-01 04:17:23 -05:00
camjac251
de70b93cc6
thumbnail generation fix and scale
ffmpeg can sometimes generate broken thumbnails (sometimes a full grey image) when it seeks in the input instead of the output. When it's added after the input, it is a bit slower but it is more stable and will fixes the issue for various formats like HEVC or Prores.

The resolution could benefit from being increased 3x for users who scale up the webpage in their browser or tablet views. It becomes scaled up 200% or 300% when viewed on a smaller screen.
2020-05-29 05:51:38 -05:00
camjac251
1efa8c4ebc
more video extensions for thumbs and reorder
added additional extensions to the video list for more support in thumbnails on files (all are common filetypes so ffmpeg should support the various codecs needed)
Also reordered the image and video list alphabetically
2020-05-25 22:47:24 -05:00
Bobby Wibowo
968fd74676
Updated
No longer initiate expired uploads check if
config.uploads.temporaryUploadsInterval is not set.

"yarn delete-expired" will now list names of the expired uploads too.
2020-05-16 21:45:14 +07:00
Bobby Wibowo
19b4a5e217
Updated Linux-only disk stats
Restore disk usage stats even when config.linuxDiskStats is off

The said config will now only toggle the 'extended' stats which are
disk usage of each directories within the uploads directory
2020-05-02 19:28:13 +07:00
Bobby Wibowo
dc59476592
Updated utilsController.js
Suppress more ffmpeg/ffprobe error messages
2020-04-19 21:52:34 +07:00
Bobby Wibowo
a6156da798
Added .m4v and .m2ts video extensions
Their thumbs can be made with ffmpeg
Run "yarn thumbs 2" to create thumbs for existing uploads with those
extensions

Rebuilt client assets and bumped v1 version string
2020-04-19 21:41:17 +07:00
Bobby Wibowo
3e3878b93c
Massively overhauled uploads filtering endpoint
Please consult the Help? button again to learn all the syntax changes!
The prompt will now also have its width expanded!

Updated dependency, knex: 0.20.13 -> 0.20.15.

Added new dependency: search-query-parser.

Updated all sub-dependencies.

Critical? Admins-only API /users/edit will no longer return NEW password
salt of the user when randomizing their password.

Added page.escape() function to js/misc/utils.js.
This will be used to escape input in upload filters input box.
The same function used in utilsController.js.

Pretty dates will now use / instead of - for date separator.
This is due to the fact that date range key for filtering uploads
can not accepts dates with - separator.
To avoid inconsistency, we will now use / separator.

Caching system of album public pages will now be disabled during
development (yarn develop).

Cleaned up domClick() function in js/dashboard.js.

If using date or expiry range keys when filtering uploads, attach
client's timezone offset to the API requets.
This will be used by the server to calculate timezone differences.

Success prompt when changing token will now auto-close.

Removed ID column from Manage Users.

Improved success prompt when editing users.
This will properly list all of the edited fields at once,
excluding user group change.
Success message for user group change will require a bit more changes
on the API endpoint, which is a bit annoying.

Rebuilt client-side assets and bumped v1 version string.
2020-04-19 02:52:11 +07:00
Bobby Wibowo
325ccfac81
Added config option to enable linux disk stats
The option is disabled by default.
Meaning you need to explicitly set it to true to restore
the old behavior.

The one that uses both "df" and "du" binaries.
I found them to be very slow with 100k+ uploads
on my ancient potato server.
2020-04-08 07:08:18 +07:00
Bobby Wibowo
46fa9677ff
Extended configuration for virus scanning
Added whitelist extensions
Added max size limit
2020-04-04 21:20:01 +07:00
Bobby Wibowo
d9ddfe8e9a
Implemented stripping tags from images
... and optionally videos using ffmpeg (still experimental).

Users can choose whether to strip tags of their uploads or not from
the home uploader's Config tab (safe.fiery.me will have it disabled
by default).

The behavior will also be applied to the downloadable ShareX config.

Server owners can choose to force either behavior.

Make sure to add the new config from config.sample.js.

---

Fixed all instances of "e.i." to "e.g.".
My English sucks okay.

Bumped v1 version string.
2019-11-29 20:42:53 +07:00
Bobby Wibowo
ee920f942f
Cache system & disk stats longer
System stats will be cached for only 1000 ms (1 s)

Disk stats will be cached for 60000 ms (60 s)
2019-11-14 15:08:56 +07:00
Bobby Wibowo
2a81e899b4
Fixed stats not re-generating after errors 2019-11-14 14:53:33 +07:00
Bobby Wibowo
7aaf913166
Actually parse error messages of du & df to String 2019-11-14 14:46:49 +07:00
Bobby Wibowo
98d38d3402
Fixed broken disk stats in linux 2019-11-14 14:42:09 +07:00
Bobby Wibowo
3d09df501d
Updated
Better 'df' handling (check the TODO entry for more details).

Simplified a few lines in dashboard.js.

Bumped v1 version string.
2019-11-14 05:06:59 +07:00
Bobby Wibowo
a28d862c14
Added group bypass to virus scanning
Also better-ish scan results handling again, I guess

Updated dependency knex: 0.20.0 -> 0.20.1
2019-11-06 03:35:04 +07:00
Bobby Wibowo
36763c2a77
Security fix
Replaced all instances of DB .whereRaw with their much safer equivalent
methods.

All previous usages of .whereRaw were vulnerable to SQL injections,
cause we were passing the data directly.

Fortunately, they were only used in API routes that required staff
(moderators included) accounts.

---

Updated dependency:
helmet: 3.21.1 -> 3.21.2
2019-10-22 10:52:52 +07:00
Bobby Wibowo
2443390199
Updated utilsController.js
Thumbnails generator: Skip video files that does not have valid duration
metadata, for some reason.
2019-10-15 17:53:23 +07:00
Bobby Wibowo
98a8d03a7f
Updated
Updated controllers to use Promise.all (concurrent processing) wherever
applicable.

Added 2 new entries to todo.md.

Don't check "Select all" checkbox in dashboard when there are no
uploads.

Bumped v1 version string.
2019-09-23 15:09:15 +07:00
Bobby Wibowo
ea37e0b7d3
Updated
Reduced album title max length from 280 to 70.
Existing albums with longer titles will have their titles truncated in
their public pages, but the original titles will still remain in db.

"Load images for preview" will now properly display its saved value.

Increased max parallel uploads to 10.

"yarn develop" will now also restart safe if some Nunjuck templates are
edited (_globals.njk, _layout.njk, and album.njk).

Better meta tags generation.

Bumped v1 version string.
2019-09-19 08:27:19 +07:00
Bobby Wibowo
9e9b0d4439
Updated
Updated some dev dependencies.

---

Gulp will now build CSS/JS files during development into dist-dev
directory, to prevent IDE's Git from unnecessarily building diff's.

Added dist-dev to ignore files.

---

The entire config fille will now be passed to Nunjuck templates for ease
of access of config values.

Root domain for use in Nunjuck templates will now be parsed from config.

Better page titles.

Updated help message for "Uploads history order" option in
homepage's config tab.

Added "Load images for preview" option to homepage's config tab.
Setting this to false will now prevent image uploads from loading
themselves for previews.

Uploads' original names in homepage's uploads history are now
selectable.

Min/max length for user/pass are now enforced in auth's front-end.

Improved performance of album public pages.
Their generated HTML pages will now be cached into memory.
Unfortunately, No-JS version of their pages will be cached separately,
so each album may take up to double the memory space.

File names in thumbnails no longer have their full URLs as tooltips.
I saw no point in that behavior.

Added video icons.
Homepage's uploads history will now display video icons for videos.

"View thumbnail" button in Dashboard is now renamed to "Show preview".
Their icons will also be changed depending on their file types.

Added max length for albums' title & description.
These will be enforced both in front-end and back-end.
Existing albums that have surpassed the limits will not be enforced.

A few other small improvements.
2019-09-17 11:13:41 +07:00
Bobby Wibowo
c9ba16e1d6
Updates (very important to read)
Client-side CSS & JS files will now be processed with Gulp.
Gulp tasks are configured in gulpfile.js file.

CSS files will be optimized with postcss-preset-env, which will
auto-add vendor prefixes and convert any parts necessary for browsers
compatibility.
Afterwards they will be minified with cssnano.

JS files will be optimized with bublé,
likewise for browsers compatibility.
Afterwards they will be minified with terser.

Unprocessed CSS & JS files will now be located at src directory, while
the processed results will be located at dist directory.

Due to bublé, the JS files should now be compatible up to IE 11
at the minimum.
Previously the safe would not work in IE 11 due to extensive usage of
template literals.
Due to that as well, JS files in src directory will now extensively use
arrow functions for my personal comfort (as they will be converted too).

The server will use the processed files at dist directory by default.
If you want to rebuild the files by your own, you can run "yarn build".
Gulp is a development dependency, so make sure you have installed all
development dependencies (e.i. NOT using "yarn install --production").

---

yarn lint -> gulp lint

yarn build -> gulp default

yarn watch -> gulp watch

yarn develop -> env NODE_ENV=development yarn watch

---

Fixed not being able to demote staff into normal users.

/api/token/verify will no longer respond with 401 HTTP error code,
unless an error occurred (which will be 500 HTTP error code).

Fixed /nojs route not displaying file's original name when a duplicate
is found on the server.

Removed is-breeze CSS class name, in favor of Bulma's is-info.

Removed custom styling from auth page, in favor of global styling.

Removed all usage of style HTML attribute in favor of CSS classes.

Renamed js/s/ to js/misc/.

Use loading spinners on dashboard's sidebar menus.

Disable all other sidebar menus when something is loading.

Changed title HTML attribute of disabled control buttons in
uploads & users list.

Hid checkboxes and WIP controls from users list.

Better error messages handling.
Especially homepage will now support CF's HTTP error codes.

Updated various icons.
Also, added fontello config file at public/libs/fontello/config.json.
This should let you edit them more easily with fontello.

Use Gatsby icon for my blog's link in homepage's footer.

A bunch of other improvements here & there.
2019-09-15 13:20:11 +07:00
Bobby Wibowo
4c741f811a
Add todo file, cause otherwise I'll always forget 2019-09-12 18:34:04 +07:00
Bobby Wibowo
2251f7b154
Stoppin' disk stats goin' to deep 😏 2019-09-10 23:47:34 +07:00
Bobby Wibowo
264bd88e88
Updated
Improved performance of /api/users/:id (admin's manage users).

Promisify fs.writeFile.

Improved performance of /api/stats.
By a lot in Linux, cause uploads size will be deferred to "du" binary.
In addition, total usage of whichever disk uploads path resides on will
also be queried using "df" binary.
Non-Linux will have to rely on manual calculation by querying DB
for each upload's size.
But logics related to uploads stats were also improved to be almost
twice as fast as before.

Improved parsing of /api/stats results on dashboard.js.
This allows ease of extending server's response by not having to update
dashboard.js by much, if at all.

Improved codes relating to item menus in dashboard's sidebar.
Finally much cleaner now 👍

No longer use /api/upload/delete API route from dashboard.
Single file deletion and bulk files deletion, both from uploads list or
by names, will now properly use a single function that will use
/api/upload/bulkdelete API route.

/api/upload/delete will still be kept indefinitely for backward support.

Fixed oddities with Select all checkbox.

Replaced all instances of modifying HTML element's style attribute with
adding/removing is-hidden CSS helper class.

Rephrased all instances of "files" to "uploads" in any display strings.

Fixed notice message when server is on private mode.

A few other improvements.
2019-09-10 23:31:27 +07:00
Bobby Wibowo
df8cac0f0b
Fixed some critical bugs 2019-09-09 01:33:07 +07:00
Bobby Wibowo
02e2e402c3
!!! MASSIVE OVERHAUL !!!
As the title says, this commit is a massive overhaul.
I've rewritten/restrucuted almost everything in the controller scripts.
Because of that, there's a considerable possibility that I've broken
something somewhere.

Notable changes:

Added temporary uploads.

Removed file name length changer from dashboard,
in favor of an equivalent in homepage config tab.
This allows non-registered users to also set file name length.

A bunch of other undocmented stuff.
I don't know, I'm too tired to remember them all.
2019-09-08 08:56:29 +07:00
Bobby Wibowo
7e3d177d00
Updated
Added logger.js to format console logs (adding timestamps).

Re-ordered modules loading in lolisafe.js, and a few other minor edits.

Updated dev dependencies.

A few other minor edits.
2019-08-27 00:02:06 +07:00
Bobby Wibowo
51f12ad900
Updated utilsController.js
Used .some() instead of .find().
Probably a tiny bit more efficient.
2019-08-23 20:35:28 +07:00
Bobby Wibowo
f4fa6b6a96
Updated
On video thumbnails generation, skip video files that do not have any
video streams/channels.
This will prevent unnecessary spawning of ffmpeg for those files.

Also suppress error logging when video streams/channels are not found.
2019-08-23 20:31:44 +07:00
Bobby Wibowo
e45d854c09
Updated
* Updated path resolving for upload folder option.
This SHOULD now allow using an absolute path, even those outside of
the lolisafe installation.

* Added a config option at uploads -> generateThumbs, named placeholder.
It's a string option that lets you set path of the placeholder image
for files whose thumbnail could not be generated.
It defaults to the old hard-coded path when set to falsy value.

* Fixed thumbs script not using upload folder from config file.

* Added filters for thumb generation error handling.
This is used to ignore some common error messages,
such as file formats not being supported.
2019-08-23 16:49:53 +07:00
Bobby Wibowo
3a398721b5
Updated
* Replaced all instances of getElementById and getElementsByClassName
with querySelector or querySelectorAll.

* Updated utilsController.js to stop disabling
no-async-promise-executor eslint rule.

* Removed unused lines in dashboard.njk.

* Refactored maxFileSize to maxSize in home.{css,js,njk}.

* Updated ClamAV codes in lolisafe.js. No more pinging.
Since querying version will also check connection anyway.

* Option "Upload to album" in homepage is now selectable.
Selecting this option will restore the uploader to not associate files
with an album.

* Fixed uploader to properly respect server's max file size.
Also updated error message of file size to use MB instead of MiB.

* Creating an album from homepage will automatically select the album.

* Updated Dropzone.js to v5.5.0.

* Bumped v1 & v3 version strings.

* Various other small fixes.
2019-08-20 09:16:34 +07:00
Bobby Wibowo
e6bb94692b
Oversights 2019-08-04 06:42:35 +07:00
Bobby Wibowo
6496b03b71
Updates
Updated deps.

Re-enabled eslint rule no-async-promise-executor.
Updated some codes to follow the said rule.
Though I had to disable the rule in a line at utilsController.js
due to complexity.
I'll give it more thoughts in the future.

Bumped v1 version string.
2019-08-04 06:37:55 +07:00
Bobby Wibowo
83eb4872e9
Updated utilsController.js
* Simplified some logics.

* Try to unlink thumbnail when failing to generate video thumbnail,
since ffmpeg may have already created an incomplete thumbnail.
2019-07-21 05:15:59 +07:00
Bobby Wibowo
5c41fc9249
Updates
utilsController.js:
+ /api/stats: Cache invalidation requests will now only store timestamp
of the request instead of purging the cache entirely.
Subsequent requests to the API will no longer attempt to generate stats
if a previous request before it is still generating, since it can
take a while in big sites.
So instead, the cached stats will be returned to them, regardless of
whether it's still valid.
This should avoid "race condition" in sites with multiple admins.

* dashboard.js:
+ Added an appendix into "Delete by names" feature in dashboard to
inform staffs that they can use the feature to delete files by other
users as well.
+ Added comments explaning the need of page.domClick() function.
+ Addition to the change regarding /api/stats route: If there is no
cached stats, but a previous request is still generating, inform
the user about it.

config.sample.js + lolisafe.js:
+ We now support configuring rate limits through config file.
No more hard-coded rate limits.
Don't forget to update your config.js accordingly.

package.json + yarn.lock:
+ Updated dependencies.
2019-06-04 02:40:24 +07:00
Bobby Wibowo
13081ef38a
Re-worked caching for statistics
I guess I'll work on adding charts someday.
2019-04-12 07:45:33 +07:00
Bobby Wibowo
3e336e8c6d
Updates
Got rid of caching of statistics.
The previous implementation wasn't perfect, and too lazy to improve it.
2019-04-06 00:42:38 +07:00
Bobby Wibowo
8c3fb78135
Upates
* Added Statistics menu to Administration items in dashboard.

* Added /api/stats route.

Imo, my implementation of this whole thing is rather dirty-ish, but hey
as long as it works.

I'll be using lolisafe2 for future devs tbh.
2019-04-06 00:32:52 +07:00
Bobby Wibowo
57c17d3157
Updates
Updated utilsController.bulkDeleteFiles() to chunk opeartion by a max of
999 values (which is SQLite's default var limit).
However, there's a risk of hitting SQLITE_BUSY since we attempt to
delete chunks all at once (as can be seen by how we use Promise.all
instead of patiently waiting the chunks one by one).
However, I'm not really sure why, but the operations will still
be finished eventually, so I'll let that be for now (?).

Fixed a silly concat in albumsController.js.
2019-02-05 10:36:14 +07:00
Bobby Wibowo
4f60b01676
Updated utilsController.js
Whoops.
2019-01-31 16:37:50 +07:00
Bobby Wibowo
d5c24165cf
Updates
Updated Cloudfalre's cache purge utility.
It will now split URLs into chunks of 30 URLs which then will be purged
one chunk at a time.
I just found out Cloudflare's API have a limit of 30 URLs for the API.
2019-01-31 16:29:34 +07:00
Bobby Wibowo
8140ff4d71
Updates
Rewritten function to generate video thumbnails with ffmpeg.
This should be much faster than ever.
This should also solve an issue where potrait videos could have their
thumbnails be taller than 200px, since it was only forcing width to be
no larger than 200px.

Updated dashboard's styling to make sure potrait thumbnails (only matter
for video thumbnails) not going out of its container.

Updated thumbs.js (yarn thumbs) script to display elapsed time for each
operation in seconds.

Bumped v1 version string (for dashboard.css).
2019-01-30 02:50:45 +07:00
Bobby Wibowo
e38037e660
Updates
Updated thumbs.js script to allow purging CF's cache of succeeded files

Updated utils.purgeCloudflareCache() function accordingly
2019-01-18 10:40:15 +07:00
Bobby Wibowo
8482049584
Updates
Removed .bmp from list of allowed image extensions for thumbs gen
Sharp doesn't support BMP format

Increased seek timestamp for video thumbs gen from 1% to 20%
2019-01-18 10:25:30 +07:00
Bobby Wibowo
209f5429cd
Updates
Added .tiff, .tif, and .svg to allowed image extensions for thumbs gen

Added .svg to allowed image extensions for preview in uploads result
2019-01-18 08:15:53 +07:00
Bobby Wibowo
8780d6429b
Updates
NoJS uploader will now display the original file names in the results.

lolisafe.js will now automaticaly purge Cloudflare's cache of frontend
pages everytime it launches.
Of course this only applies when cacheControl is on in config file.
This sorta makes scripts/cfpurge.js script obsolete.
2019-01-09 17:11:45 +07:00
Bobby Wibowo
b431d11eba
Updates
I think proper Cache-Control for frontend now.

Fixed the way it handles "home" when purging frontend cache.
2019-01-06 15:26:43 +07:00
Bobby Wibowo
e55a04c156
Updates
Added cfpurge.js to scripts directory.
This can be used to purge cache of frontend pages and uploads.
Do "node scripts/cfpurge.js --help" for usage.

Removed "randver" from package.js/scripts.
I've installed randomstring globally instead and just simply do:
randomstring n
Back then I didn't know it could be used that way.
2019-01-06 13:27:17 +07:00
Bobby Wibowo
765f8c9880
Better image thumbnailer
Images smaller than 200x200 will no longer be resized upwards, instead
they will be padded with transparent pixels.
This was the old behavior before we switched from GM to sharp.

With GM, its resize() function would do exactly that, but I couldn't
figure out how to do the same with only sharp's resize() function, so I
had to make do with a combination of resize() and extend().

Also updated error messages in dashboard when trying to load
out-of-index page in uploads/users lists.

Updated v1 version string due to dashboard.js being modified.
2019-01-03 08:54:46 +07:00
Bobby Wibowo
d723c0f562
Updates [!! update config.js !!]
Added extended support for URL uploads.
Namely URL proxy support and separate extensions filter (as in separate
from the primary extensions filter).
There's also a new option to set a disclaimer message that will be
printed underneath the URL uploads form.

Trust proxy is now toggleable from the configuration file.
I think they should only be enabled when you're behind proxy such as
Cloudflare or Incapsula.
I'm not sure how it behaves with only a bare nginx reverse proxy though.

Empty files can now be filtered.

Sorted preset extensions filter in config.sample.js.

Rephrased some options in config.sample.js as well.

maxTries now default to 3 in config.sample.js.

Various other small changes.
2018-12-20 18:53:37 +07:00
Bobby Wibowo
f3d1aa1c8c
Updates
Some small fixes

Removed GET route of /upload/delete.
I first wanted to implement a ShareX-compatible deletion URL,
but then I figured I'd need to setup delete token system,
and I was like, "screw that, I don't even use ShareX",
so yeah.
2018-12-19 01:14:24 +07:00
Bobby Wibowo
52d336cc45
Updated ESLint rule: curly
No more enforced curly for if/else/for/while/do blocks w/ one statement.

With that said, auto-fixed all JS files to follow the rule.

I'd also like to apologize for the inconveniences this commit cause,
after all it was me who intentionally enforced curly rule back then.

Why the change of heart?
After doing some more non-JS codes recently, I realized it was
pretty stupid of me to enforce that.
2018-12-19 00:01:28 +07:00
Bobby Wibowo
da86f605c6
Updates [!! run database/migration.js !!]
Added description column into albums.
So yeah, now albums can have description.
It'll only be shown in the album's edit popup and public link.

HTML chars will now be escaped from album's name and description.

Removed message warning about CDN cache from album's public link.
A shortened version will be shown as the download button's tooltip.

Darkened color of textarea's placeholder.

Bumped v1 version string.
2018-12-13 16:09:46 +07:00
Bobby Wibowo
6af52341c9
Init ids-cache branch 2018-12-03 16:18:52 +07:00
Bobby Wibowo
b6bd833110
Replace graphicsmagick with sharp 2018-12-03 14:20:13 +07:00
Bobby Wibowo
6543a87b11
Updates
Reworked unique name generator to prevent the same unique identifier from being used if it was already used with a different extension (e.i. If a file named aBcD.jpg already exists, then files such as aBcD.png or aBcD.txt may not exist).
This is mainly to deal with the fact that thumbnails are only being saved as PNG, so if the same unique name is being used by multiple image/video extensions, then only one of them will have the proper thumbnail.
If you already have existing files with matching unique name but varying extensions, unfortunately you can only deal with them manually for now (either allocating new unique names or deleting them altogether).

Added a new config option to filter files with no extension.

Files with no extensions will no longer have their original name appended to the allocated random name (e.i. A file named "textfile" used to become something like "aBcDtextfile", where "aBcD" was the allocated random name. Now it will only just become "aBcD").
In relation to that, utils.extname() function will now always return blank string if the file name does not seem to have any extension.
Though files such as '.DS_Store' (basically anything that starts with a dot) will still be accepted.
Examples:
.hiddenfile => .hiddenfile
.hiddenfile.sh => .sh
.hiddenfile.001 => .hiddenfile.001
.hiddenfile.sh.001 => .sh.001

Simplified error messages of /api/upload/finishchunks.

Most, if not all, of the error responses for /api/upload* will now have HTTP status code 400 (bad request) instead of 200 (ok).
I plan to generalize this for the other API routes in the future.

Updated home.js to properly handle formatted error message when the response's status code is not 200 (ok).

Bumped v1 version string (due to home.js).
2018-11-29 00:52:12 +07:00
Bobby Wibowo
4862512471
Removed stray console.log() 2018-10-13 18:09:54 +07:00
Bobby Wibowo
fcf4c00de7
Moved permission-related functions to permissionController
Fix: non-root staffs are now able to delete files by any users (previously they could only list them).
2018-10-13 18:06:58 +07:00
Bobby Wibowo
c3d4c237cb
Init account-manager branch 2018-10-10 02:52:41 +07:00
Bobby Wibowo
5bd343638a
Updates
Moved utils.getPrettyBytes() and utils.getPrettySize() to client's dashboard.js.

Thus, server will no longer return prettified size and date (it'll be prettified by the client instead).

To be honest, I don't even know why I had them in server-side, it's obviously better this way.
2018-10-09 01:54:16 +07:00
Bobby Wibowo
30c011ce5c
Replaced snekfetch with node-fetch 2018-09-23 23:28:15 +07:00
Bobby Wibowo
65442c18c5
Reworked extension parsing
Removed "path-complete-extname" module in favor of an in-house solution, utilsController.extname().

For now the function will attempt to preserve multi-archive extensions (.001, .002, and so on), as well as some known tarballs (.tar.gz and the likes).

The function will always return lower case extension. It should be fine, but do keep it in mind.
2018-09-18 02:32:27 +07:00
Bobby Wibowo
7587bfa408
Updates
* Updated utilsController.js: Unlink thumb of type symlink whenever generateThumbs() is called.

* Updated thumbs.js: Add stats (success/error/skipped).
2018-09-05 00:29:53 +07:00
Bobby Wibowo
36da76357e
Updates
* Added an experimental virus scanning feature using ClamAV. This has only been tested with an Ubuntu machine.

* File extensions will now be parsed with path-complete-extname module. This will ensure extensions such as .tar.gz are properly parsed.

Notice: It may take a minute or so to start the safe with virus scanning, as apparently the module takes a while to create the engine. I'm guessing since it'll be loaded to memory? Either way, once the engine is created, everything should work fine. Virus scanning should also not have that much of an impact to the upload time.
2018-09-02 03:37:26 +07:00
Bobby Wibowo
7f23734d67
Updates
* Faster upload response. Back-end will no longer wait for album timestamps to be updated before sending out response.

* Added a simple thumbnail generation script at scripts/thumbs.js. You can use this to generate thumbnails for existing files before enabling the option in config.js.

* Various other code improvements.
2018-05-12 21:01:14 +07:00
Bobby Wibowo
bd722129de
Updates, now supports uploading by URLs!
* Added upload by URLs. It has its own max size (config.uploads.urlMaxSize), make sure your config matches config.sample.js.
Here's a brief video showing it in action: https://i.fiery.me/CUhQ.mp4.

* /api/upload now supports uploading by URLs. Devs will only need to POST a JSON request containing a key named "urls", which is an array of the urls to upload.

* Added file extension filter to /api/upload/finishchunks.

* Added proper total chunks size check to /api/upload/finishchunks.

* Various code improvements.
2018-05-11 21:34:13 +07:00
Bobby Wibowo
479db54cd3
Updates (experimental)
* Possible performance improvement. Some bulk db queries will now be executed in a single query instead of spawning multiple async task for each query. This is sorta experimental though, use it at your own risk (though I'll use it right away at safe.fiery.me).

* It's now possible for root user to add files to other users' albums through the API route. I don't plan on allowing root user to list other users' album list from the dashboard, I just thought that there'd be no harm in extending the API a little bit.

* Kinda better error logging for uncaught exception and unhandled rejection. Their stack trace should be logged now.
2018-05-11 00:25:52 +07:00
Bobby Wibowo
c952e3e8ce
Updated utilsController.js
Possibly better Cloudflare error handling.
2018-05-09 17:18:16 +07:00
Bobby Wibowo
fbd38a8bdd
Updated utilsController.js
Forgot to symlink 'unavailable thumb' placeholder for failed video thumbs.
2018-05-09 17:07:23 +07:00
Bobby Wibowo
f043f65ca1
Updates
* Added Cloudflare purge cache support. Check configuration sample at config.sample.js.
When it's enabled, whenever files are being deleted, it will send a POST request to Cloudflare's API to purge cache of the deleted files.
This adds a new dependency called "snekfetch". It's lightweight though.

* uploadsController.delete() will now wrap uploadsController.bulkDelete() instead.
2018-05-09 16:53:27 +07:00
Bobby Wibowo
7991a63315
Updates (please update your config.js)
NOTICE: Please update your config.js. Use config.sample.js as the template.
There were a couple of renames and restructures.

* Album zipper API route will now internally save its state when it's generating zip files, and any subsequent requests will silently be "postponed" until the first spawned task is finished. This will guarantee that there are no multiple zipping tasks for the same album. The method may seem a bit hackish though.

* All instances of console.log(error) were replaced with console.error(error). This will guarantee that any error goes to stderr instead of stdout.

* Deleting file by names will now properly remove successful files from the textarea. There was a logic flaw.

* Failure to generate thumbnails will no longer print the full stack, but instead only the error message. It will also then symlink a template image from /public/images/unavailable.png (it's only a simple image that says that it failed to generate thumbnail).
This haven't been tested in Windows machines, but it'll probably work fine.
I thought of adding a new column to files table which will store information whether the thumbnail generation is sucessful or not, but oh well, I'll go with this method for now.
2018-05-09 15:41:30 +07:00
Bobby Wibowo
08410faa9a
Updates (breaking changes!)
* Updated API route: /upload/bulkdelete.
It now accepts an additional property named "field". In it you can now enter either "id" or "name", which will set whether it will bulk delete by ids or names respectively. It also no longer accepts property named "ids", instead it has to be named "values" (which of course is an array of either ids or names). So yeah, now the API route can be used to bulk delete by ids and names.
In the future this will be expanded to bulk deleting files by username (only accessible by root of course).

* Added a form to bulk delete files by names for the hardcore user, like me (https://i.fiery.me/AHph.png).

* Some design update. Mainly forms restructuring aimed at tight screens.

* Changing file name length, requesting new token and setting new password will no longer reload the dashboard page on success. Instead it will simply silently reload the form.

* utils.bulkDeleteFilesByIds() replaced by utils.bulkDeleteFiles() which now can either by ids or names. This will be the one that will eventually be extended for deleting by username.

* Various other code improvements.
2018-05-06 02:44:58 +07:00
Bobby Wibowo
61e1896945
Updates
* A bunch of refactors in public JS files (home.js, dashboard.js, etcetera).

* Added lazyload to home page (for thumbs of uploaded images), dashboard (for thumbs view) and albums' public link.
Albums' public link will silently fallback to loading all thumbs at once if JavaScript is disabled.

* A bunch of others code improvements. Honestly I'm too lazy to track all the changes.
2018-04-29 19:47:24 +07:00
Bobby Wibowo
674d20c62c
Updates
Regular code improvements.
2018-04-21 04:39:06 +07:00
Bobby Wibowo
dd43acecea
Updates
* Added VSCode settings to git repo. Now you can match yours with mine, if you want.

* Added .jsbeautifyrc for js-beautify (to be used by VSCode's Beautify extension).

* Refactored all instances of require('**/*.js') with require('**/*') wherever applicable (basically gotten rid of the .js extension).

* Refactored path in all instances of require() wherever applicable.

* Sorted instances of require() wherever applicable.

* Fixed 500 HTTP error trying to load an error page for 505 HTTP error.

* Removed special treatement of NoJS page from uploadsController.processFilesForDisplay().

* Updated version string of all static files.

* Beautified all HTML, HANDLEBARS and CSS files.

* Refactored the structure of footer links in homepage and No-JS uploader. This should now fix homepage going out-of-bound in smaller screens.

* Added CSS prefixes wherever applicable.

* Improved back-end side of No-JS uploader. This will now handle errors properly.

* No-JS uploader will now show max file size.

* No-JS uploader will now show a proper message when private mode is enabled and/or registration is disabled.
2018-04-13 23:20:57 +07:00
Bobby Wibowo
3ca692d8c7
Updates
* Refactored all instances of "failedIds" and "albumIds" to "failedids" and "albumids" respectively. Abandoning camel case for these ones.

* Refactored the way it looks into which albums the files are supposed to be added into.
For /api/upload/finishchunks, you can add "albumid" to each object in files[] to specify which album you want the finsihed chunks to be added into. Each object may have different album IDs.
For /api/upload, which is regular uploads, unfortunately you can only choose one album at a time (by adding "albumid" to the request headers, like usual). It uses the same function as the one used for finishchunks to add the files into album, so it shouldn't be hard to extend this ability to regular uploads, someday in the future.

* Fixed a bug in /api/upload/finishchunks. Previously you couldn't ever get it to work.

* Updated error message when successful uploads could not be added to album.

* "albumid" will no longer be added to request headers if they are chunked uploads. They'd have been ignored anyways.
2018-04-05 19:54:24 +07:00
Bobby Wibowo
b1dbb931c1
Updates
* Updated eslint-plugin-import dev dependency.

* Added 2 new ESLint rules: "prefer-const" and "object-shorthand".

* Refactor all JS files to follow the new ESLint rules.

* Refactored all instances of for-i into for-of wherever applicable.
2018-04-05 17:52:57 +07:00
Bobby Wibowo
7f10cccf70
Updates
* If files have to be added to an album on upload, it will now wait until they have either been succesfully added to the album or errored.

* File names in thumb view will now show their full URL on their tooltip.
2018-04-05 00:38:15 +07:00
Bobby Wibowo
a61060249f
Updates
* Updated README.md to get rid of all the warnings from linter. Just because.

* Removed an unnecessary logic check in albumsController.addFiles().

* Refactored "successIds" to "updateDbIds" in utilsController.bulkDeleteFilesByIds().
2018-04-03 22:01:19 +07:00
Bobby Wibowo
8724d45ce0
Updates
* Refactored all instances of "An error occurred" by appending an exclamation mark.

* Added the ability to add/remove files to/from album (API route: /api/albums/addfiles - https://s.fiery.me/dCAqLEQ9.mp4).

* Added the ability to purge files associated with an album when deleting the said album (set "purge" key to true in the JSON POST request to /api/albums/delete).

* Updated icons.

* Some other refactors, probably.
2018-03-30 09:39:53 +07:00
Bobby Wibowo
0067c8fe83
Updates
* Refactored all instances of "err" into "error".

* Added bulk delete feature (API route: /api/uploads/bulkdelete). It accepts an array of IDs (its key must be "ids" in the JSON POST request). Don't forget it still requires a token in the headers. (https://s.fiery.me/6rjMAYoC.mp4)

* Removed fontello.css from auth.html.

* Updated a bunch of styling.

* Added "copy link to clipboard" button to thumbs view.

* Added "view thumbnail" button to list view. Clicking the row will no longer trigger thumb view, instead you have to press that button.

* Updated icons.

* ... and perhaps some others that I can't remember?
2018-03-30 06:22:08 +07:00
Bobby Wibowo
2dd724f88f
Updates
* Switched standard to eslint with eslint-config-standard (and 4 more eslint plugins needed by standard).

* Added "curly" eslint rule with "all" option. I like it.

* Refactored all JS files to apply the new "curly" eslint rule.

* Renewed axios.min.js, dropzone.min.js and sweetalert.min.js. Re-minified and added a small comment stating their version and copyright statement.

* Some buttons in dashboard will now show loading icon whenever they're waiting for response from the server.

* Updated README.md and .gitignore.
2018-03-29 00:40:50 +07:00
Bobby Wibowo
66a63ca6d6
Updates (YAY, CHUNKED UPLOADS!)
* Added new dependency: rimraf. This will be used by chunked upload support to bulk delete temporary chunk files.

* Added chunked uploads support :3

* Updated Dropzone to 5.2.0.

* More improvements to thumbnail view. Delete button will now only appear on hover. Some other details, such as file name, size and album/owner will also appear on hover. Touch devices will have all of those appear always visible by default.

* Image thumbnails will now appear on home page after successful uploads (only for WEBP, JPG, JPEG, BMP, GIF and PNG files). WEBP may not work properly in Firefox though.

* Refactored home.js to use const/let and some other stuff.

* Refactored album view. It will now display properly on mobile screen. Download Album button will also no longer be located at the top right, but right below the subtitle.

* Updated some version strings.

* And maybe some others that I can't remember.
2018-03-28 18:36:28 +07:00
Bobby Wibowo
3fa5b24ee5
Updates
* Better auth handling.

* Deleting a file will no longer cause the dashboard to load the very first page of uploaded files list. It will instead reload the currently viewed page.

* Updated dropzone (I guess).
2018-03-25 02:47:41 +07:00
Bobby Wibowo
83f3b36f15
Updates
* Added "Size" column to list view of uploaded files. The size will be use 'pretty' view (kB, MB, etc).

* Added delete button to thumbs view of uploaded files.

* All instances of "file length" in public views renamed to "file name length". The latter makes more sense.
2018-03-24 23:45:51 +07:00
Bobby Wibowo
616124446f
Updates (WARNING!)
WARNING: Please turn off lolisafe before upgrading, then run "node database/migration.js" once after upgrading. Ignore all errors/warnings about duplicate column name. Afterwards make sure your config.js follows the new format in config.sample.js (specifically fileLength and generateThumbnails options).

* generateImageThumbnails and generateVideoThumbnails options in config.js is now renamed to an object named generateThumbnails, with image and video as its properties.

* fileLength option is now an object with min, max, default and userChangeable as its properties.

* User may now change their preferred file length (following the previous option, of course).

* Updated a bunch of responses messages. Mainly appending a dot to the messages.

* New APIs:
/fileLength/config to get an object of the current fileLength config (exactly what is in the config.js file).
/fileLength/change to change user's preferred file length.

* And maybe some others ...?
2018-03-24 20:52:47 +07:00
Bobby Wibowo
66f3b0739d
Updates
Experimental changes to allow generating thumbnails only for EITHER images or videos.
2018-02-07 13:22:31 +07:00
Bobby Wibowo
bcdfcd7064
Various updates
* Switched ESLint + Aqua to Standard. I'm a big fan of Standard. Updated yarn.lock file too.

* Lots of refactors to follow the rules of Standard.

* Fixed issue with uploading as a not logged in user.
2018-01-24 03:06:30 +07:00
Pitu
702075b66d ES6 rewrite 2017-10-03 21:13:38 -03:00
Pascal Temel
b05dac6743 cleanup 2017-09-24 05:54:13 +02:00
Pascal Temel
4b63ea2d1b add wmv, avi and mov thumbnail support 2017-09-24 05:35:09 +02:00
Pitu
026e0a3ef0 Derps were made, and now fixed 2017-03-18 01:36:50 -03:00
Pitu
8a96a0df1d Moved thumbnail processing to it's own utils file 2017-03-17 01:14:10 -03:00