As the title says, this commit is a massive overhaul.
I've rewritten/restrucuted almost everything in the controller scripts.
Because of that, there's a considerable possibility that I've broken
something somewhere.
Notable changes:
Added temporary uploads.
Removed file name length changer from dashboard,
in favor of an equivalent in homepage config tab.
This allows non-registered users to also set file name length.
A bunch of other undocmented stuff.
I don't know, I'm too tired to remember them all.
Upgraded dependencies.
Stop adding cache-control header to album zip files unless
config.cacheControl is enabled.
Updated CSS files.
Moved thumbnail-related styling to thumbs.css.
Various other fixes & improvements.
Moved render.js from public/js to public/js/s.
Removed sharex.js in favor of public/js/s/utils.js.
Moved getPrettyDate() & getPrettyBytes() to that JS file as well.
Added lsKeys global variable wherever applicable.
Previously the idea was only used in dashboard.js.
Added No-JS version of album public pages.
You'll only have to add ?nojs to the URL.
Viewing the regular version with JS disabled will show a notice with
a link to the No-JS version of the particular album.
Overall page size of the regular version will now be lower as well,
since there'll be no need to add noscript tag for each thumbs.
No longer show Administrator section to non-admin in the dashboard.
Moderators will ONLY be able to see manage users menu as well.
Simplified FAQ wherever applicable.
Added a new FAQ about bug report or feature request.
Updated link for Firefox extension.
Also pushed Firefox link before Chrome, cause I like it more.
Added browser settings menu to dashboard.
This allows you to choose file size unit (kilobyte vs kibibyte) for that
specific browser.
The preference will be used on homepage, dashboard and album pages.
This also allows you to set chunk size and maximum parallel uploads
for the homepage uploader.
All menu links in the dashboard will now scroll to the content once
loaded.
Previously it would only do so with manage uploads/users when
switching pages.
Refactored all instances of for-in & for-of loop from browser JS files.
For the sake of uniformity, for now.
Added logger.js to format console logs (adding timestamps).
Re-ordered modules loading in lolisafe.js, and a few other minor edits.
Updated dev dependencies.
A few other minor edits.
* Updated path resolving for upload folder option.
This SHOULD now allow using an absolute path, even those outside of
the lolisafe installation.
* Added a config option at uploads -> generateThumbs, named placeholder.
It's a string option that lets you set path of the placeholder image
for files whose thumbnail could not be generated.
It defaults to the old hard-coded path when set to falsy value.
* Fixed thumbs script not using upload folder from config file.
* Added filters for thumb generation error handling.
This is used to ignore some common error messages,
such as file formats not being supported.
Updated utilsController.bulkDeleteFiles() to chunk opeartion by a max of
999 values (which is SQLite's default var limit).
However, there's a risk of hitting SQLITE_BUSY since we attempt to
delete chunks all at once (as can be seen by how we use Promise.all
instead of patiently waiting the chunks one by one).
However, I'm not really sure why, but the operations will still
be finished eventually, so I'll let that be for now (?).
Fixed a silly concat in albumsController.js.
No more enforced curly for if/else/for/while/do blocks w/ one statement.
With that said, auto-fixed all JS files to follow the rule.
I'd also like to apologize for the inconveniences this commit cause,
after all it was me who intentionally enforced curly rule back then.
Why the change of heart?
After doing some more non-JS codes recently, I realized it was
pretty stupid of me to enforce that.
Added description column into albums.
So yeah, now albums can have description.
It'll only be shown in the album's edit popup and public link.
HTML chars will now be escaped from album's name and description.
Removed message warning about CDN cache from album's public link.
A shortened version will be shown as the download button's tooltip.
Darkened color of textarea's placeholder.
Bumped v1 version string.
Moved utils.getPrettyBytes() and utils.getPrettySize() to client's dashboard.js.
Thus, server will no longer return prettified size and date (it'll be prettified by the client instead).
To be honest, I don't even know why I had them in server-side, it's obviously better this way.
Removed "path-complete-extname" module in favor of an in-house solution, utilsController.extname().
For now the function will attempt to preserve multi-archive extensions (.001, .002, and so on), as well as some known tarballs (.tar.gz and the likes).
The function will always return lower case extension. It should be fine, but do keep it in mind.
* Added an experimental virus scanning feature using ClamAV. This has only been tested with an Ubuntu machine.
* File extensions will now be parsed with path-complete-extname module. This will ensure extensions such as .tar.gz are properly parsed.
Notice: It may take a minute or so to start the safe with virus scanning, as apparently the module takes a while to create the engine. I'm guessing since it'll be loaded to memory? Either way, once the engine is created, everything should work fine. Virus scanning should also not have that much of an impact to the upload time.
Trying to purge empty albums (purge = also deleting all of the files associated with it) will no longer throw out warning about failing to delete any of the associated files (since there are supposed to be none for empty albums anyways).
* Faster upload response. Back-end will no longer wait for album timestamps to be updated before sending out response.
* Added a simple thumbnail generation script at scripts/thumbs.js. You can use this to generate thumbnails for existing files before enabling the option in config.js.
* Various other code improvements.
* Possible performance improvement. Some bulk db queries will now be executed in a single query instead of spawning multiple async task for each query. This is sorta experimental though, use it at your own risk (though I'll use it right away at safe.fiery.me).
* It's now possible for root user to add files to other users' albums through the API route. I don't plan on allowing root user to list other users' album list from the dashboard, I just thought that there'd be no harm in extending the API a little bit.
* Kinda better error logging for uncaught exception and unhandled rejection. Their stack trace should be logged now.
* Added Cloudflare purge cache support. Check configuration sample at config.sample.js.
When it's enabled, whenever files are being deleted, it will send a POST request to Cloudflare's API to purge cache of the deleted files.
This adds a new dependency called "snekfetch". It's lightweight though.
* uploadsController.delete() will now wrap uploadsController.bulkDelete() instead.
NOTICE: Please update your config.js. Use config.sample.js as the template.
There were a couple of renames and restructures.
* Album zipper API route will now internally save its state when it's generating zip files, and any subsequent requests will silently be "postponed" until the first spawned task is finished. This will guarantee that there are no multiple zipping tasks for the same album. The method may seem a bit hackish though.
* All instances of console.log(error) were replaced with console.error(error). This will guarantee that any error goes to stderr instead of stdout.
* Deleting file by names will now properly remove successful files from the textarea. There was a logic flaw.
* Failure to generate thumbnails will no longer print the full stack, but instead only the error message. It will also then symlink a template image from /public/images/unavailable.png (it's only a simple image that says that it failed to generate thumbnail).
This haven't been tested in Windows machines, but it'll probably work fine.
I thought of adding a new column to files table which will store information whether the thumbnail generation is sucessful or not, but oh well, I'll go with this method for now.
* Added "create new album" button at homepage uploader. Due to this, albums list will always be shown even when you don't have any albums (and of course the list will be empty).
Preview: https://i.fiery.me/ITQ5.mp4 (ignore the fact that the prompt says "Edit album", it's been fixed before this commit goes live)
* Refactored various bits of the codes to use async/await instead of Promise.then. This will obviously cause the browser's requirement to raise but hell, it's the modern browsers age.
* Various other code improvements.
* Updated API route: /upload/bulkdelete.
It now accepts an additional property named "field". In it you can now enter either "id" or "name", which will set whether it will bulk delete by ids or names respectively. It also no longer accepts property named "ids", instead it has to be named "values" (which of course is an array of either ids or names). So yeah, now the API route can be used to bulk delete by ids and names.
In the future this will be expanded to bulk deleting files by username (only accessible by root of course).
* Added a form to bulk delete files by names for the hardcore user, like me (https://i.fiery.me/AHph.png).
* Some design update. Mainly forms restructuring aimed at tight screens.
* Changing file name length, requesting new token and setting new password will no longer reload the dashboard page on success. Instead it will simply silently reload the form.
* utils.bulkDeleteFilesByIds() replaced by utils.bulkDeleteFiles() which now can either by ids or names. This will be the one that will eventually be extended for deleting by username.
* Various other code improvements.
* Bumped node requirement to >= 8.0.0 (due to async/await).
* Moved ESLint config to .eslintrc.json.
* Moved ESLint ignore to .eslintignore.
* Bumped ESLint's ecmaVersion to 8, although it was probably already automatically set to that before.
* Bugfix line 110 of albumsController.js.
* A bunch of refactors in public JS files (home.js, dashboard.js, etcetera).
* Added lazyload to home page (for thumbs of uploaded images), dashboard (for thumbs view) and albums' public link.
Albums' public link will silently fallback to loading all thumbs at once if JavaScript is disabled.
* A bunch of others code improvements. Honestly I'm too lazy to track all the changes.
Improvements related to albums:
* Changed "rename album" option with a better "edit album" feature. With it you can also disable download or public link and even request a new public link (https://i.fiery.me/fz1y.png).
This also adds a new API route: /api/albums/edit.
The old API route, /api/albums/rename, is still available but will silently be using the new API in backend.
* Deleting album will now also delete its zip archive if exists.
* Renaming albums will also rename its zip archive if exists.
* Generating zip will use async fs.readFile instead of fs.readFileSync. This should improve generating speed somewhat.
* The codes that tries to generate random identifier for album will now check whether an album with the same identifier already exists. It will also rely on "uploads.maxTries" config option to limit how many times it will try to re-generate a new random identifier.
* Added a new config option "uploads.albumIdentifierLength" which sets the length of the randomly generated identifier.
* Added "download" and "public" columns to "albums" table in database/db.js.
Existing users can run "node database/migration.js" to add the columns.
Others:
* uploadsController.getUniqueRandomName will no longer accept 3 paramters (previously it would accept a callback in the third parameter). It will now instead return a Promise.
* Album name of disabled/deleted albums will no longer be shown in uploads list.
* Added "fileLength" column to "users" table in database/db.js.
* Renamed HTTP404.html and HTTP500.html in /pages/error to 404.html and 500.html respectively. I'm still using symlinks though.
* Added a new CSS named sweetalert.css which will be used in homepage, auth and dashboard. It will style all sweetalert modals with dark theme (matching the current color scheme used in this branch).
* Updated icons (added download icon).
* Some other improvements/tweaks here and there.
"generateZips" is no longer merely accepting boolean value, it's now an object with 2 properties: "enabled" and "maxTotalSize". The former is of course to enable/disable zip generation, while the latter is to limit the total size of all the files in an album. If the total size exceeds the set limit, it will not generate a zip file. It'll probably be useful when used alongside Cloudflare's Cache Everything (since Cloudflare will only cache files with a maximum size of 512MB).
* When version string is not specified in the URL query of /api/zip/:identifier (the API route to generate and download ZIP archive of the album), it will redirect the request with the album's editedAt timestamp as the value in the version string (only if editedAt timestamp exists). This will ensure that any bare calls to the API (without version string) will always be redirected to its latest state. This is mainly useful when being used with CDN that respects query string.
* Download button and CDN notice will no longer be visible in empty album pages.
* Matched empty album phrase as the one being used with the zipping API.
Restored CDN warning to album pages. Turns out adding Page Rule to /api/album/zip/* with Cache Level set to Cache Everything and Origin Cache Control set to On will work just fine. The version string is still necessary to ensure that clients are downloading the very latest version of the archive though.
Also last time I was using zipGeneratedAt timestamp, which was dumb, this time it is using editedAt timestamp, which was my original plan but kinda forgotten.
Thanks to Zephyrrus for the hints about nunjucks, sort of (he did not really give out any specific hints to me, I stalked his fork, lmao).
* Replaced Handlebars with Nunjucks.
* Replaced all static HTML files with their NJK-equivalent (excluding error pages).
* Renamed "albumDomain" to "homeDomain" in config.sample.js (make sure you update your config.js too).
* Updated dependencies: knex and eslint-plugin-import (dev).
* Updated vscode's settings.json (I may update this again very soon).
* Added VSCode settings to git repo. Now you can match yours with mine, if you want.
* Added .jsbeautifyrc for js-beautify (to be used by VSCode's Beautify extension).
* Refactored all instances of require('**/*.js') with require('**/*') wherever applicable (basically gotten rid of the .js extension).
* Refactored path in all instances of require() wherever applicable.
* Sorted instances of require() wherever applicable.
* Fixed 500 HTTP error trying to load an error page for 505 HTTP error.
* Removed special treatement of NoJS page from uploadsController.processFilesForDisplay().
* Updated version string of all static files.
* Beautified all HTML, HANDLEBARS and CSS files.
* Refactored the structure of footer links in homepage and No-JS uploader. This should now fix homepage going out-of-bound in smaller screens.
* Added CSS prefixes wherever applicable.
* Improved back-end side of No-JS uploader. This will now handle errors properly.
* No-JS uploader will now show max file size.
* No-JS uploader will now show a proper message when private mode is enabled and/or registration is disabled.
* Refactored all instances of "failedIds" and "albumIds" to "failedids" and "albumids" respectively. Abandoning camel case for these ones.
* Refactored the way it looks into which albums the files are supposed to be added into.
For /api/upload/finishchunks, you can add "albumid" to each object in files[] to specify which album you want the finsihed chunks to be added into. Each object may have different album IDs.
For /api/upload, which is regular uploads, unfortunately you can only choose one album at a time (by adding "albumid" to the request headers, like usual). It uses the same function as the one used for finishchunks to add the files into album, so it shouldn't be hard to extend this ability to regular uploads, someday in the future.
* Fixed a bug in /api/upload/finishchunks. Previously you couldn't ever get it to work.
* Updated error message when successful uploads could not be added to album.
* "albumid" will no longer be added to request headers if they are chunked uploads. They'd have been ignored anyways.
* Updated eslint-plugin-import dev dependency.
* Added 2 new ESLint rules: "prefer-const" and "object-shorthand".
* Refactor all JS files to follow the new ESLint rules.
* Refactored all instances of for-i into for-of wherever applicable.
* Removing files from an album will now update the state of the said album. Which means, the Download Album as zip feature will properly make a new zip instead of re-using the old one which would obviously still have the said files.
* "migration.js" will no longer try to create column if it already exists.
* Changed type of all columns that used to be DATETIME to INTEGER. Apparently SQLite would have stored them as INTEGER anyways, so you don't have to change anything else.
* Updated README.md to get rid of all the warnings from linter. Just because.
* Removed an unnecessary logic check in albumsController.addFiles().
* Refactored "successIds" to "updateDbIds" in utilsController.bulkDeleteFilesByIds().
* Refactored all instances of "An error occurred" by appending an exclamation mark.
* Added the ability to add/remove files to/from album (API route: /api/albums/addfiles - https://s.fiery.me/dCAqLEQ9.mp4).
* Added the ability to purge files associated with an album when deleting the said album (set "purge" key to true in the JSON POST request to /api/albums/delete).
* Updated icons.
* Some other refactors, probably.
* Refactored all instances of "err" into "error".
* Added bulk delete feature (API route: /api/uploads/bulkdelete). It accepts an array of IDs (its key must be "ids" in the JSON POST request). Don't forget it still requires a token in the headers. (https://s.fiery.me/6rjMAYoC.mp4)
* Removed fontello.css from auth.html.
* Updated a bunch of styling.
* Added "copy link to clipboard" button to thumbs view.
* Added "view thumbnail" button to list view. Clicking the row will no longer trigger thumb view, instead you have to press that button.
* Updated icons.
* ... and perhaps some others that I can't remember?
* Switched standard to eslint with eslint-config-standard (and 4 more eslint plugins needed by standard).
* Added "curly" eslint rule with "all" option. I like it.
* Refactored all JS files to apply the new "curly" eslint rule.
* Renewed axios.min.js, dropzone.min.js and sweetalert.min.js. Re-minified and added a small comment stating their version and copyright statement.
* Some buttons in dashboard will now show loading icon whenever they're waiting for response from the server.
* Updated README.md and .gitignore.
* Better auth handling.
* Deleting a file will no longer cause the dashboard to load the very first page of uploaded files list. It will instead reload the currently viewed page.
* Updated dropzone (I guess).
WARNING: Please turn off lolisafe before upgrading, then run "node database/migration.js" once after upgrading. Ignore all errors/warnings about duplicate column name. Afterwards make sure your config.js follows the new format in config.sample.js (specifically fileLength and generateThumbnails options).
* generateImageThumbnails and generateVideoThumbnails options in config.js is now renamed to an object named generateThumbnails, with image and video as its properties.
* fileLength option is now an object with min, max, default and userChangeable as its properties.
* User may now change their preferred file length (following the previous option, of course).
* Updated a bunch of responses messages. Mainly appending a dot to the messages.
* New APIs:
/fileLength/config to get an object of the current fileLength config (exactly what is in the config.js file).
/fileLength/change to change user's preferred file length.
* And maybe some others ...?
* It will now properly create "editedAt" and "zipGeneratedAt" columns into "albums" table, which will then be used for the "Download Album" feature. Previously, due to the lack of those columns, people could not download albums. Existing installation will have to do some manual patches, which I will describe further in the commit's comments.
* Thumbnail-less files will properly show the extensions in albums. Previously it would have two dots.
* Switched ESLint + Aqua to Standard. I'm a big fan of Standard. Updated yarn.lock file too.
* Lots of refactors to follow the rules of Standard.
* Fixed issue with uploading as a not logged in user.
