both headers cannot co-exist at the same time, so we pass the expected
content-length value into 2nd param of Response.stream(), so that the
internal can decide to add it only when required
maintaining it is an unnecessary complexity
it's a feature that doesn't scale too well anyways
also renamed "queryDbForFileCollisions" to
"queryDatabaseForIdentifierMatch"
and updated config description accordingly
this should also now properly free the internal onHold Set
instead of converting from the default latin1 using Buffer
changing busboy config was not possible with express + multer,
so i did not notice it's instead possible with hyper-express
also moved it from middlewares to handlers directory
reasoning is that this class is better suited to handle routes directly
instead of being a global middleware
since IO stat to check if request path matches a physical file in the
disk every single time is not very performant
unfortunately to capture multipart errors, we have to wrap the entire
callback function with try-catch block
but it appears overall processing is still slightly faster than
expressjs + multer
custom pages may now override any built-in pages on the fly as lolisafe
is running
also added internal persistent cache feature into NunjucksRenderer
front-end pages will now be persistently cached during production
this adds new production dependency rate-limiter-flexible
this deprecates old rateLimits option in config
to use the new rate limiters, the new option is named rateLimiters and
rateLimitersWhitelist
please consult config.sample.js
rate limiters will also be now processed before any other middlewares,
as only makes sense
this should have better lifecycle and use less memory over time, since
we can define max items in cache
at the moment hard-coded to 10 cached pages (inclusive of nojs version
if ever generated)
this means internal Map-store may actually contain more keys than limit,
but only if the extraenous keys are held-keys (i.e. has null value)
this expects that you don't manually set keys with null values into the
store however
we used to store number directly into the string size field, and
during the conversion it seemed to always add ".0" at the final string,
probably because the driver or sqlite3 itself assumes float
please run yarn migrate after pulling this commit
if you skip converting the DB, file duplicates check will fail to
function
and in the future im planning to do size statistics in bigint, which
will also fail if not converted
this will display all information recorded from the specified file, but
only to the users that own them (it requires token)
this page also has a delete file button, allowing us to provide link to
this page for sharex deletion url option
once again, this is only for authenticated users, and will only show
file that the users own, unless said user is a moderator or higher
db is now initiated via utilsController, and gets re-used by main
lolisafe.js script and the other controllers
this should now make sure we only have 1 active db connection pool ever
this is a pretty big refactor due to how we used to reference db,
but it should functionally be identical
every now and then i got curious of what people add to their forks,
and sometimes i'd be like, "hey, that makes sense!"
i remember that back in the days, i switched to nunjucks because of
someone's fork too
we used to use handlebars, which i've had some misgivings with for
awhile back then
either way all i'm saying is that it's the case this time around with
this commit
thanks for reading
49d12004e8
this should properly dispose of unfinished write & hasher streams on
any errors if they have been initiated
also do content-length header check a bit more early
first layer is via sending HEAD request to the url to determine its size
via content-length header
however not all hosts properly set the header, so we ignore it if
it isn't a valid number
next via size option in fetch(), which supposedly limits response body
size during the request itself (?)
lastly via checking actual bytes written to physical file as reported by
fs.createWriteStream()
this supersedes the old temporaryUploadAges, while maintaining full
backwards-compatibility.
please consult config.sample.js if you want to start using this
this required expanding our custom error classes with support for
arbitrary internal api error codes
however it'll only be used for invalid token errors for now (10001)
no plan to assign codes to other existing api errors
at that point it's probably better to redo the whole api infrastructure
change logic to list physical files instead, since the zipGeneratedAt
attribute may still exist despite site owners having already done
physical clean-ups
only usergroup and file extension bypass
real file size can't be determined before passthrough scan,
so there's no bypass by max file size
please read the comments in sample config file
refactored utils.clamscan into utils.scan
GIFs are known to not work without custom globally-installed libvips
with ImageMagick or GraphicsMagick support.
https://sharp.pixelplumbing.com/api-output#gifhttps://sharp.pixelplumbing.com/install#custom-libvips
It's highly recommended to update your config following the changes to
the sample config file.
This also addressed a bug where images would still get recorded to DB
despite them not existing physically due to strip tags errors.
Made the codes for stats generation a bit more readable.
Usage percentage for file systems will now properly reflect "non-root"
usage percentage in ext2/3/4 file systems.
Also Object.freeze() permissions object in permissionController.
I don't think it's much to be worried about, as no "set" will be done to
it during the service's operation, but oh well, might as well.
ClientError will default to 400 HTTP error code.
ServerError will default to 500 HTTP error code.
Following the previous commit, these for now are only being used in
albumsController. More will soon follow.
Additionally fixed existing album names can sometimes be re-used when
editing an album.
This will be used for errors that are to be delivered to users, AND not
to be logged into the server (as in it stacktraces and all).
This will eventually remove the need to throw string literals.
In this commit, this has only been implemented on albumsController.js,
but more will soon to come.
well, that was a pointless endeavor.
i wasn't thinking clearly.
also updated all scripts that previously used them, to use a combo of
for-loop and setInterval to print progress.
it will re-map body of /api/album/get/:id into upstream-compatible body.
prep for lolisafe albums support for magane plugin.
/api/album/:id/:page will stil respond with the old format as that's
what the dashboard use and expect.
list views of uploads, users and albums in dashboard will now show
total items count on the table's top right corner.
they'll also be collapsed on initial page load
this uses a new client-side dependency, bulma-collapsible
https://github.com/creativebulma/bulma-collapsible
/api/albums to fetch albums list now support simple reply, where only
their ids and names will be returned.
this simple reply will also return all of the user's albums, instead of
being limited to only 9 or 25 entires like before.
fixed add to album in dashboard, and album selector in homepage uploader
being limited to only 25 albums.
with preset delay in between (60 secs if rate limited, 5 secs for
unexpected errors)
an alternative to global queue mechanism in #342
not perfect, but easier to implement
closes#342
much more expandable, and should be easier to understand overall.
make more statistics operations run concurrently to speed them up.
make linuxDiskStats config key obsolete by using systeminformation
package to also query for any mounted file systems.
