Bobby Wibowo
4907ef9ad7
chore: indent albumsController.js
2022-08-09 17:00:26 +07:00
Bobby Wibowo
edf7c091e4
feat: wrap upload db insert within transaction
2022-08-09 16:57:55 +07:00
Bobby Wibowo
d8b78d29ed
feat: hard-code prevent registering as "root"
...
and allow migration script to not throw when root user is missing
this facilitates safely removing root user altogether via database query
if you don't use it
2022-08-08 06:22:18 +07:00
Bobby Wibowo
253042e24e
fix: improve filtering uploads by album ids
...
database logic and dashboard display
2022-08-08 06:08:40 +07:00
Bobby Wibowo
b21fa66e59
fix: possible event listeners memory leak
2022-08-05 00:54:44 +07:00
Bobby Wibowo
c6c485447f
feat: token failure rate limit on login/register
...
also removed default 2 reqs in 5s rate limiter for login/register routes
from sample config, as it's pretty much redundant now
2022-08-04 23:34:58 +07:00
Bobby Wibowo
a406f85215
feat: rate limit token auth failures
...
hard-coded to max 6 failures in 10 minutes
2022-08-04 23:09:14 +07:00
Bobby Wibowo
3e0aa1361d
fix: detect uploads timed out by uwebsockets
2022-08-04 22:08:40 +07:00
Bobby Wibowo
7381cac0e9
chore: indents and comments
2022-08-04 21:59:50 +07:00
Bobby Wibowo
d7d6a29123
feat: cleaned up routes init
...
asserting auth and JSON body will now be done via route-specific
mini middlewares (authController's requireUser or optionalUser)
2022-08-04 21:59:06 +07:00
Bobby Wibowo
2351528a42
fix: redundant logic
2022-08-03 17:31:49 +07:00
Bobby Wibowo
46c8867223
fix: internally prepend chunksData UUID with IP
...
even less chance for a collision to occur
2022-08-03 17:28:42 +07:00
Bobby Wibowo
0ebefe083a
refactor: removed clamscan passthrough support
...
unfortunately it simply was not reliable enough
and maintaining it is simply adding more complexity to the codes
moreover it was only possible to passthrough regular non-chunked uploads
2022-08-02 16:19:57 +07:00
Bobby Wibowo
164cadd8b9
feat: increased regular users' max sort keys to 2
...
possible use case, sorting by albumid, then size
moderators and above still have no limits
2022-08-01 15:21:23 +07:00
Bobby Wibowo
ac38b6f06e
feat: if sort uploads by album id, sort null last
...
also improved indenting on some lines of codes
2022-08-01 15:20:14 +07:00
Bobby Wibowo
323c107f64
fix: ServeStatic
...
init setContentDisposition and setContentType functions immediately as
private functions to reduce complexity
so instead check for the required map/store before using them
also fixed content-type override ending up with duplicate headers
2022-08-01 07:29:49 +07:00
Bobby Wibowo
21ec4a7479
fix: 416 status code handling
2022-07-31 16:46:35 +07:00
Bobby Wibowo
0598a63989
refactor: serve handlers/middlewares
...
moved shared codes into serveUtils to reduce complexity
2022-07-31 16:34:06 +07:00
Bobby Wibowo
527498bb1e
perf: list albums db query
2022-07-31 15:55:27 +07:00
Bobby Wibowo
285e79c5a7
feat: configurable uploads/albums/users per page
...
please check sample.config.js for new options
if missing from config, defaults to 25 per page (old defaults)
2022-07-31 15:51:32 +07:00
Bobby Wibowo
2389974c7d
feat: ServeStaticQuick
...
chokidar is now a production dependency
please read the comments in ServeStaticQuick.js for a description of
what the class does
public and dist directories are now served with that class by default
before starting hyper-express on the listen port, await for all
ServeLiveDirectory and ServeStaticQuick instances
2022-07-31 14:31:25 +07:00
Bobby Wibowo
d40d1e396f
fix: ServeStatic with zero bytes files
2022-07-31 14:17:06 +07:00
Bobby Wibowo
b1566c5abf
refactor: ServeLiveDirectory
...
ensure forward slashes path
refactored init method
ensure internal res.type is set before attempting to call external
setHeaders function, to allow overrides
2022-07-31 14:08:13 +07:00
Bobby Wibowo
b9badcc944
fix: ServeStatic ensure forward slashes path
2022-07-31 14:06:17 +07:00
Bobby
4591b8bb42
refactor: generateUniqueToken -> getUniqueToken
...
this now matches lifecycle with similar functions in upload and album
controllers
also added a new util function .mask() for basic string masking
2022-07-30 08:37:57 +07:00
Bobby
b7dcf30578
feat: console logs for identifiers on debug only
2022-07-30 08:35:26 +07:00
Bobby
548af312a7
chore: authController.js
2022-07-30 08:02:17 +07:00
Bobby
c23bc90412
refactor: album random identifier generation
2022-07-30 08:01:19 +07:00
Bobby Wibowo
8782a004d6
chore: uploadController.js
2022-07-29 10:17:17 +07:00
Bobby Wibowo
681a3ca32f
fix: ServeStatic content-length transfer-encoding
...
both headers cannot co-exist at the same time, so we pass the expected
content-length value into 2nd param of Response.stream(), so that the
internal can decide to add it only when required
2022-07-29 10:16:49 +07:00
Bobby Wibowo
8748dcefb0
feat: parse content-disposition on url uploads
2022-07-29 10:15:11 +07:00
Bobby Wibowo
fae28f9aa2
feat: deprecate uploads.cacheFileIdentifiers conf
...
maintaining it is an unnecessary complexity
it's a feature that doesn't scale too well anyways
also renamed "queryDbForFileCollisions" to
"queryDatabaseForIdentifierMatch"
and updated config description accordingly
this should also now properly free the internal onHold Set
2022-07-29 09:14:55 +07:00
Bobby Wibowo
03eff45e8c
refactor: uploadController.js
...
some logic improvements
2022-07-28 13:26:15 +07:00
Bobby
301cf3377d
fix: set upload name utf8 encoding via busboy conf
...
instead of converting from the default latin1 using Buffer
changing busboy config was not possible with express + multer,
so i did not notice it's instead possible with hyper-express
2022-07-28 10:19:28 +07:00
Bobby Wibowo
b9a1604440
refactor: rename handler/middleware class files
2022-07-25 07:48:31 +07:00
Bobby Wibowo
235a1c56e1
refactor: move devmode flag to utils
2022-07-25 07:39:35 +07:00
Bobby Wibowo
ec4d54573a
fix: errors thrown when url uploads get rejected
2022-07-25 07:32:39 +07:00
Bobby Wibowo
5bab3a495e
feat: allow to disable file hashing completely
2022-07-25 07:32:25 +07:00
Bobby Wibowo
6ba30a23c6
feat: improved chunked uploads lifecycle
...
added checks when there's an attempt to uploads chunks to same file in
parallel
improved final file size checks
2022-07-25 07:09:28 +07:00
Bobby Wibowo
ee8f1914ca
feat: req content-type for upload api
2022-07-25 06:13:12 +07:00
Bobby Wibowo
1b109e0dc0
feat: req content-type json -> application/json
2022-07-25 06:12:55 +07:00
Bobby
4ca64b141f
refactor: do not await assertRequestType
...
it's not an async function, lmao
2022-07-22 08:50:26 +07:00
Bobby
951737d7d0
feat: handle upload post api based on request type
...
it'd previously always try to parse as multipart first
now it'll immediately assume the upload post api is for url uploads if
the request type is json
2022-07-22 08:42:11 +07:00
Bobby
776ab8ab37
feat: assert request content-type in post apis
2022-07-22 08:40:40 +07:00
Bobby Wibowo
aa85d04d34
fix: url uploads failing
2022-07-22 04:20:37 +07:00
Bobby Wibowo
96d276b396
fix: check if req.path_parameters is set
2022-07-22 02:03:59 +07:00
Bobby Wibowo
07d0237031
refactor: res.query -> .query_parameters
...
direct hyper-express prop get
2022-07-22 02:02:59 +07:00
Bobby Wibowo
25f87b3a49
chore: albumsController.js
2022-07-22 01:44:53 +07:00
Bobby Wibowo
7b9fca0bc3
refactor: req.params -> .path_parameters
...
direct hyper-express prop get
2022-07-22 01:44:15 +07:00
Bobby Wibowo
76a73b7e83
refactor: ServeStatic.middleware -> .handler
...
also moved it from middlewares to handlers directory
reasoning is that this class is better suited to handle routes directly
instead of being a global middleware
since IO stat to check if request path matches a physical file in the
disk every single time is not very performant
2022-07-22 01:12:35 +07:00
Bobby Wibowo
30e9227a78
feat: custom pages use ServeLiveDirectory
...
they now have conditional GET suppor too
2022-07-22 01:09:17 +07:00
Bobby Wibowo
51e12e13c0
refactor: rateLimiter.js
2022-07-22 00:57:57 +07:00
Bobby Wibowo
d6020d81ae
feat: serveStatic with accept-ranges support
...
for streaming support
and with conditional GET support
2022-07-22 00:01:25 +07:00
Bobby Wibowo
d9fd98f7de
feat: improved serveLiveDirectory
...
allow disabling etag and lastModified headers if required
2022-07-21 23:56:57 +07:00
Bobby Wibowo
1b4b73b67c
feat: improved errorsController.js
...
mainly handling generic errors
2022-07-21 23:56:08 +07:00
Bobby Wibowo
c0e91e205c
chore: rateLimiter.js
2022-07-21 23:55:48 +07:00
Bobby Wibowo
97bd8f9e5a
chore: serveLiveDirectory.js
2022-07-21 21:14:56 +07:00
Bobby Wibowo
e7a15ecc47
feat: custom livedirectory middleware
...
with conditional gets support
2022-07-21 21:13:46 +07:00
Bobby Wibowo
ad22285661
refactor: res.set -> res.header
...
res.set() is an expressjs-compat function with unnecessary checks for
our use case
2022-07-21 20:28:10 +07:00
Bobby
e6753ab15d
fix: handle connection drop on multiform upload
2022-07-15 01:40:57 +07:00
Bobby
27f3bc3119
perf: don't wrap multipart handler in try-catch
...
fixed in https://github.com/kartikk221/hyper-express/releases/tag/6.3.0
2022-07-15 01:06:28 +07:00
Bobby Wibowo
f40c9e0287
chore: serveStatic.js
2022-07-14 18:34:25 +07:00
Bobby Wibowo
7f6c29b136
fix: properly clean rejected empty files
...
should also now properly clean temp files from other unexpected errors
2022-07-14 18:18:39 +07:00
Bobby Wibowo
5ee82ce680
perf: don't wait for unlink promise in fail upload
...
just let it run in the background and respond to client immediately
2022-07-14 18:17:46 +07:00
Bobby Wibowo
7710e63d70
fix: prevent hashStream.update() after .dispose()
2022-07-14 17:01:59 +07:00
Bobby Wibowo
29b16edc04
perf: improve uploads flow
...
lessen temporary objects/variables creation,
and refactor some variable names to be more obvious
2022-07-14 16:35:06 +07:00
Bobby Wibowo
ac63f8b76d
perf: no try-catch block on cloudflare cache purge
2022-07-14 14:41:55 +07:00
Bobby Wibowo
c32f18a697
fix: uploads mimetype not properly set
2022-07-12 15:26:53 +07:00
Bobby Wibowo
0f6409132a
feat: busboy limits and multipart errors handler
...
unfortunately to capture multipart errors, we have to wrap the entire
callback function with try-catch block
but it appears overall processing is still slightly faster than
expressjs + multer
2022-07-12 14:39:16 +07:00
Bobby Wibowo
7f9d05da26
feat: multer -> hyper-express multipartfield
...
get outta here multer, lmao
2022-07-12 13:07:13 +07:00
Bobby Wibowo
e9736f436c
feat: have res.render return the compiled html
2022-07-12 10:41:58 +07:00
Bobby Wibowo
80d59ff2f0
feat: front-end pages middleware
...
custom pages may now override any built-in pages on the fly as lolisafe
is running
also added internal persistent cache feature into NunjucksRenderer
front-end pages will now be persistently cached during production
2022-07-12 10:31:59 +07:00
Bobby Wibowo
9852dec3d1
chore: RateLimiter clean ups
2022-07-12 10:30:36 +07:00
Bobby Wibowo
a356ce5ad3
refactor: standardize paths in require()'s
2022-07-12 08:51:22 +07:00
Bobby Wibowo
79631ce624
feat: RateLimiter custom middleware class
...
this adds new production dependency rate-limiter-flexible
this deprecates old rateLimits option in config
to use the new rate limiters, the new option is named rateLimiters and
rateLimitersWhitelist
please consult config.sample.js
rate limiters will also be now processed before any other middlewares,
as only makes sense
2022-07-12 08:48:09 +07:00
Bobby Wibowo
26ae853362
fix: errorsController
...
not properly printing errors when headers already sent
2022-07-12 08:05:45 +07:00
Bobby Wibowo
eb4057d10c
chore: don't call next on async middleware
...
you're not supposed to do that, lmao
2022-07-12 08:04:50 +07:00
Bobby Wibowo
b0913eaf59
refactor: ServeStatic custom middleware (WIP)
...
currently when enabled will force-close lolisafe
i still need to find a decent backend library to make life easier
2022-07-12 06:29:21 +07:00
Bobby Wibowo
af754d7d71
feat: NunjucksRenderer custom middleware class
2022-07-12 06:24:04 +07:00
Bobby Wibowo
7733967624
refactor: errorsController func names
2022-07-12 06:21:21 +07:00
Bobby Wibowo
38d86779ae
refactor: HUGE REFACTOR for hyper-express
2022-07-10 19:46:25 +07:00
Bobby Wibowo
b89945d693
chore: remove controllers/handlers
2022-07-10 19:43:46 +07:00
Bobby Wibowo
59c023588e
refactor: authController pass errors
2022-07-10 14:24:18 +07:00
Bobby Wibowo
b3a304729f
refactor: uploadController pass errors
2022-07-10 14:20:49 +07:00
Bobby Wibowo
0f47ed76b0
feat: tokenController pass errors
2022-07-10 14:17:59 +07:00
Bobby Wibowo
3a415165b4
feat: utilsController pass errors
2022-07-10 14:15:14 +07:00
Bobby Wibowo
99a7a2a677
refactor: albumsController pass errors
...
motivation: less try-catch
2022-07-10 14:11:29 +07:00
Bobby Wibowo
d970f1d0a5
feat: errorsController
...
errors should instead be passed along to express via next(), to let
express call the errorsController
2022-07-10 14:10:59 +07:00
Bobby
2ca2fef301
feat: use SimpleDataStore for album pages cache
...
this should have better lifecycle and use less memory over time, since
we can define max items in cache
at the moment hard-coded to 10 cached pages (inclusive of nojs version
if ever generated)
2022-07-06 17:51:34 +07:00
Bobby
b9d0f787d7
feat: improved SimpleDataStore.hold() again
...
this time uses an internal Set()
this should hopefully be the final iteration of this helper class until
a new "strategy" needs to be added
2022-07-06 17:49:13 +07:00
Bobby
17c863f724
feat: SimpleDataStore.hold() no longer use up size
...
this means internal Map-store may actually contain more keys than limit,
but only if the extraenous keys are held-keys (i.e. has null value)
this expects that you don't manually set keys with null values into the
store however
2022-07-06 17:14:00 +07:00
Bobby
ecb30cd159
feat: SimpleDataStore allow pre-holding key
2022-07-06 16:37:54 +07:00
Bobby Wibowo
d31181b4be
refactor: SimpleDataStore.STRATEGIES
2022-07-03 11:08:00 +07:00
Bobby Wibowo
39aaa25f3f
perf: SimpleDataStore
2022-07-03 11:00:48 +07:00
Bobby Wibowo
8a1ff434d9
feat: in-memory caching of content-disposition
...
please read config.sample.js
ignore if not serving files with node or not having the option turned on
2022-07-03 10:35:36 +07:00
Bobby Wibowo
1404cf9328
fix: utf-8 filename breaks
...
https://github.com/expressjs/multer/issues/1104
2022-07-03 10:18:04 +07:00
Bobby Wibowo
ab96bd5d99
feat: queue cloudflare purge cache with fastq
2022-06-29 17:52:16 +07:00
Bobby Wibowo
38e673226f
fix: inconsistent size field in DB !! yarn migrate
...
we used to store number directly into the string size field, and
during the conversion it seemed to always add ".0" at the final string,
probably because the driver or sqlite3 itself assumes float
please run yarn migrate after pulling this commit
if you skip converting the DB, file duplicates check will fail to
function
and in the future im planning to do size statistics in bigint, which
will also fail if not converted
2022-06-29 14:35:00 +07:00
Bobby Wibowo
9d38c431dc
fix: statistics failing to respond on errors
2022-06-29 14:29:47 +07:00
Bobby Wibowo
b117fa9ad1
feat: expose utils to nunjucks templates
...
also fix nunjucks templates not adhering to env vars overrides for
domains config options, because they attempted to read config directly
2022-06-29 13:58:09 +07:00
