Commit Graph

283 Commits

Author SHA1 Message Date
Bobby
93dc820368
feat: fetch() with AbortController for timeout
to be used with URL uploads

please consult the comments in the source files

this adds new dependency abort-controller
2022-09-14 01:33:31 +07:00
Bobby
3df67f87be
feat: timeout url uploads to 10s
uwebsockets.js has a 10s timeout of no communication between client and
server, for it to simply drop the connection

there's no point waiting any further at that point since we can't reply
to client anymore
2022-08-24 04:37:13 +07:00
Bobby
6ad8ed7ae6
fix: only accept files upload with files[] field
was always supposed to only accept said field, but i forgot to include
the logic during migration from expressjs to hyper-express
2022-08-24 04:17:07 +07:00
Bobby Wibowo
0d05da40b0
fix: fallback deletion url to relative path
if homeDomain is not configured via config file or env var

the deletion url is a frontend page, so it cannot simply assume it's on
the same domain as uploaded files
2022-08-19 09:29:11 +07:00
Bobby Wibowo
edf7c091e4
feat: wrap upload db insert within transaction 2022-08-09 16:57:55 +07:00
Bobby Wibowo
253042e24e
fix: improve filtering uploads by album ids
database logic and dashboard display
2022-08-08 06:08:40 +07:00
Bobby Wibowo
b21fa66e59
fix: possible event listeners memory leak 2022-08-05 00:54:44 +07:00
Bobby Wibowo
3e0aa1361d
fix: detect uploads timed out by uwebsockets 2022-08-04 22:08:40 +07:00
Bobby Wibowo
7381cac0e9
chore: indents and comments 2022-08-04 21:59:50 +07:00
Bobby Wibowo
d7d6a29123
feat: cleaned up routes init
asserting auth and JSON body will now be done via route-specific
mini middlewares (authController's requireUser or optionalUser)
2022-08-04 21:59:06 +07:00
Bobby Wibowo
2351528a42
fix: redundant logic 2022-08-03 17:31:49 +07:00
Bobby Wibowo
46c8867223
fix: internally prepend chunksData UUID with IP
even less chance for a collision to occur
2022-08-03 17:28:42 +07:00
Bobby Wibowo
0ebefe083a
refactor: removed clamscan passthrough support
unfortunately it simply was not reliable enough

and maintaining it is simply adding more complexity to the codes

moreover it was only possible to passthrough regular non-chunked uploads
2022-08-02 16:19:57 +07:00
Bobby Wibowo
164cadd8b9
feat: increased regular users' max sort keys to 2
possible use case, sorting by albumid, then size

moderators and above still have no limits
2022-08-01 15:21:23 +07:00
Bobby Wibowo
ac38b6f06e
feat: if sort uploads by album id, sort null last
also improved indenting on some lines of codes
2022-08-01 15:20:14 +07:00
Bobby Wibowo
285e79c5a7
feat: configurable uploads/albums/users per page
please check sample.config.js for new options

if missing from config, defaults to 25 per page (old defaults)
2022-07-31 15:51:32 +07:00
Bobby
b7dcf30578
feat: console logs for identifiers on debug only 2022-07-30 08:35:26 +07:00
Bobby Wibowo
8782a004d6
chore: uploadController.js 2022-07-29 10:17:17 +07:00
Bobby Wibowo
8748dcefb0
feat: parse content-disposition on url uploads 2022-07-29 10:15:11 +07:00
Bobby Wibowo
fae28f9aa2
feat: deprecate uploads.cacheFileIdentifiers conf
maintaining it is an unnecessary complexity
it's a feature that doesn't scale too well anyways

also renamed "queryDbForFileCollisions" to
"queryDatabaseForIdentifierMatch"
and updated config description accordingly

this should also now properly free the internal onHold Set
2022-07-29 09:14:55 +07:00
Bobby Wibowo
03eff45e8c
refactor: uploadController.js
some logic improvements
2022-07-28 13:26:15 +07:00
Bobby
301cf3377d
fix: set upload name utf8 encoding via busboy conf
instead of converting from the default latin1 using Buffer

changing busboy config was not possible with express + multer,
so i did not notice it's instead possible with hyper-express
2022-07-28 10:19:28 +07:00
Bobby Wibowo
ec4d54573a
fix: errors thrown when url uploads get rejected 2022-07-25 07:32:39 +07:00
Bobby Wibowo
5bab3a495e
feat: allow to disable file hashing completely 2022-07-25 07:32:25 +07:00
Bobby Wibowo
6ba30a23c6
feat: improved chunked uploads lifecycle
added checks when there's an attempt to uploads chunks to same file in
parallel

improved final file size checks
2022-07-25 07:09:28 +07:00
Bobby Wibowo
ee8f1914ca
feat: req content-type for upload api 2022-07-25 06:13:12 +07:00
Bobby Wibowo
1b109e0dc0
feat: req content-type json -> application/json 2022-07-25 06:12:55 +07:00
Bobby
4ca64b141f
refactor: do not await assertRequestType
it's not an async function, lmao
2022-07-22 08:50:26 +07:00
Bobby
951737d7d0
feat: handle upload post api based on request type
it'd previously always try to parse as multipart first
now it'll immediately assume the upload post api is for url uploads if
the request type is json
2022-07-22 08:42:11 +07:00
Bobby
776ab8ab37
feat: assert request content-type in post apis 2022-07-22 08:40:40 +07:00
Bobby Wibowo
aa85d04d34
fix: url uploads failing 2022-07-22 04:20:37 +07:00
Bobby Wibowo
96d276b396
fix: check if req.path_parameters is set 2022-07-22 02:03:59 +07:00
Bobby Wibowo
7b9fca0bc3
refactor: req.params -> .path_parameters
direct hyper-express prop get
2022-07-22 01:44:15 +07:00
Bobby
e6753ab15d
fix: handle connection drop on multiform upload 2022-07-15 01:40:57 +07:00
Bobby
27f3bc3119
perf: don't wrap multipart handler in try-catch
fixed in https://github.com/kartikk221/hyper-express/releases/tag/6.3.0
2022-07-15 01:06:28 +07:00
Bobby Wibowo
7f6c29b136
fix: properly clean rejected empty files
should also now properly clean temp files from other unexpected errors
2022-07-14 18:18:39 +07:00
Bobby Wibowo
5ee82ce680
perf: don't wait for unlink promise in fail upload
just let it run in the background and respond to client immediately
2022-07-14 18:17:46 +07:00
Bobby Wibowo
7710e63d70
fix: prevent hashStream.update() after .dispose() 2022-07-14 17:01:59 +07:00
Bobby Wibowo
29b16edc04
perf: improve uploads flow
lessen temporary objects/variables creation,
and refactor some variable names to be more obvious
2022-07-14 16:35:06 +07:00
Bobby Wibowo
c32f18a697
fix: uploads mimetype not properly set 2022-07-12 15:26:53 +07:00
Bobby Wibowo
0f6409132a
feat: busboy limits and multipart errors handler
unfortunately to capture multipart errors, we have to wrap the entire
callback function with try-catch block

but it appears overall processing is still slightly faster than
expressjs + multer
2022-07-12 14:39:16 +07:00
Bobby Wibowo
7f9d05da26
feat: multer -> hyper-express multipartfield
get outta here multer, lmao
2022-07-12 13:07:13 +07:00
Bobby Wibowo
38d86779ae
refactor: HUGE REFACTOR for hyper-express 2022-07-10 19:46:25 +07:00
Bobby Wibowo
b3a304729f
refactor: uploadController pass errors 2022-07-10 14:20:49 +07:00
Bobby
2ca2fef301
feat: use SimpleDataStore for album pages cache
this should have better lifecycle and use less memory over time, since
we can define max items in cache
at the moment hard-coded to 10 cached pages (inclusive of nojs version
if ever generated)
2022-07-06 17:51:34 +07:00
Bobby Wibowo
1404cf9328
fix: utf-8 filename breaks
https://github.com/expressjs/multer/issues/1104
2022-07-03 10:18:04 +07:00
Bobby Wibowo
38e673226f
fix: inconsistent size field in DB !! yarn migrate
we used to store number directly into the string size field, and
during the conversion it seemed to always add ".0" at the final string,
probably because the driver or sqlite3 itself assumes float

please run yarn migrate after pulling this commit
if you skip converting the DB, file duplicates check will fail to
function

and in the future im planning to do size statistics in bigint, which
will also fail if not converted
2022-06-29 14:35:00 +07:00
Bobby Wibowo
b117fa9ad1
feat: expose utils to nunjucks templates
also fix nunjucks templates not adhering to env vars overrides for
domains config options, because they attempted to read config directly
2022-06-29 13:58:09 +07:00
Bobby
b17b24b159
feat: new page /file/:identifier
this will display all information recorded from the specified file, but
only to the users that own them (it requires token)

this page also has a delete file button, allowing us to provide link to
this page for sharex deletion url option

once again, this is only for authenticated users, and will only show
file that the users own, unless said user is a moderator or higher
2022-06-28 12:03:49 +07:00
Bobby
5d1bea39ef
feat: new api /api/upload/get/:identifier
this api only returns file that the user owns (thus token must be set)
2022-06-28 11:57:56 +07:00