Commit Graph

245 Commits

Author SHA1 Message Date
Bobby Wibowo
29b16edc04
perf: improve uploads flow
lessen temporary objects/variables creation,
and refactor some variable names to be more obvious
2022-07-14 16:35:06 +07:00
Bobby Wibowo
c32f18a697
fix: uploads mimetype not properly set 2022-07-12 15:26:53 +07:00
Bobby Wibowo
0f6409132a
feat: busboy limits and multipart errors handler
unfortunately to capture multipart errors, we have to wrap the entire
callback function with try-catch block

but it appears overall processing is still slightly faster than
expressjs + multer
2022-07-12 14:39:16 +07:00
Bobby Wibowo
7f9d05da26
feat: multer -> hyper-express multipartfield
get outta here multer, lmao
2022-07-12 13:07:13 +07:00
Bobby Wibowo
38d86779ae
refactor: HUGE REFACTOR for hyper-express 2022-07-10 19:46:25 +07:00
Bobby Wibowo
b3a304729f
refactor: uploadController pass errors 2022-07-10 14:20:49 +07:00
Bobby
2ca2fef301
feat: use SimpleDataStore for album pages cache
this should have better lifecycle and use less memory over time, since
we can define max items in cache
at the moment hard-coded to 10 cached pages (inclusive of nojs version
if ever generated)
2022-07-06 17:51:34 +07:00
Bobby Wibowo
1404cf9328
fix: utf-8 filename breaks
https://github.com/expressjs/multer/issues/1104
2022-07-03 10:18:04 +07:00
Bobby Wibowo
38e673226f
fix: inconsistent size field in DB !! yarn migrate
we used to store number directly into the string size field, and
during the conversion it seemed to always add ".0" at the final string,
probably because the driver or sqlite3 itself assumes float

please run yarn migrate after pulling this commit
if you skip converting the DB, file duplicates check will fail to
function

and in the future im planning to do size statistics in bigint, which
will also fail if not converted
2022-06-29 14:35:00 +07:00
Bobby Wibowo
b117fa9ad1
feat: expose utils to nunjucks templates
also fix nunjucks templates not adhering to env vars overrides for
domains config options, because they attempted to read config directly
2022-06-29 13:58:09 +07:00
Bobby
b17b24b159
feat: new page /file/:identifier
this will display all information recorded from the specified file, but
only to the users that own them (it requires token)

this page also has a delete file button, allowing us to provide link to
this page for sharex deletion url option

once again, this is only for authenticated users, and will only show
file that the users own, unless said user is a moderator or higher
2022-06-28 12:03:49 +07:00
Bobby
5d1bea39ef
feat: new api /api/upload/get/:identifier
this api only returns file that the user owns (thus token must be set)
2022-06-28 11:57:56 +07:00
Bobby Wibowo
9d4c08e460
feat: some env vars support 2022-06-22 13:27:38 +07:00
Bobby Wibowo
954df13aa2
fix: better null domain fallback logic 2022-06-22 11:22:35 +07:00
Bobby
8799189a04
feat: initiate db connection only once
db is now initiated via utilsController, and gets re-used by main
lolisafe.js script and the other controllers

this should now make sure we only have 1 active db connection pool ever

this is a pretty big refactor due to how we used to reference db,
but it should functionally be identical
2022-06-04 04:21:56 +07:00
Bobby
d2b8656800
fix: url uploads stricter error handling
this should properly dispose of unfinished write & hasher streams on
any errors if they have been initiated

also do content-length header check a bit more early
2022-05-08 15:35:28 +07:00
Bobby
ce71a9e8d6
feat: better size check for url uploads
first layer is via sending HEAD request to the url to determine its size
via content-length header
however not all hosts properly set the header, so we ignore it if
it isn't a valid number

next via size option in fetch(), which supposedly limits response body
size during the request itself (?)

lastly via checking actual bytes written to physical file as reported by
fs.createWriteStream()
2022-05-08 15:19:29 +07:00
Bobby
71a6adc3d3
fix: url uploads failing with missing content-type 2022-05-08 15:15:10 +07:00
Bobby
eeb1611b2a
fix: failsafes for chunks timeout 2022-05-08 12:01:18 +07:00
Bobby
f3b7d5e56d
fix: empty string for age header failing 2022-05-08 12:00:27 +07:00
Bobby
2d147e748b
feat: usergroup-based file retention periods
this supersedes the old temporaryUploadAges, while maintaining full
backwards-compatibility.

please consult config.sample.js if you want to start using this
2022-05-07 02:17:31 +07:00
Bobby
3d94ae599e
feat: debug log for clean files 2022-04-23 05:01:56 +07:00
Bobby
88d32a6dcf
refactor: uploadController.js 2022-04-23 04:48:59 +07:00
Bobby
86c26cb50c
feat: some bypass support to passthrough scanning
only usergroup and file extension bypass

real file size can't be determined before passthrough scan,
so there's no bypass by max file size
please read the comments in sample config file

refactored utils.clamscan into utils.scan
2022-04-23 04:44:01 +07:00
Bobby
9af52e068d
docs: config and uploadController 2022-04-16 23:06:25 +07:00
Bobby
4b9740cf5c
docs: uploadController.js 2022-04-16 20:44:11 +07:00
Bobby
b5aa8d0758
fix: passthrough scan 2022-04-16 13:33:11 +07:00
Bobby
db254c602b
feat: experimental clamscan passthrough support
when enabled, passthrough scanning will be used for non-chunked uploads

upload processing will be significantly faster if scanning is required
2022-04-15 16:41:05 +07:00
Bobby
6788dc2094
fix: align with clamscan v2 2022-04-15 15:36:50 +07:00
Bobby
ffc82f6a2a
feat: logger.debug() outputs only on dev mode 2022-04-15 15:30:32 +07:00
Bobby
b86aa8dc08
fix: private upload custom response config option 2021-06-08 08:01:11 +07:00
Bobby Wibowo
48488667c8
feat: limit upload to specific usergroups 2021-05-22 20:59:00 +07:00
Bobby Wibowo
5a74776978
fix: thumbnails not generated with mixed case
e.g. .jpG, .JPG, .PnG, etc.

Also simplified some codes.
2021-02-01 05:23:53 +07:00
Bobby Wibowo
631d14d887
feat: better extension names parsing
It will now preserve upper/lower/mixed case.

It will now properly preserve all tarballs (#370).
2021-01-29 23:15:24 +07:00
Bobby Wibowo
a5607c00f6
refactor: make use of improved utils.authorize() 2021-01-08 10:50:25 +07:00
Bobby Wibowo
b5af733dc2
refactor: Client/ServerError on uploadController 2021-01-08 09:44:04 +07:00
Bobby Wibowo
0dfdccb25e
refactor: move custom multer storage
controllers dir: multerStorageController.js to utils/multerStorage.js
2021-01-08 07:25:27 +07:00
Bobby Wibowo
51ab9a6fc5
fs.copyFile() for chunks data on non-default path
Closes #314
2020-11-21 06:31:36 +07:00
Bobby Wibowo
9c7241d145
Allow filtering audio files with is:audio
For now only support FLAC, MP3, WAV and WMA
More extensions will come at a later date
2020-11-03 22:51:29 +07:00
Bobby Wibowo
88f852584c
!! REPLACED ClamAV BACKEND: clamdjs -> clamscan !!
Update your config file!
2020-11-01 06:35:56 +07:00
Bobby Wibowo
47dd512910
Removed custom ESLint curly rule
Sigh, why did you do this, past me..?

Also fixed "Delete uploads by names".
2020-10-31 01:12:09 +07:00
Bobby Wibowo
22e6c2c3c6
Updated
Replaced all Array.concat() with spread operator and/or push().
Fixed some faulty search logics when using exclusion filters.
Fixed capitalization of mac font in _variables.scss. Stylelint somehow
fricked up again in my previous commit.
2020-10-11 17:32:22 +07:00
Bobby Wibowo
50dc747984
Bug fixed the new collision check
Geez.
2020-09-27 08:04:41 +07:00
Bobby Wibowo
ea15b145b1
Added alt stricter file name collision checks 2020-09-27 05:18:42 +07:00
Bobby Wibowo
08db3e55a8
Allow sorting by original names 2020-07-25 16:39:04 +07:00
Bobby Wibowo
49ec795941
Disabled file delete GET API
40dd66297612b2a8429c8fcad2b07170a3ba582b#commitcomment-40060391
2020-06-21 22:21:38 +07:00
Bobby Wibowo
40dd662976
Removed delete URL from upload response
I wasn't aware ShareX wouldn't do a basic GET request to the delete URL,
but would instead open your browser to it.
Unfortunately, due to the structure of lolisafe auth setting token
through HTTP header, which had always been done by JS as it has to get
the value from local storage, just visiting the API from browser will
merely respond with "No token provided".
We'd need cookie-based auth, which I do not want to add.
The old me that added deletion API long ago might have been aware of
this fact, but the me of a few hours ago already forgot..
I'll give this further thoughts at a later date, as ShareX isn't really
a priority to me.

Bumped v1 version string and rebuilt client assets.
2020-06-20 04:01:16 +07:00
Bobby Wibowo
51c8df71bc
Added deletion URL for ShareX or derivatives
For registered users only!
This requires adding a basic GET API for file deletion, so that I did.
Configs which guests download will not include pattern for delete URL,
so they won't get notified of unusable delete URL or anything like that.

dev: Improved logger.debug() to support specifying options for node's
Util.inspect() if an object is set as its last param
(assuming >1 params).
Default options now also includes enabling colors.

src/js/utils.js: Simplified dynamic ShareX config generator.
Among other things, it will now use JSON.stringify().
I don't even remember why we didn't use that in the first place..

Some logic improvements in src/js/home.js.

Bumped v1 version string and rebuilt client assets.
2020-06-20 01:28:23 +07:00
Bobby Wibowo
a4d33c67a2
Updated uploadController.js 2020-06-16 03:01:32 +07:00
Bobby Wibowo
585331c6e5
Added timeout on chunked uploads
The service will now automatically clean up any leftover data from any
inactive & unfinished chunked upload attempts.

Updated config.sample.js for updated description and new sub-option
for chunkSize option.
2020-06-15 23:48:43 +07:00