Commit Graph

179 Commits

Author SHA1 Message Date
Bobby
2a8901a903
fix: ipv6 support for chunked uploads 2023-04-09 04:13:29 +07:00
Bobby
729883a4f5
feat: remove all root bypasses
initial migration to usergroup-system for root user is governed by
superadminAccount and superadminForcePromote fields in config file

those will have default values expected for non-fork installs, but
existing fork installs will not have them and thus will never trigger
superadmin force-promotion from migration script
2023-02-25 18:16:11 +07:00
Bobby
96689278f1
refactor: defer reject uploads after consume
always store incoming uploads to disk first before deciding whether to
reject and delete them depending on configured rejects/limits

doing this is more stable/reliable than attempting to immediately
reject the incoming connections, at least if we want to respond with
actual error messages
2022-10-16 11:08:53 +07:00
Bobby
c5bee11a42
feat: prep stats api to allow pre-category refresh
via /api/stats/:category
2022-10-08 16:39:28 +07:00
Bobby
3d51fd2e31
fix: purgeCloudflareCache
regression when ConfigManager was introduced
2022-10-06 04:20:02 +07:00
Bobby
ff58054158
fix: type-is uploads filtering
also improved its parsing logic
2022-10-06 03:26:19 +07:00
Bobby
0c0b2aa3d1
feat: StatsManager 2022-10-06 02:46:49 +07:00
Bobby
ec0e51a7b8
feat: Constants 2022-10-06 02:44:31 +07:00
Bobby
cd2baba0d6
feat: ScannerManager 2022-10-06 02:42:36 +07:00
Bobby
95e5956313
feat: ConfigManager 2022-10-06 02:39:51 +07:00
Bobby
a49842a932
fix: size attribute of uploads with stripped tags
it'd previously always keep size attribute before the tags stripping
2022-10-04 05:42:00 +07:00
Bobby
f3bf3f6500
feat: use fs-jetpack in utilsController.js 2022-10-04 05:33:25 +07:00
Bobby
c47d3f5437
feat: better upload.onHold debugging messages
expose util.inspect via utils when on dev mode
2022-09-24 06:33:58 +07:00
Bobby
32efacd411
feat: configure timemark for video thumbnails 2022-09-24 06:24:56 +07:00
Bobby
93dc820368
feat: fetch() with AbortController for timeout
to be used with URL uploads

please consult the comments in the source files

this adds new dependency abort-controller
2022-09-14 01:33:31 +07:00
Bobby
96522a425f
chore: indents 2022-08-23 15:16:02 +07:00
Bobby
09fea107f3
feat: env SERVE_STATIC_QUICK=0 yarn start
restore an older behavior of serving static files with
ServeLiveDirectory middleware instead of ServeStaticQuick, in case of
issues with streaming files from disk via createReadStream

if you do encounter this issue, serving uploaded files with node will
likely encounter issues as well, but you can choose to serve them
directly with nginx, etc., so i guess it's not beyond hope
2022-08-21 21:23:14 +07:00
Bobby Wibowo
bd71035517
fix: missing cpu temp and swap logic 2022-08-19 10:48:45 +07:00
Bobby Wibowo
991cfe134e
feat: add cpu temp and swap memory to statistics
temp is currently hard-coded to C

also increased system info caching to 1s
2022-08-19 10:38:24 +07:00
Bobby Wibowo
86d9db3137
feat: split service info in statistics
additionally add cpu detail in system info
2022-08-19 10:18:35 +07:00
Bobby Wibowo
6ff735badb
feat: wrap add to album db query in transaction
additionally allow superadmins to arbitrarily add/remove files to/from
albums via manual API calls, instead of only allowing root user
2022-08-09 17:18:56 +07:00
Bobby Wibowo
b21fa66e59
fix: possible event listeners memory leak 2022-08-05 00:54:44 +07:00
Bobby Wibowo
7381cac0e9
chore: indents and comments 2022-08-04 21:59:50 +07:00
Bobby Wibowo
d7d6a29123
feat: cleaned up routes init
asserting auth and JSON body will now be done via route-specific
mini middlewares (authController's requireUser or optionalUser)
2022-08-04 21:59:06 +07:00
Bobby Wibowo
0ebefe083a
refactor: removed clamscan passthrough support
unfortunately it simply was not reliable enough

and maintaining it is simply adding more complexity to the codes

moreover it was only possible to passthrough regular non-chunked uploads
2022-08-02 16:19:57 +07:00
Bobby Wibowo
285e79c5a7
feat: configurable uploads/albums/users per page
please check sample.config.js for new options

if missing from config, defaults to 25 per page (old defaults)
2022-07-31 15:51:32 +07:00
Bobby
4591b8bb42
refactor: generateUniqueToken -> getUniqueToken
this now matches lifecycle with similar functions in upload and album
controllers

also added a new util function .mask() for basic string masking
2022-07-30 08:37:57 +07:00
Bobby Wibowo
fae28f9aa2
feat: deprecate uploads.cacheFileIdentifiers conf
maintaining it is an unnecessary complexity
it's a feature that doesn't scale too well anyways

also renamed "queryDbForFileCollisions" to
"queryDatabaseForIdentifierMatch"
and updated config description accordingly

this should also now properly free the internal onHold Set
2022-07-29 09:14:55 +07:00
Bobby Wibowo
235a1c56e1
refactor: move devmode flag to utils 2022-07-25 07:39:35 +07:00
Bobby Wibowo
1b109e0dc0
feat: req content-type json -> application/json 2022-07-25 06:12:55 +07:00
Bobby
776ab8ab37
feat: assert request content-type in post apis 2022-07-22 08:40:40 +07:00
Bobby Wibowo
ac63f8b76d
perf: no try-catch block on cloudflare cache purge 2022-07-14 14:41:55 +07:00
Bobby Wibowo
38d86779ae
refactor: HUGE REFACTOR for hyper-express 2022-07-10 19:46:25 +07:00
Bobby Wibowo
3a415165b4
feat: utilsController pass errors 2022-07-10 14:15:14 +07:00
Bobby
2ca2fef301
feat: use SimpleDataStore for album pages cache
this should have better lifecycle and use less memory over time, since
we can define max items in cache
at the moment hard-coded to 10 cached pages (inclusive of nojs version
if ever generated)
2022-07-06 17:51:34 +07:00
Bobby Wibowo
8a1ff434d9
feat: in-memory caching of content-disposition
please read config.sample.js
ignore if not serving files with node or not having the option turned on
2022-07-03 10:35:36 +07:00
Bobby Wibowo
ab96bd5d99
feat: queue cloudflare purge cache with fastq 2022-06-29 17:52:16 +07:00
Bobby Wibowo
9d38c431dc
fix: statistics failing to respond on errors 2022-06-29 14:29:47 +07:00
Bobby Wibowo
9d4c08e460
feat: some env vars support 2022-06-22 13:27:38 +07:00
Bobby
8799189a04
feat: initiate db connection only once
db is now initiated via utilsController, and gets re-used by main
lolisafe.js script and the other controllers

this should now make sure we only have 1 active db connection pool ever

this is a pretty big refactor due to how we used to reference db,
but it should functionally be identical
2022-06-04 04:21:56 +07:00
Bobby
2d147e748b
feat: usergroup-based file retention periods
this supersedes the old temporaryUploadAges, while maintaining full
backwards-compatibility.

please consult config.sample.js if you want to start using this
2022-05-07 02:17:31 +07:00
Bobby
4ecec788d0
feat: console logs of temp uploads check ups 2022-05-07 02:01:33 +07:00
Bobby
395361d5fc
fix: count of album zips generated in stats
change logic to list physical files instead, since the zipGeneratedAt
attribute may still exist despite site owners having already done
physical clean-ups
2022-05-05 14:54:21 +07:00
Bobby
7dd4f50e9e
feat: allow markdown in album description 2022-05-05 12:55:21 +07:00
Bobby
86c26cb50c
feat: some bypass support to passthrough scanning
only usergroup and file extension bypass

real file size can't be determined before passthrough scan,
so there's no bypass by max file size
please read the comments in sample config file

refactored utils.clamscan into utils.scan
2022-04-23 04:44:01 +07:00
Bobby
1c0fd26496
refactor: pass less data into user object 2022-04-16 21:36:34 +07:00
Bobby
db254c602b
feat: experimental clamscan passthrough support
when enabled, passthrough scanning will be used for non-chunked uploads

upload processing will be significantly faster if scanning is required
2022-04-15 16:41:05 +07:00
Bobby
6788dc2094
fix: align with clamscan v2 2022-04-15 15:36:50 +07:00
Bobby
ffc82f6a2a
feat: logger.debug() outputs only on dev mode 2022-04-15 15:30:32 +07:00
Bobby
e31af2d267
fix: round down system uptime
invalidate system info cache quicker (1s to 0.5s)
2021-09-08 00:46:08 +07:00