Re-introduce monkey patched GSON to make converter run better.

fixup! Re-introduce monkey patched GSON to make converter run better.

fixup! Re-introduce monkey patched GSON to make converter run better.
This commit is contained in:
Viktor Lofgren 2023-06-12 17:42:30 +02:00 committed by Viktor
parent d1a004bea6
commit 266ad2e4de
7 changed files with 1691 additions and 0 deletions

View File

@ -11,6 +11,9 @@ java {
} }
} }
dependencies { dependencies {
implementation project(':third-party:monkey-patch-gson')
implementation project(':code:common:model') implementation project(':code:common:model')
implementation project(':code:api:index-api') implementation project(':code:api:index-api')
implementation project(':code:common:service-discovery') implementation project(':code:common:service-discovery')

View File

@ -19,6 +19,9 @@ application {
tasks.distZip.enabled = false tasks.distZip.enabled = false
dependencies { dependencies {
implementation project(':third-party:monkey-patch-gson')
implementation project(':code:common:process') implementation project(':code:common:process')
implementation project(':third-party:porterstemmer') implementation project(':third-party:porterstemmer')

View File

@ -75,6 +75,7 @@ include 'third-party:uppend'
include 'third-party:openzim' include 'third-party:openzim'
include 'third-party:count-min-sketch' include 'third-party:count-min-sketch'
include 'third-party:monkey-patch-opennlp' include 'third-party:monkey-patch-opennlp'
include 'third-party:monkey-patch-gson'
dependencyResolutionManagement { dependencyResolutionManagement {

View File

@ -17,3 +17,4 @@ or lack an artifact, or to override some default that is inappropriate for the t
### Monkey Patched ### Monkey Patched
* [Stanford OpenNLP](monkey-patch-opennlp/) - Apache-2.0 * [Stanford OpenNLP](monkey-patch-opennlp/) - Apache-2.0
* [GSON](monkey-patch-gson/) - Apache-2.0

View File

@ -0,0 +1,17 @@
plugins {
id 'java'
}
java {
toolchain {
languageVersion.set(JavaLanguageVersion.of(17))
}
}
dependencies {
implementation libs.bundles.gson
}
test {
useJUnitPlatform()
}

13
third-party/monkey-patch-gson/readme.md vendored Normal file
View File

@ -0,0 +1,13 @@
# Monkey Patched GSON
Stanford OpenNLP - Apache-2.0
## Rationale
GSON makes some assumptions that make it not work very well
for deserializing extremely large JSON objects. This patch
makes the code technically leak memory, but the way it's used
makes this not much of a problem.
It should only be applied to the converter or possibly
loader processes, not the services.

File diff suppressed because it is too large Load Diff