Enable JVM Heap trimming by default

[fbricon]

@maxandersen and @franz1981 taught me java 17 has support for jvm reclaiming memory back to the os:

• https://bugs.openjdk.org/browse/JDK-8293114 this is the feature
• this is what we have for G1 instead: https://bugs.openjdk.org/browse/JDK-8204089

And Netty behaviour is what's missing from the puzzle (see netty/netty#11845 (comment))

Meaning that while idle, https://bugs.openjdk.org/browse/JDK-8204089 just shrink back the heap RSS to xms

we need 2 set of VM args to be passed:

• autotrim interval -> which requires something like -XX:+UnlockExperimentalVMOptions -XX:TrimNativeHeapInterval=5000
• G1 periodic GC: -> G1PeriodicGCInterval, G1PeriodicGCInvokesConcurrent and G1PeriodicGCSystemLoadThreshold (for https://bugs.openjdk.org/browse/JDK-8204089)

OpenJDK supports -XX:+IgnoreUnrecognizedVMOptions, in case users run an older JDK 17 that doesn't support the new flags, and OpenJ9 simply ignores unknown flags, so it's probably safe to try it out on insider builds.

@rgrunber @testforstephen @jdneo WDYT?

[franz1981]

thanks @fbricon !

[rgrunber]

Seems reasonable. If JDT-LS would look like less of a memory hog, that's nice. After initial project import, I would guess a lot of the heap space could be reclaimed.

While on the topic of trimming the heap, there are another set of flags (specific to JDT Core) that I tried out a while ago on eclipse-jdt/eclipse.jdt.core#1526 (comment) that seemed to have promising results. (update: ugh, but on the downside, they may slow down project import.. needs to be tried)

[testforstephen]

We can put it in the insiders first and then see if it affects the performance of project import and features such as code completion.

[franz1981]

@rgrunber beware that the 2 features act on different fronts:

• autotrim help with off-heap allocations (as presented by Native memory tracking), which fall into Just-in-time compilation native memory usage, JNI external libraries which perform malloc/free (or internal ones, due to network/file I/O/zipping libraries ops), classloading and manually allocated NIO direct buffers (eg the ones subject to MaxDirectMemory): the "limit" for the JIT ones is that the Arena used by the JIT will free every each 5 seconds, hardcoded (see https://github.com/openjdk/jdk/blob/jdk-21%2B35/src/hotspot/share/memory/arena.cpp#L119) and until memory is NOT yet free, autotrim doesn't have much to trim. Maybe I can speak with the JDK folk to make it tunable as well...
• G1 periodic GC instead release RSS memory back just related the java heap: eg after starting quarkus with N extensions, some of them uses the Java heap to parse the configuration, creating allocation spikes and enlarging the heap past xms to xmx. The same happen if there's some load for some time....Tuning G1 periodic GC will allow, if "idle" (to be defined/configured) and N seconds, to return back to the minimum (till xms) heap occupation that will fit with the amount of live data (which should be very few, at idle). This will make G1 able to return it back to the OS (lowering RSS).

[rgrunber]

I ran vscode-java with -XX:NativeMemoryTracking=summary & then experimented with/without -XX:+UnlockExperimentalVMOptions -XX:TrimNativeHeapInterval=5000. I just did an import of https://github.com/eclipse-jdtls/eclipse.jdt.ls/ (with vscode-pde installed). I didn't see any noticeable differences under any of the headings for jcmd ${PID} VM.native_memory .

We do use the parallel GC by default though. See below for the default options we set.

vscode-java/package.json

Line 253 in 86bf3ae

"default": "-XX:+UseParallelGC -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xmx1G -Xms100m -Xlog:disable",

. These options were done as part of #1262 . Would be curious to hear your thoughts on something like -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 as those were 2 options we didn't adopt.

We could potentially, detect if a user has set their own options that include the G1 GC, and if so add our option dynamically prior to starting. We do add some options dynamically in

vscode-java/src/javaServerStarter.ts

Line 86 in 86bf3ae

function prepareParams(requirements: RequirementsData, workspacePath, context: ExtensionContext, isSyntaxServer: boolean): string[] {

.

[franz1981]

I didn't see any noticeable differences under any of the headings for jcmd ${PID} VM.native_memory .

In order to benefit of this one specifically you should start an application which does something "interesting enough" at startup (eg Quarkus/spring) triggering some native compilation. Later on, after >5 seconds, the JIT arenas can free some memory back to the OS, and in-between 5 seconds (interleaved or not, hence waiting >10 seconds is the safer choice) you can measure the process RSS as described by https://quarkus.io/guides/performance-measure#platform-specific-memory-reporting

Sadly, native memory tracking isn't currently able to detect the actual RSS footprint, but just what the OS allocator believe to be its live data (malloc'd not freed yet) and not the actual footprint underlying it.
@tstuefe can confirm it

[tstuefe]

I didn't see any noticeable differences under any of the headings for jcmd ${PID} VM.native_memory .

In order to benefit of this one specifically you should start an application which does something "interesting enough" at startup (eg Quarkus/spring) triggering some native compilation. Later on, after >5 seconds, the JIT arenas can free some memory back to the OS, and in-between 5 seconds (interleaved or not, hence waiting >10 seconds is the safer choice) you can measure the process RSS as described by https://quarkus.io/guides/performance-measure#platform-specific-memory-reporting

Sadly, native memory tracking isn't currently able to detect the actual RSS footprint, but just what the OS allocator believe to be its live data (malloc'd not freed yet) and not the actual footprint underlying it. @tstuefe can confirm it

Worse, NMT only tells you how much the hotspot has allocated currently, leaving out native memory consumption from the rest of the process.

Before the very first trim, you can find out how much memory is retained by the glibc by doing

thomas@starfish$ jcmd xxxx VM.info | grep retained
C-Heap outstanding allocations: 46526K, retained: 2104565K (may have wrapped)

This info comes from the glibc itself.

• "outstanding allocations" is what the whole process has allocated from c heap (hotspot, jdk, third-party code, system libs)
• "retained" is what the glibc has cached for future use and not returned to the OS.
  Caveats:
• retained is only valid until the first trim. This is because glibc does the trimming "blind" - it calls madvise on larger retained blocks without any bookkeeping. So this number does not change due to trimming, even though trimming returns retained memory to the OS
• on older glibcs, these numbers wrap at 4GB (before 2.37 I think). Hence the "may have wrapped" moniker.

You can also find out RSS via jcmd:

thomas@starfish$ jcmd xxx VM.info | grep -i resident
Resident Set Size: 2681496K (peak: 2683284K) (anon: 2658664K, file: 22832K, shmem: 0K)

[franz1981]

Many thanks @tstuefe I believe the option to get RSS out of jcmd to be a great addition because would work the same regardless being in a container or not!

And adding quarkusio/quarkus#36691 (reply in thread) as a positive data point of using trimming, coming from a user.