SEGFULT in portLibrary.str_printf() calls

[ehsankianifar]

When I was investigating an issue in openj9, I noticed an Assertion is causing segfult instead of printing the proper message.
the error happens when the assertion tries to generate an output in this method.
I was working on a JDK21 custom build and this is the stack trace:

#0  0x00000e26fffc92d0 in ?? ()
#1  0x000003fff7e51fbc in omrGcDebugAssertionOutput (portLibrary=0x3fff862b8f0, omrVMThread=0x3fffdff6900, format=<optimized out>) 
    at /root/jdk21_debug/omr/gc/base/ModronAssertions.cpp:48
#2  0x000003fff7e681e8 in MM_LargeObjectAllocateStats::getSizeClassIndex (this=this@entry=0x3fff8080a10, size=size@entry=0) 
    at /root/jdk21_debug/omr/gc/base/EnvironmentBase.hpp:260
#3  0x000003fff7e6ccca in MM_LargeObjectAllocateStats::decrementFreeEntrySizeClassStats (this=0x3fff8080a10, freeEntrySize=0, freeEntrySizeClassStats=0x3fff8080a78, count=1)
    at /root/jdk21_debug/omr/gc/stats/LargeObjectAllocateStats.cpp:1093
#4  0x000003fff7e43578 in MM_MemoryPoolAddressOrderedList::internalAllocateTLH (largeObjectAllocateStats=0x3fff8080a10, lockingRequired=true, addrTop=@0x3fff6dfdd98: 0xfdb21310,
    addrBase=@0x3fff6dfdd90: 0xfdb01310, maximumSizeInBytesRequired=131072, env=0x3fff864c9d8, this=0x3fff8080500) 
    at /root/jdk21_debug/omr/gc/base/MemoryPoolAddressOrderedList.cpp:677
#5  MM_MemoryPoolAddressOrderedList::allocateTLH (this=0x3fff8080500, env=0x3fff864c9d8, allocDescription=0x3fff6dfe340, maximumSizeInBytesRequired=131072, addrBase=@0x3fff6dfdd90: 0xfdb01310,
    addrTop=@0x3fff6dfdd98: 0xfdb21310)

I also check with GDB and the pointer it was returning (0xe22fffc9f74) was not a valid memory address:

(gdb) print portLibrary->str_vprintf
$1 = (uintptr_t (*)(OMRPortLibrary *, char *, uintptr_t, const char *, typedef __va_list_tag __va_list_tag *)) 0xe22fffc9f74

It might be an openJ9 build issue but I did not investigate since I was working on a different task.
To reproduce the bug, the easiest way might be to run a java application in GDB and put a breakpoint here and change the logValue to a negative number to make the assertions fail and triggers the issue.
This part of the code would be triggered anytime that GC increases the heap size so any java application with a bunch of heap allocation would work.

[ehsankianifar]

I was building and testing on a Linux on Z machine. I don't know if it is happens on other platforms like X86 or not!