Elastic JVM for Scalable Java EE Applications Running in Containers #JakartaTechTalks

Ruslan Synytsky
Elastic JVM
for Java EE Applications
Running in Containers

Agenda
● Java Memory Usage Problems
● JDK Improvements for Elastic Java Memory Scaling
● Elasticity and Jakarta EE
● Garbage Collection Testing Results

Java Memory Consumption Problems
The most widely acknowledged issue with Java EE is large memory requirements (40%), then
slow startup times (40%), followed by missing technologies and speciﬁcations (20%)
Jakarta EE Developer Survey 2018

Unreleased Heap Memory
Over-Allocation and Underutilization

Reasons to Seek JVM Elasticity
Cron jobs for data
processing
Getting resources from
common pool
Elastic cloud hosting with
pay-per-use
ELASTICITY
NO
ELASTICITY
Automated scaling of
resources on the fly without
JVM restart and downtimes
JVM restart required with
every change of resources
amount leads to downtimes or
OOMError

OOM Error and OOM Killer
● OutOfMemoryError exception is usually thrown when there is insufficient
space to allocate an object in the Java heap or insufficient native memory to
support the loading of a Java class
● oom_kill is a job that helps to sacrifice one or more processes in order to
free up memory for the system

Understanding of the OutOfMemoryError Exception
● java.lang.OutOfMemoryError: Java heap space
● java.lang.OutOfMemoryError: GC Overhead limit exceeded
● java.lang.OutOfMemoryError: Requested array size exceeds VM limit
● java.lang.OutOfMemoryError: Metaspace
● java.lang.OutOfMemoryError: request size bytes for reason. Out of swap
space?
● java.lang.OutOfMemoryError: Compressed class space
● java.lang.OutOfMemoryError: reason stack_trace_with_native_method
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html

Understanding of the OutOfMemoryError Exception

Runtime Environments
● Application Containers
● System Containers
● Virtual Machines

Pay-Per-Use Billing Model
Using automatic vertical scaling, cloud providers can oﬀer economically
advantageous pricing based on the real resource consumption
Forbes - Deceptive Cloud Eﬃciency: Do You Really Pay As You Use?

Heap Vertical Scaling
Unreleased Heap Memory

Calling Full GC Periodically (Before OpenJDK12)
https://github.com/jelastic-jps/java-memory-agent
Compacting GC cycles are not triggered automatically and must be
executed explicitly
Workaround:
inject an agent which monitors the memory usage and calls System.gc()
periodically:
-javaagent:jelastic-gc-agent.jar=period=300,debug=true

G1 and Full GC
java -XX:+UseG1GC -Xmx2g -Xms32m -jar app.jar 0
Memory grew from 32 MB to 1 GB in 25 seconds
https://github.com/jelastic/java-vertical-scaling-test

Timely Reduce Unused Committed Memory (JEP 346)
Make the G1 garbage collector automatically give back Java heap memory to
the operating system when idle
● G1PeriodicGCInterval
● G1PeriodicGCSystemLoadThreshold
● G1PeriodicGCInvokesConcurrent
JEP 346: Promptly Return Unused Committed Memory from G1
java -Xms32M -Xmx2g -XX:+UseG1GC -XX:G1PeriodicGCSystemLoadThreshold=0.6
-XX:G1PeriodicGCInterval=900k -jar app.jar

Improved Elasticity
Automatically Released Heap

Community Recognition
Special Appreciation
to Rodrigo Bruno
Senior/Postdoc Researcher at the Systems
Group in ETH Zurich.
PhD in Técnico (University of Lisbon)

Load Testing Logic
https://github.com/jelastic/java-vertical-scaling-test/blob/ma
ster/src/com/jelastic/verticalscaling/Load.java#L50
java [OPTIONS] -jar app.jar <sleep> <mode>
where
sleep - 10
mode - 1

Auto Testing Package
https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest.yml

G1 Collector (-XX:+UseG1GC)
The Garbage-First (G1) is a server-style Garbage Collector for multiprocessor
machines with a large amount of memory. The heap is partitioned into
ﬁxed-sized regions and G1 tracks the live data in those regions. When Garbage
Collection is required, it collects from the regions with less live data ﬁrst.
● 2004, Sun Microsystems
JEP 346: Promptly Return Unused Committed Memory from G1

G1
-Xmx3g -Xms32m -XX:+UseCompressedOops -XX:+UseG1GC -XX:G1PeriodicGCInterval=1k

G1 and G1PeriodicGCSystemLoadThreshold
https://github.com/lxc/lxcfs/
Using LXCFS to Improve Container Resource Visibility
Threshold for the current system load as returned by the hosts getloadavg() call to determine whether a
periodic garbage collection should be triggered:
● a current system load higher than the tigger value prevents periodic garbage collections
● zero value indicates that this threshold check is disabled
If running in Docker container then use

Shenandoah GC (-XX:+UseShenandoahGC)
Shenandoah GC is a concurrent garbage collector for the JVM. GC tries to
perform most of the activities in parallel without interrupting application
performance. Such parallelism makes “stop-the-world” (STW) pauses extremely
short. Another inherent advantage is an eﬃcient work with small and large heaps
with no impact on STW pauses’ length.
● 2014, Christine H. Flood, Red Hat
https://wiki.openjdk.java.net/display/shenandoah/Main#Main-Heuristics

-Xmx3g -Xms32m -XX:+UseCompressedOops
-XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact
Shenandoah

G1 vs Shenandoah - CPU Usage
G1
Shenandoah

ZGC (-XX:+UseZGC)
ZGC is low latency scalable garbage collector. Designed for use with
applications that require a large heap and low latency. It uses a bunch of one
generation and performs most (but not all) garbage collection in parallel with
uninterrupted application work. This greatly limits the impact of garbage
collection on your application response time.
● 2018, Per Liden, Oracle
JEP 351: ZGC: Uncommit Unused Memory - available from JDK 13 Release

-Xmx3g -Xms32m -XX:+UnlockExperimentalVMOptions
-XX:+UseZGC -XX:ZUncommitDelay=1 -XX:ZCollectionInterval=30
ZGC @ Oracle OpenJDK

C4 GC
● 2010, Gil Tene, Azul Systems
The C4 (Continuously Concurrent Compacting Collector) is an updated
generational form of the Azul Pauseless GC Algorithm and is the default
collector of Zing®. C4 diﬀerentiates itself from other generational garbage
collectors by supporting simultaneous – generational concurrency: the
diﬀerent generations are collected using concurrent (non-stop-the-world)
mechanisms that can be simultaneously and independently active. Unlike
other algorithms, it is not ‘mostly’ concurrent, but fully concurrent, so it
never falls back to a stop-the-world compaction.

-Xmx500m -Xms32m -XX:+UseZST
C4 @ Zing
pmem.conf -> cgroups enabled

ConcMarkSweep GC (-XX:+UseConcMarkSweepGC)
ConcMarkSweep GC collector is designed for applications that prefer shorter
garbage collection pauses and which can aﬀord to share processor resources
with the garbage collector while the application is running. It makes sense to use
such a collector when applications requirements for time garbage collection
pauses are low.

-Xmx3g -Xms32m -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
+ periodical jcmd <pid> GC.run
ConcMarkSweep

Serial GC (-XX:+UseSerialGC)
Serial GC performs garbage collection in a single thread and has the lowest
consumption of memory among all GC types but, at the same time, it makes
long pauses that can lead to application performance degradation.

-Xmx3g -Xms32m -XX:+UseCompressedOops -XX:+UseSerialGC
Serial

OpenJ9
OpenJ9 uses the Generational Concurrent (-Xgcpolicy:gencon) policy by
default, which is best suited to transactional applications that have many short
lived objects. Alternative policies are available, including those that cater for
applications with large Java heaps (-Xgcpolicy:balanced), applications that are
sensitive to response-time (-Xgcpolicy:metronome), or applications that require
high application throughput (-Xgcpolicy:optthruput).
● 2017, Eclipse Foundation

-XX:+IdleTuningCompactOnIdle -XX:+IdleTuningGcOnIdle -XX:IdleTuningMinIdleWaitTime=1
-Xjit:waitTimeToEnterDeepIdleMode=1000
Bash command to check the real usage
while true
do
pid=$(pgrep -f java | tail -n1)
used=$(ps -orss --no-headers --pid $pid)
echo "scale=2 ; $used / 1024/1024" | bc
sleep 1
done
Inconsistent behaviour with -XX:+IdleTuningGcOnIdle, mem not released back to OS on Idle
OpenJ9

Epsilon GC (-XX:+UseEpsilonGC)
Epsilon GC is a passive GC that handles memory allocation and doesn’t clear it
when objects are no longer used. When your application exhausts the Java
heap, the JVM goes down. So, EpsilonGC prolongs an application life until the
memory will run out and dumps the memory, that can be useful for application
memory usage debugging, as well as measuring and managing application
performance.
● 2014, Aleksey Shipilev, Red Hat

-XX:+UnlockExperimentalVMOptions -XX:+UseEpsilonGC
Epsilon

Parallel GC (-XX:+UseParallelGC)
Parallel GC is a “stop-the-world” multithreaded Garbage Collector similar to the
serial collector. The primary diﬀerence is that multiple threads are used to speed
up garbage collection. By default, both minor and major collections are
executed in parallel to further reduce garbage collection costs.

-Xmx3g -Xms32m -XX:+UseCompressedOops -XX:+UseParallelGC
Parallel

Main Points of Vertical Scaling
A - initial usage
B - maximum usage
C - growth speed
D - duration before release
E - release speed
F - minimum usage after release
Different GCs provide different results and fine tuning options

Running GC
Tests in Kubernetes

Auto Testing Package for Kubernetes
https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest-k8s.yaml

Load Testing Logic
java [OPTIONS] -jar app.jar <sleep> <mode>
where
sleep - 100
mode - 2
https://github.com/jelastic/java-vertical-scaling-test/blob/master/
src/com/jelastic/verticalscaling/Load.java#L64

ZGC @ Oracle OpenJDK in Kubernetes

Joint Comparison - Several Load Cycles
RAM CPU

Memory Usage in Layers
The source code should not
have memory leaks and should
timely release unused objects

Load Testing Logic
● simple .war artifact deployed with Payara 5.192
● JSP that sets 1MB attribute in session
● 1m session timeout
● https://github.com/jelastic/java-vertical-scaling-test/tree/payara/webapp
Load test GETs Payara webapp endpoint n times (n differs depending on Java version and GC in use):
for i in {1..n}; do curl -s localhost:8080 > /dev/null; done

Shenandoah
-Xmx3g -Xms32m -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact
for i in {1..1000}; do curl -s localhost:8080 > /dev/null; done

G1 @ Dragonwell
-Xmx3g -Xms32m -XX:+UseCompressedOops -Xms3g
-XX:+G1ElasticHeap -XX:+ElasticHeapPeriodicUncommit
-XX:ElasticHeapPeriodicYGCIntervalMillis=10000
-XX:ElasticHeapYGCIntervalMinMillis=1000
-XX:ElasticHeapInitialMarkIntervalMinMillis=1000
-XX:ElasticHeapPeriodicInitialMarkIntervalMillis=300000
-XX:ElasticHeapPeriodicUncommitStartupDelay=1
-XX:ElasticHeapPeriodicMinYoungCommitPercent=10
for i in {1..6000}; do curl -s localhost:8080 > /dev/null; done

Heap Resizing
Restart for Xmx Resize

-XX:SoftMaxHeapSize @ ZGC
SoftMaxHeapSize is set for the GC to
strive not to grow heap size beyond the
speciﬁed size unless it is highly needed:
● to keep the heap footprint down, while
maintaining the capability to deal with
a temporary increase in heap space
requirement
● with lots of margin, to increase
conﬁdence that you will not run into
an allocation stall because of an
unforeseen increase in allocation rate

JEP draft: Dynamic Max Memory Limit
Xmx can be set higher than the container max memory limit
(Cmx). And both Smx and Cmx can be adjusted on the ﬂy
without the need to restart JVM or container.
At the moment the heap size can go beyond
SoftMaxHeapSize (Smx) and there is no guarantee on how
much the heap will grow other than up to Xmx.
The problem arises when Smx < Cmx < Used Heap < Xmx:
the JVM will be killed by the OS OOM Killer as it exceeds
the amount of memory given to the container.
We suggest to provide an option for making
SoftMaxHeapSize as the hard limit, so when overshoot
happens JVM will throw OOM Error which is not as bad
OOM Kill.
Dynamic Max Memory Limit @ G1

-Xsoftmx @ OpenJ9
https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/openj9/xsoftmx/index.html
Runtime adjustable heap size (-Xsoftmx) allows to adjust heap size dynamically
and take advantage of hot-add of memory.
You can set this option on the command line, then modify it at run time by using
the com.ibm.lang.management.MemoryMXBean.setMaxHeapSize().
This option can be useful in virtualized or cloud environments, for example,
where the available memory might change dynamically to meet business needs.
By default, -Xsoftmx is set to the same value as -Xmx.

C4 is fully elastic and can return all empty pages to the OS after each GC cycle.
However, C4 sticks to the Xmx it was given, and avoid doing heavy elastic memory dance,
since relinquishing memory mappings and reestablishing them on Linux kernels is
bandwidth-limited in practice by the rate of page mapping invalidation the kernel can handle.
C4 goes above Xmx rather than go between Xms and Xmx. JavaMemMax option controls
the true maximum. In the future it will allow both scenarios where above-Xmx is allowed and
where above-Xmx is prohibited.
Two modes:
● Contingency (default mode) - goes above Xmx if it absolutely has to and will work hard
to collect and stay below Xmx.
● Insurance (best eﬀort elasticity) - borrows available memory and goes above Xmx in
order to delay GC whenever possible.
JavaMemMax @ С4 + ZST (Zing System Tools)

Keep Only Best Java Memories
Learn More
Get In Touch
@siruslan
rs@jelastic.com

Elastic JVM for Scalable Java EE Applications Running in Containers #JakartaTechTalks

Recommended

More Related Content

What's hot (20)

Similar to Elastic JVM for Scalable Java EE Applications Running in Containers #JakartaTechTalks (20)

More from Jelastic Multi-Cloud PaaS (20)

Recently uploaded (20)

Elastic JVM for Scalable Java EE Applications Running in Containers #JakartaTechTalks