State of Java Elasticity. Tuning Java Efficiency - GIDS.JAVA LIVE 2020

A STATE OF JAVA ELASTICITY
Tuning Java Eﬃciency

About Speaker
● Ruslan Synytsky
● CEO and co-founder of Jelastic PaaS
● Java Champion
● Two-times Duke’s Choice Award Winner
● Former lead of engineering team at
National Data Center (NDC) at National
Space Agency of Ukraine
● @siruslan

Agenda
● Java Memory Usage Problems and Use Cases
● Recent Improvements in OpenJDK
● Testing Elasticity with Diﬀerent Garbage Collectors
● Upcoming Improvements in OpenJDK

Unreleased Heap Memory
The Problem Symptoms
Over-Allocation and Underutilization

Java Memory Usage Pain Points
Q: Return memory from JVM to OS
Q: Java VM - does the freed memory return to the OS?
Q: More flexible memory management
Q: Force JVM to free memory
Q: Why Java does not release memory?
Q: Does GC release back memory to OS?
Q: Why does this java process not release memory?
...

Large Java Memory Requirements
WHAT IS THE MOST CHALLENGING ASPECT OF WORKING WITH JAVA EE?
Jakarta EE Developer Survey 2018
“The most widely acknowledged issue
when employing with Java EE is large
memory requirements (40%)”

Reasons to Seek Java Elasticity Getting resources from common
Desktop
Applications,
IDEs
Scheduled
tasks for data
processing
Cloud cost
saving with
pay-per-use
NEED FOR ELASTICITY
Automated scaling of
resources on the ﬂy without
JVM restart and downtimes
Dynamic
stateful
workloads
Right-sizing
problem with
containers

Transition from Monolith to Microservices

Resource Limit vs Real Usage in VM

Resource Limit vs Real Usage in Container

Elasticy Use Case in the Cloud

Elasticy Use Case in the Cloud
Pool of Resources

Real Statistics of Resource Consumption with Containers

Using automatic vertical scaling, cloud provides can oﬀer economically
advantageous pricing based on the actual resource consumption
Forbes - Deceptive Cloud Eﬃciency: Do You Really Pay As You Use?
Pay-As-You-Go Pay-per-Use
Pay-As-You-Go vs Pay-per-Use

Understanding JVM Footprint
MAXIMUM MEMORY USAGE
[-Xmx] + [-XX:MaxMetaspaceSize] + [-XX:MaxDirectMemorySize] + Num_Of_Threads * [-Xss]
Useful discussion at SoW Java using much more memory than heap size and presentation Memory Footprint of a Java Process

-Xmx=[25% of OS Total Memory]
Useful options to adjust defaults in % -XX:MaxRAMPercentage
-XX:MaxMetaspaceSize=[unlimited]
-XX:MaxDirectMemorySize=[-Xmx]
-Xss[1m]
Alternative syntax -XX:ThreadStackSize=[1m]
Specify these options explicitly if you face related OutOfMemoryError
Default Limits (Depend on The Platform!)

Understanding of the OutOfMemoryError Exceptions
OutOfMemoryError exception is usually thrown when there is insuﬃcient
space to allocate an object in the Java heap or insuﬃcient native memory to
support the loading of a Java class
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html
● java.lang.OutOfMemoryError: Java heap space
● java.lang.OutOfMemoryError: GC Overhead limit exceeded
● java.lang.OutOfMemoryError: Requested array size exceeds VM limit
● java.lang.OutOfMemoryError: Metaspace
● java.lang.OutOfMemoryError: Compressed class space
● java.lang.OutOfMemoryError: reason stack_trace_with_native_method
● java.lang.OutOfMemoryError: request size bytes for reason. Out of swap space?

An Example of OutOfMemoryError Exception
Used Heap > Xmx

An Example of OOM Killer Situation (Even Worse)
oom_kill is a job that helps to sacriﬁce one or
more processes in order to free up memory
for the system

The Heap Vertical Scaling Problem Symptoms
Unreleased Heap Memory

Calling Full GC Periodically (Workaround)
https://github.com/jelastic-jps/java-memory-agent
As compacting GC cycles are not triggered automatically, we execute
them explicitly by injecting an agent which monitors the memory usage
and calls System.gc() periodically:
-javaagent:jelastic-gc-agent.jar=period=300,debug=true
Consider enabling -XX:+ExplicitGCInvokesConcurrent

G1 and Full GC
java -XX:+UseG1GC -Xmx2g -jar app.jar
https://github.com/jelastic/java-vertical-scaling-test

Simple Jenkins Example
After Full GC call initiated by Jelastic GC agent the memory consumption goes down to 0.5G. If we
disable the agent then the memory usage at the idle JVM will stay at 3G forever which is a significant
waste of resources. If you put additional load on Jenkins the picture will look even worse.
Container limit - 8G, max heap limit (Xmx) - 6.5G and the memory consumption after the simple initial
sign in action goes to 3G.

Timely Reduce Unused Committed Memory (JEP 346)
Make the G1 garbage collector automatically give back Java heap memory
to the operating system when idle
● -XX:G1PeriodicGCInterval=[milliseconds]
● -XX:G1PeriodicGCSystemLoadThreshold=[ﬂoat]
● -XX:+G1PeriodicGCInvokesConcurrent
JEP 346: Promptly Return Unused Committed Memory from G1
java -Xmx2g -XX:+UseG1GC -XX:G1PeriodicGCInterval=900k
-XX:G1PeriodicGCSystemLoadThreshold=0.6 -jar app.jar
Available from Java 12

Immediately Improved Heap Elasticity
Automatically Released Heap

G1PeriodicGCSystemLoadThreshold in Docker Container
Using LXCFS to Improve Container Resource Visibility
For getting correct loadavg while running in Docker container use https://github.com/lxc/lxcfs/

Load Testing Logic
https://github.com/jelastic/java-vertical-scaling-test/blob
/master/src/com/jelastic/verticalscaling/Load.java#L50
java [OPTIONS] -jar app.jar <sleep>

Auto Testing Package
https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest.yml

G1 Collector (-XX:+UseG1GC)
The Garbage-First (G1) is a server-style Garbage Collector for
multiprocessor machines with a large amount of memory. The heap is
partitioned into ﬁxed-sized regions and G1 tracks the live data in those
regions. When Garbage Collection is required, it collects from the regions
with less live data ﬁrst.
● 2004, Sun Microsystems
JEP 346: Promptly Return Unused Committed Memory from G1

G1
-Xmx3g -XX:+UseCompressedOops -XX:+UseG1GC -XX:G1PeriodicGCInterval=1k

Shenandoah GC (-XX:+UseShenandoahGC)
Shenandoah GC is a concurrent garbage collector for the JVM. GC tries to
perform most of the activities in parallel without interrupting application
performance. Such parallelism makes “stop-the-world” (STW) pauses
extremely short. Another inherent advantage is an eﬃcient work with small
and large heaps with no impact on STW pauses’ length.
● 2014, Christine H. Flood, Red Hat
https://wiki.openjdk.java.net/display/shenandoah/Main#Main-Heuristics

-Xmx3g -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact
Shenandoah

ZGC (-XX:+UseZGC)
ZGC is low latency scalable garbage collector. Designed for use with
applications that require a large heap and low latency. It uses a bunch of one
generation and performs most (but not all) garbage collection in parallel with
uninterrupted application work. This greatly limits the impact of garbage
collection on your application response time.
● 2018, Per Liden, Oracle
JEP 351: ZGC: Uncommit Unused Memory - available from JDK 13 Release

-Xmx3g -XX:+UseZGC -XX:ZUncommitDelay=1 -XX:ZCollectionInterval=30
ZGC @ Oracle OpenJDK

C4 (-XX:+UseZST)
The C4 (Continuously Concurrent Compacting Collector) is an
updated generational form of the Azul Pauseless GC Algorithm and is
the default collector of Zing®. C4 diﬀerentiates itself from other
generational garbage collectors by supporting simultaneous –
generational concurrency: the diﬀerent generations are collected using
concurrent (non-stop-the-world) mechanisms that can be
simultaneously and independently active. Unlike other algorithms, it is
not ‘mostly’ concurrent, but fully concurrent, so it never falls back to a
stop-the-world compaction.
● 2010, Gil Tene, Azul Systems

-Xmx500m -XX:+UseZST
C4 @ Zing
pmem.conf -> cgroups enabled

ConcMarkSweep GC (-XX:+UseConcMarkSweepGC)
ConcMarkSweep GC collector is designed for applications that prefer
shorter garbage collection pauses and which can aﬀord to share processor
resources with the garbage collector while the application is running. It
makes sense to use such a collector when applications requirements for time
garbage collection pauses are low.

-Xmx3g -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
+ 4 calls of Full GC via jcmd <pid> GC.run
ConcMarkSweep (Deprecated since Java 9)
Consider using -XX:-ShrinkHeapInSteps

Serial GC (-XX:+UseSerialGC)
Serial GC performs garbage collection in a single thread and has the lowest
consumption of memory among all GC types but, at the same time, it makes
long pauses that can lead to application performance degradation.

-Xmx3g -XX:+UseCompressedOops -XX:+UseSerialGC
+ 4 calls of Full GC via jcmd <pid> GC.run
Serial
Consider using -XX:-ShrinkHeapInSteps

OpenJ9
OpenJ9 uses the Generational Concurrent (-Xgcpolicy:gencon) policy by
default, which is best suited to transactional applications that have many
short lived objects. Alternative policies are available, including those that
cater for applications with large Java heaps (-Xgcpolicy:balanced),
applications that are sensitive to response-time (-Xgcpolicy:metronome), or
applications that require high application throughput (-Xgcpolicy:optthruput).
● 2017, Eclipse Foundation

-Xmx3g -XX:+UseCompressedOops
-XX:+IdleTuningCompactOnIdle -XX:+IdleTuningGcOnIdle -XX:IdleTuningMinIdleWaitTime=1
-Xjit:waitTimeToEnterDeepIdleMode=1000
Bash command to check the real usage
while true
do
pid=$(pgrep -f java | tail -n1)
used=$(ps -orss --no-headers --pid $pid)
echo "scale=2 ; $used / 1024/1024" | bc
sleep 1
done
OpenJ9 (Not Enough Tuning and Testing Experience*)
Inconsistent behaviour with -XX:+IdleTuningGcOnIdle, mem not released back to OS on Idle

Parallel GC (-XX:+UseParallelGC)
Parallel GC is a “stop-the-world” multithreaded Garbage Collector similar to
the serial collector. The primary diﬀerence is that multiple threads are used
to speed up garbage collection. By default, both minor and major collections
are executed in parallel to further reduce garbage collection costs.

-Xmx3g -XX:+UseCompressedOops -XX:+UseParallelGC
+ periodical jcmd <pid> GC.run
Parallel

Main Points of Elastic Vertical Scaling
A - initial usage
B - maximum usage
C - growth speed
D - duration before release
E - release speed
F - minimum usage after release
Different GCs provide different results and fine tuning options

Running GC
Tests in Kubernetes

Auto Testing Package for Kubernetes
https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest-k8s.yaml

Load Testing Logic
java [OPTIONS] -jar app.jar <sleep> <mode>
where
sleep - 100
mode - 2
https://github.com/jelastic/java-vertical-scaling-test/blob/ma
ster/src/com/jelastic/verticalscaling/Load.java#L64

ZGC @ Oracle OpenJDK in Kubernetes

Joint Comparison - Several Load Cycles
RAM CPU

Running GC
Tests with Jakarta EE

● simple .war artifact deployed with GlassFish 5
● JSP that sets 1MB attribute in session
● 1 min session timeout
● https://github.com/jelastic/java-vertical-scaling-test/tree/payara/web
app
Load Testing Logic
Load test GETs webapp endpoint n times
for i in {1..n}; do curl -s localhost:8080 > /dev/null; done

GlassFish with Shenandoah
-Xmx3g -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact
for i in {1..1000}; do curl -s localhost:8080 > /dev/null; done

Dynamic Max Heap Resizing - Solving Right Sizing Problem
Restart for Xmx Resize

-XX:SoftMaxHeapSize @ ZGC
SoftMaxHeapSize is set for the GC to
strive not to grow heap size beyond the
speciﬁed size unless it is highly needed:
● to keep the heap footprint down, while
maintaining the capability to deal with a
temporary increase in heap space
requirement
● with lots of margin, to increase conﬁdence
that you will not run into an allocation stall
because of an unforeseen increase in
allocation rate
Using -XX:SoftMaxHeapSize

-XX:G1PeriodicGCInterval and -XX:SoftMaxHeapSize
G1PeriodicGCInterval (G1) / ZCollectionInterval (ZGC) is a time-based
solution which has no direct impact on how much the heap will grow during
the given interval.
SoftMaxHeapSize is a size-based solution which controls how large the heap
can grow, but has no direct relation with uncommit (returning unused
memory back to OS)
Both options can be used, whichever condition is met ﬁrst will trigger a GC.
ZGC (Java13+) https://bugs.openjdk.java.net/browse/JDK-8222181
G1, Shenandoah (TBD) https://bugs.openjdk.java.net/browse/JDK-8236073

-Xsoftmx @ OpenJ9
https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/openj9/xsoftmx/index.html
Runtime adjustable heap size (-Xsoftmx) allows to adjust heap size
dynamically and take advantage of hot-add of memory.
You can set this option on the command line, then modify it at run time by
using the com.ibm.lang.management.MemoryMXBean.setMaxHeapSize().
This option can be useful in virtualized or cloud environments, for example,
where the available memory might change dynamically to meet business
needs.
By default, -Xsoftmx is set to the same value as -Xmx.

C4 is fully elastic and can return all empty pages to the OS after each GC cycle.
However, C4 sticks to the Xmx it was given, and avoid doing heavy elastic memory dance,
since relinquishing memory mappings and reestablishing them on Linux kernels is
bandwidth-limited in practice by the rate of page mapping invalidation the kernel can
handle.
C4 goes above Xmx rather than go between Xms and Xmx. JavaMemMax option controls
the true maximum. In the future it will allow both scenarios where above-Xmx is allowed
and where above-Xmx is prohibited.
Two modes:
● Contingency (default mode) - goes above Xmx if it absolutely has to and will work
hard to collect and stay below Xmx.
● Insurance (best eﬀort elasticity) - borrows available memory and goes above Xmx in
order to delay GC whenever possible.
JavaMemMax @ С4 + ZST (Zing System Tools)

Xmx can be set higher than the Container Limit. Also both
SoftMaxHeapSize and Container Limit can be adjusted on the ﬂy
without the need to restart JVM or container.
At the moment the heap size can go beyond SoftMaxHeapSize and
there is no guarantee on how much the heap will grow other than up
to Xmx.
The problem arises when Used Heap comes close to Container Limit.
Most likely the JVM will be killed by the OOM Killer as it exceeds the
amount of memory available in the container.
JEP Draft: Dynamic Max Memory Limit @ G1
We can introduce manageable HardMaxHeapSize. In this case JVM will throw OOM Error (losing
one operation) instead of OOM Kill (losing whole JVM).
Xmx
Container Limit
SoftMaxHeapSize

JEP 387: Elastic Metaspace (Target Release Java 16)
To deal with potential problems involving virtual memory fragmentation or uncommit speed, we will add a new
production command-line option to control metaspace reclamation behavior:
-XX:MetaspaceReclaimPolicy=(balanced|aggressive|none)
■ balanced: Most applications should see an improvement in metaspace memory footprint while the
negative effects of memory reclamation should be marginal. This mode is the default, and aims for
backward compatibility.
■ 'aggressive': Offers increased memory-reclamation rates at the cost of increased virtual-memory
fragmentation.
■ 'none': Disables memory reclamation altogether.
Applications that use many small class loaders may suffer from
unreasonably high metaspace usage. Applications with heavy class
loading and unloading activity can thus accrue a lot of unused space in
the metaspace freelists. That space can be returned to the operating
system to be used for other purposes if it is not fragmented, but that’s
often not the case.

Keep Only Best Java Memories
Learn More
Get In Touch
@jelastic
info@jelastic.com

State of Java Elasticity. Tuning Java Efficiency - GIDS.JAVA LIVE 2020

More Related Content

What's hot (20)

Similar to State of Java Elasticity. Tuning Java Efficiency - GIDS.JAVA LIVE 2020 (20)

More from Jelastic Multi-Cloud PaaS (20)

Recently uploaded (20)

State of Java Elasticity. Tuning Java Efficiency - GIDS.JAVA LIVE 2020