Get Lower Latency and Higher Throughput for Java Applications
0 likes1,000 views
The document discusses performance challenges in Java applications related to latency, throughput, and warmup times, emphasizing issues such as garbage collection and deoptimisations. It introduces the Azul Platform Prime, an alternative JVM that enhances performance with features like the Continuous Concurrent Compacting Collector (C4) and the Falcon JIT compiler. The document also highlights real-world improvements and cost savings achieved by implementing these technologies in a cloud-based IT security company.
![Vector Code Example
■ Conditional array cell addition loop
● Hard for compiler to identify for vector instruction use
private void addArraysIfEven(int a[], int b[]) {
if (a.length != b.length)
throw new RuntimeException("length mismatch");
for (int i = 0; i < a.length; i++)
if ((b[i] & 0x1) == 0)
a[i] += b[i];
}](https://image.slidesharecdn.com/p99conf-simonritterv2-211003053816/85/Get-Lower-Latency-and-Higher-Throughput-for-Java-Applications-16-320.jpg)
Get Lower Latency and Higher Throughput for Java Applications
- 1. Brought to you by Get Lower Latency and Higher Throughput for Java Applications Simon Ritter Deputy CTO at
- 2. Simon Ritter Deputy CTO ■ Java Champion and two times JavaOne Rockstar ■ 99th Percentile is the hard part of performance ■ Away from work, my son and I are restoring a Classic Mini
- 3. JVM Performance Challenges ■ Latency ● Biggest issue is Garbage Collection ● Stop-the-world pauses for almost all collectors ● Pauses are typically proportional to heap size, not live data ■ Throughput ● Adaptive JIT compilation: Interpreted, C1 compiled, C2 compiled ● Deoptimisations ● Level of optimisation is key ■ Warmup ● Time taken to get to fully optimised code for all hot methods ● Restart of an application requires the same warmup work to be carried out
- 4. Azul Platform Prime: An Alternative JVM ■ Based on OpenJDK source code ■ Passes all Java SE TCK/JCK tests ● Drop-in replacement for other JVMs ● No application code changes, no recompilation ■ Hotspot collectors replaced with C4 ■ C2 JIT compiler replaced with Falcon ■ ReadyNow! warm up elimination technology
- 5. Azul Continuous Concurrent Compacting Collector (C4)
- 6. C4 Basics ■ Generational (young and old) ● Uses the same GC collector for both ● For efficiency rather than pause containment ■ All phases are parallel ■ No STW compacting fallback ● Heap scales from 512Mb to 12Tb (with no change to GC latency) ■ Algorithm is mark, relocate, remap ■ Only supported on Linux ● Sophisticated OS memory management interaction
- 7. Loaded Value Barrier ■ Read barrier ● Tests all object references as they are loaded ■ Enforces two invariants ● Reference is marked through ● Reference points to correct object position ■ Minimal performance overhead ● Test and jump (2 instructions) ● x86 architecture reduces this to one micro-op
- 8. Concurrent Mark Phase Root Set GC Threads App Threads X X X X X
- 9. Relocation Phase Compaction A B C D E A’ B’ C’ D’ E’ A -> A’ B -> B’ C -> C’ D -> D’ E -> E’
- 10. Remapping Phase App Threads GC Threads A -> A’ B -> B’ C -> C’ D -> D’ E -> E’ X X X
- 11. Measuring Platform Performance ■ jHiccup ■ Spends most of its time asleep ● Minimal effect on performance ● Wakes every 1 ms ● Records delta of time it expects to wake up ● Measured effect is what would be experienced by your application ■ Generates histogram log files ● These can be graphed for easy evaluation
- 12. Eliminating ElasticSearch Latency HotSpot Azul Prime 128Gb heap Prime:128GB: Prime:128GB:
- 13. Eliminating ElasticSearch Latency HotSpot Azul Prime 128Gb heap Prime:128GB: Prime:128GB:
- 14. Azul Falcon JIT Compiler
- 15. Advancing Adaptive Compilation ■ Replacement for C2 JIT compiler ■ Azul Falcon compiler ● Based on latest compiler research ● LLVM project ■ Better performance ● Better intrinsics ● More inlining ● Fewer compiler excludes
- 16. Vector Code Example ■ Conditional array cell addition loop ● Hard for compiler to identify for vector instruction use private void addArraysIfEven(int a[], int b[]) { if (a.length != b.length) throw new RuntimeException("length mismatch"); for (int i = 0; i < a.length; i++) if ((b[i] & 0x1) == 0) a[i] += b[i]; }
- 17. Traditional JVM JIT Per element jumps 2 elements per iteration
- 18. Falcon JIT Using AVX2 vector instructions 32 elements per iteration Broadwell E5-2690-v4
- 19. Recent Customer Success Story ■ Leading cloud-based IT security company ● Cloud security, compliance and other services ■ Big Kafka user ● 2.5 billion messages across Kafka clusters daily ● Initially approached us about their Cassandra clusters and eliminating latency ■ Kafka improvements ● 20% performance gain, out-of-the-box, with no tuning ● Falcon improved code generation ● Resulted in a 15% saving in cloud hardware costs ● Platform Core was effectively cheaper than free!
- 20. ReadyNow! Warmup Elimination Technology ■ Save JVM JIT profiling information ● Classes loaded ● Classes initialised ● Instruction profiling data ● Speculative optimisation failure data ■ Data can be gathered over much longer period ● JVM/JIT profiles quickly ● Significant reduction in deoptimisations ■ Able to load, initialise and compile most code before main()
- 22. Compile Stashing Effect Performance Time Performance Time Without Compile Stashing With Compile Stashing Up to 80% reduction in compile time and 60% reduction in CPU load
- 23. Summary
- 24. Improving Java Performance ■ Collect and re-use profiles to reduce warm-up time ■ Use alternative JIT compilation strategies ■ Eliminate GC STW pauses through use of read-barrier ■ Azul working to deliver better Java performance.
- 25. Brought to you by Simon Ritter [email protected] @speakjava