Java-light-speed NebraskaCode.pdf
- 1. Java at light speed
- 2. 3 Who am I? • Software Engineer and Developer Advocate at IBM • Based in the Austin, TX USA • Focus on OSS cloud native Java technologies Rich Hagarty @rhagarty8
- 3. 4 Agenda • Java runtimes – cloud optimized? • Open Liberty • Java on cloud – a good fit? • JVM and JIT Compiler – good or bad? • Eclipse OpenJ9 JVM and Semeru Runtimes • Semeru Cloud Compiler • Liberty InstantOn
- 6. 8 Cloud Performance • Modular runtime o Low memory, high throughput o Low operating cost • CI/CD optimized o Availability on all Cloud platforms o Cloud-native tool integration o Optimized for Kubernetes • Simple developer experience o Integration with all popular IDEs and build tools o API support – Jakarta EE and MicroProfile
- 8. 11 6 reasons why Liberty Just enough runtime Low operating cost Continuous delivery Zero migration Kubernetes optimized 80% disk and 56% memory saving 4x increased density over Tomcat & Spring Boot Zero-effort security fixing & zero technical debt 100% v2v & fixpack migration saving Self-tuned optimal perf, production-ready, kube-native Developer experience Container & kube-native experience, rapid inner loop Lightweight, highly- efficient runtime CI/CD optimized operational experience Simple true-to- production developer experience
- 9. 12 <feature>servlet-4.0</feature> <feature>jaxrs-2.1</feature> Kernel • You control which features are loaded into each Liberty instance Just Enough Runtime jsonp-1.1 servlet- 4.0 http-2.0 appmgr Java EE jaxrs-2.1 jaxrsClien t-2.1
- 10. 13 13 Zero Migration üNo configuration behavior changes üNo runtime feature behavior changes üNo removals Never apply an ifix again Skipping a release does not introduce additional migration work Stay current with a rebuild (no app or config changes necessary)
- 11. 15 Developer experience Repositories Build IDEs Dev Mode APIs Testing
- 12. 16 Dev Mode No rebuild No redeploy No install No restart Just code!
- 13. Plan Code Build Test Release Deploy Run Monitor Cloud-native Liberty © 2022 IBM Corporation Testcontainers Zero Migration Continuous Delivery Quality Security An end-to-end cloud-native DevOps delivery experience with Liberty
- 14. 19 19 Java on Cloud Always a Good Fit?
- 15. Architectural style Functions Monolith Microservice Scale Smaller Larger Macroservice Lightweight Runtimes and Frameworks Full Function Runtimes Function as a Service Java runtimes face spectrum of application scale and styles
- 16. Time Responses per second Startup Steady state Time Memory Resident Set Time How much of application has runtime seen Varies with load Varies with load Varies with load Traditional focus of Java runtime technologies Traditional focus on steady state performance Rampup CPU consumption
- 17. 23 Architecture Styles vs Typical Java workloads Architecture Characteristics Compatibility Monoliths Long-lived, efficient Macroservices Long-lived Microservices Fast start-up/ramp-up for scaling up and down (to zero) to meet demand. Small footprint. Serverless/Functions Fast start-up (in MS), short-lived
- 18. 24 24 The Java Virtual Machine (JVM)
- 19. 25 What is the JVM? The Java Virtual Machine (JVM) is the runtime environment to execute Java code. The JVM is responsible for taking your application bytecode and converting it to a machines language for execution. Most other languages compile code for a specific system but in Java, the compiler converts the code for a Java Virtual Machine that can be used on any system hence the phrase “Write once, run anywhere”.
- 20. 26 How the JVM works Class Loader Bytecode Verifier Interpreter JIT Java Bytecode JVM Java Code Java Compiler javac Run Time Compile Time OS Hardware
- 21. 27 JVM Interpreter Class Loader Bytecode Verifier Interpreter JIT Java Bytecode JVM • Machine independent bytecodes are interpreted by the JVM at runtime • This ensures portability of Java programs across different architectures • But it affects performance because interpretation is relatively slow. OS Hardware
- 22. 28 Just-in-Time Compiler Class Loader Bytecode Verifier Interpreter JIT Java Bytecode JVM • The answer to the performance issue is the JIT compiler, which transforms sequences of bytecodes into machine code that is cached by the JVM • Unit of compilation is a method. To save overhead, only “hot” methods are compiled • Compiled native machine code executes 10x faster than a bytecode-by-bytecode interpreter • Generated code is saved in a "code cache" for future use for lifetime of JVM OS Hardware
- 23. 29 29 JVM on Cloud – issues?
- 24. 30 JVM – The Good 30 • Device independent – write once, run anywhere • > 25 years of improvements • JIT produces optimized machine code through use of Profilers • Efficient garbage collection • Longer it runs, the better it runs (JVM collects more profile data, JIT compiles more methods)
- 25. 31 JVM – The Bad 31 • Initial execution run is “interpreted”, which is relatively slow • “Hot Spot” methods compiled by JIT can create CPU and memory spikes • CPU spikes cause lower QoS • Memory spikes cause OOM issues, including crashes • Slow start-up time • Slow ramp-up time
- 26. 32 JVM Spikes 32 0 50 100 150 200 250 300 350 400 0 30 60 90 CPU utilization (%) Time (sec) Daytrader7 CPU consumption CPU spikes caused by JIT compilation 0 100000 200000 300000 400000 500000 600000 0 30 60 90 Resident set size (KB) Time (sec) Daytrader7 memory footprint Footprint spikes caused by JIT compilation
- 27. 33 Problem #1: Container Size 33 Main issues: •Need to over-provision to avoid OOM •Very hard to do – JVMs have a non-deterministic behavior 0 100000 200000 300000 400000 500000 600000 0 30 60 90 Resident set size (KB) Time (sec) Daytrader7 memory footprint Footprint spikes caused by JIT compilation
- 28. 34 #2: Container Instances (Auto-Scaling) 34 Main issues: •Slow start-up and ramp-up times •CPU spikes can cause auto-scaler to incorrectly launch additional instances
- 29. 35 Solution Requirements 35 •Minimize/eliminate CPU and memory spikes •Improve start-up and ramp-up times
- 30. 36 36 Eclipse OpenJ9 JVM IBM Semeru Runtimes
- 31. 37 Overview of Eclipse OpenJ9 Designed from the start to span all the operating systems needed by IBM products This JVM can go from small to large Can handle constrained environments or memory rich ones Renowned for its small footprint, fast start-up and ramp-up time Is used by the largest enterprises on the planet 37
- 32. 38 Eclipse OpenJ9 OpenJ9 combines with OpenJDK to form a full JDK Link to GitHub repo: https://github.com/eclipse-openj9/openj9 38
- 34. 40 IBM Semeru Runtimes “The part of Java that’s really in the clouds” IBM-built OpenJDK runtimes powered by the Eclipse OpenJ9 JVM No cost, stable, secure, high performance, cloud optimized, multi- platform, ready for development and production use Open Edition • Open source license (GPLv2+CE) • Available for Java 8 (LTS), 11 (LTS), 16, 17 (LTS), 18, 19, 20 Certified Edition • IBM license • Java SE TCK certified. • Available for Java 11, 17 40
- 35. 41 IBM Semeru Runtimes “The part of Java that’s really in the clouds” Other OpenJDK Distributions: • OpenJDK builds by Oracle • Adoptium Eclipse Temurin • Azul Zulu • BellSoft Liberica • Amazon Corretto • Alibaba Dragonwell • Microsoft Build of OpenJDK • Red Hat Build of OpenJDK • Oracle GraalVM 41
- 36. 42 42 Semeru Cloud Compiler For Microservices
- 37. 43 JIT-as-a-Service Decouple the JIT compiler from the JVM and let it run as an independent process Offload JIT compilation to remote process Remote JIT Remote JIT JVM JIT JVM JIT Kubernetes Control Plane Treat JIT compilation as a cloud service • Auto-managed by orchestrator • A mono-to-micro solution • Local JIT still available 43
- 38. 44 Eclipse OpenJ9 JITServer • JITServer feature is available in the Eclipse OpenJ9 JVM • “Semeru Cloud Compiler” when used with Semeru Runtimes 44
- 39. 45 JITServer advantages for JVM Clients 45 Provisioning Easier to size; only consider the needs of the application Performance Improved ramp-up time due to JITServer supplying extra CPU power when the JVM needs it the most. Reduced CPU consumption with JITServer AOT cache Cost Reduced memory consumption means increased application density and reduced operational cost. Efficient auto-scaling – only pay for what you need/use. Resiliency If the JITServer crashes, the JVM can continue to run and compile with its local JIT
- 40. 46 JITServer value in Kubernetes • https://blog.openj9.org/2021/10/20/save-money-with-jitserver-on-the-cloud-an-aws- experiment/ • Experimental test bed o ROSA (RedHat OpenShift Service on AWS) • Demonstrate that JITServer is not tied to IBM HW or SW o OCP cluster: 3 master nodes, 2 infra nodes, 3 worker nodes • Worker nodes have 8 vCPUs and 16 GB RAM (only ~12.3 GB available) o Four different applications • AcmeAir Microservices • AcmeAir Monolithic • Petclinic (Springboot framework) • Quarkus o Low amount of load to simulate conditions seen in practice o OpenShift Scheduler to manage pod and node deployments/placement 46
- 41. 47 JITServer improves container density and cost Default config AM 500 B 550 C 550 F 450 P 450 P 450 B 550 F 450 AM 500 A 350 AM 500 M 200 Q 350 P 450 Q 350 D 600 D 1000 F 450 B 550 Q 350 AM 500 AM 500 AM 500 B 550 B 550 A 350 C 550 F 450 M 200 P 450 P 450 P 450 Q 350 Q 350 D 1000 AM 500 B 550 P 450 AM 500 B 550 B 550 C 550 C 550 F 450 F 450 P 450 Q 350 Q 350 D 1000 D 1000 Q 350 AM 250 AM 250 P 250 P 250 F 250 F 250 B 400 C 350 Q 150 Q 150 M 150 AM 250 AM 250 P 250 P 250 F 250 B 400 Q 150 Q 150 J 1200 A 250 B 400 B 400 C 350 D 1000 D 1000 D 600 AM 250 AM 250 P 250 P 250 F 250 F 250 B 400 C 350 Q 150 Q 150 M 150 AM 250 AM 250 P 250 P 250 F 250 B 400 Q 150 Q 150 J 1200 A 250 B 400 B 400 C 350 D 1000 D 1000 JITServer config Legend: AM: AcmeAir monolithic A: Auth service B: Booking service C: Customer service D: Database (mongo/postgres) F: Flight service J: JITServer M: Main service P: Petclinic Q: Quarkus Total=8250 MB Total=8550 MB Total=8600 MB Total=9250 MB Total=9850 MB 6.3 GB less 47
- 42. 48 Throughput comparison 0 200 400 600 800 1000 1200 0 240 480 720 960 1200 1440 1680 1920 Throughput (pages/sec) Time (sec) AcmeAir Microservices 0 200 400 600 800 1000 1200 0 240 480 720 960 1200 1440 1680 1920 Throughput (pages/sec) Time (sec) AcmeAir Monolithic 0 20 40 60 80 100 120 140 160 180 0 240 480 720 960 1200 1440 1680 1920 Throughput (pages/sec) Time (sec) Petclinic 0 500 1000 1500 2000 2500 3000 3500 0 240 480 720 960 1200 1440 1680 1920 Throughput (pages/sec) Time (sec) Quarkus Machine load: 17.5% from apps 7% from OpenShift 48 è JITServer and default configuration achieve the same level of throughput at steady-state JITServer Baseline
- 43. 49 Conclusions from high density experiments • JITServer can improve container density and reduce operational costs of Java applications running in the cloud by 20-30% • Steady-state throughput is the same despite using fewer nodes 49
- 44. 50 Horizontal Pod Autoscaling in Kubernetes • Better autoscaling behavior with JITServer due to faster ramp-up • Less risk to trick the HPA due to transient JIT compilation overhead 50 Setup: Single node Microk8s cluster (16 vCPUs, 16 GB RAM) JVMs limited to 1 CPU, 500MB JITServer limited to 8 CPUs and has AOT cache enabled Load applied with JMeter, 100 threads, 10 ms think-time, 60s ramp-up time Autoscaler: scales up when average CPU utilization exceeds 0.5P. Up to 15 AcmeAir instances 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 0 60 120 180 240 300 360 420 480 Throughput (pages/sec) Time (sec) AcmeAir throughput when using Kubernetes autoscaling Baseline JITServer+AOTcache
- 45. 51 51 Demo
- 46. 52 Final thoughts • JIT provides advantage, but compilation adds overhead • Disaggregate JIT from JVM è JIT compilation as a service • Eclipse OpenJ9 JITServer (a.k.a Semeru Cloud Compiler) o Available now on Linux for Java 8, Java 11 and Java 17 (IBM Semeru Runtimes) o Especially good for constrained environments (micro-containers) o Kubernetes ready (Helm chart available, Prometheus integration, Certified OpenShift/K8s Operator) o Can improve ramp-up, autoscaling and performance of short lived applications o Can reduce peak memory footprint, increase app density and reduce costs o Competing solutions are proprietary (not open-source) and require licensing 52
- 47. 53 Link to Semeru Cloud Compiler (JITServer) articles and blogs: 53
- 49. 56 Is Native Image the answer? Static AOT compile to native image (GraalVM Native Image) 56 PROS CONS • Small footprint • Fast start-up • Great for short-lived apps • Limited JVM (static vs dynamic compiler optimization, limited GC) • Limited Java features (dynamic, APIs) • Moving target (frameworks, ecosystem, standards) • Dev/Prod parity (devs use compiler?)
- 50. What is Liberty InstantOn ? Liberty InstantOn is a holistic solution that provides fast startup without compromise in containers • Use checkpoint / restore - CRIU Project - https://criu.org - Linux technology used to freeze a running process state - Checkpoint state used to restore an application • Select point in startup to freeze an application - Restore application and run from the point where checkpoint was taken – Liberty InstantOn feature requires support for checkpoint / restore to be added to the JDK layer for Liberty to exploit – IBM Semeru JDK is used by default in Liberty containers – Semeru InstantOn : refers to the checkpoint / restore support in the IBM Semeru JDK – Collaborative effort between Java framework (Liberty) and JDK (Semeru) to create production- ready InstantOn feature that GAed in Liberty 23006 – Since this feature targets only container environments, you will need to download the relevant Liberty 23006 (or newer) container image to try out 57
- 51. 58 Linux CRIU capable of reducing startup time dramatically • ~10X reduction in first response time by using Liberty Instant On prototype using CRIU • Liberty Instant On is an effort to explore productization of CRIU use with Liberty • Main design goals • Usability : make it easy for an end user to consume the feature • Functionality : make it possible for end user to run their application without changes 58 0 1000 2000 3000 4000 5000 6000 Liberty baseline Liberty Instant On prototype PingPerf - First Response Time(ms) (lower is better) 0 1000 2000 3000 4000 5000 6000 Liberty baseline Liberty Instant On prototype Rest CRUD - First Response Time(ms) (lower is better) 0 1000 2000 3000 4000 5000 6000 Liberty baseline Liberty Instant On prototype Daytrader7 - First Response Time(ms) (lower is better) 1470 128 3082 213 5462 310 System Configuration: ------------------------------- SUT: LinTel – SLES 15 sp3 - Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz, 2 physical cores(4 cpu), 32GB RAM.
- 52. Liberty + Semeru InstantOn : fast startup using Linux CRIU Characteristics Semeru InstantOn Semeru JVM Graal Native Full Java support Yes Yes No ‘Instant on’ Yes No Yes High throughput Yes Yes No Low memory (under load) Yes Yes Yes? Dev-prod parity Yes Yes No Dev Build Prod Prod Prod checkpoint restore restore restore Target Liberty application container deployments Start application containers in milliseconds, ideal for serverless Leverages Linux CRIU to perform checkpoint / restore Make it really easy to consume for a user of Liberty containers 59
- 53. Liberty + Semeru InstantOn : easy to try existing apps Checkpoint and Restore in User Space Queries the OS for resources a process is using. At checkpoint it saves the state of the process tree to files. On restore it resumes the application by re- creating the process and re-run the system calls used to create the original state. Functionality is available in libcriu and exposed via native APIs. CRIU Project - https://criu.org Semeru JDK CRIU Liberty framework User application Uses the Semeru JDK’s CRIUSupport API to control CRIU Uses CRIU native APIs from within the JVM Does not need any awareness of CRIU Uses Linux features to checkpoint and restore 60
- 54. Liberty + Semeru InstantOn : easy to try existing apps Semeru InstantOn Semeru InstantOn is based on OpenJ9 CRIUSupport which is a Java API that offers access to Linux CRIU functionality, facilitates JVM compensations and provides tuning and serviceability capabilities. It is offered as a container image build solution on UBI8. Features • Time and CPU count compensations • Single thread hook support mechanism • Full Java security function after restore, only a small subset of security function before checkpoint keeps checkpoint application container image secure • Tracing and logging support on restore images • Add certain JVM options and env vars on restore Semeru JDK CRIU Liberty framework User application Uses the Semeru JDK’s CRIUSupport API to control CRIU Uses CRIU native APIs from within the JVM Does not need any awareness of CRIU Uses Linux features to checkpoint and restore 61
- 55. Liberty + Semeru InstantOn : easy to try existing apps Open Liberty InstantOn A framework that simplifies the use of checkpoint and restore by creating an abstraction which offers predetermined phases where a checkpoint can be taken and standard compensations (hooks) that an application will need. Makes use of OpenJ9 CRIUSupport to facilitate checkpoint and single thread run hooks. Features: • Easy startup guide • Container image build solution • Works on existing applications Semeru JDK CRIU Liberty framework User application Uses the Semeru JDK’s CRIUSupport API to control CRIU Uses CRIU native APIs from within the JVM Does not need any awareness of CRIU Uses Linux features to checkpoint and restore 62
- 56. Liberty + Semeru InstantOn : easy to try existing apps User applications on Liberty InstantOn Liberty InstantOn makes it easy for users to boost startup time. The user simply selects which phase they want to take a checkpoint in. No changes need to be made to the application. Semeru JDK CRIU Liberty framework User application Uses the Semeru JDK’s CRIUSupport API to control CRIU Uses CRIU native APIs from within the JVM Does not need any awareness of CRIU Uses Linux features to checkpoint and restore 63
- 57. InstantOn:Where to checkpoint? 64 Liberty InstantOn leverages Semeru to provide a seamless checkpoint/restore solution for developers. With checkpoint restore, there is a tradeoff between startup time and the complexity of the restore. Checkpoint phases • beforeAppStart • afterAppStart Choice of where to checkpoint depends on what application start code does, e.g., any of the below would make afterAppStart unsuitable • Accessing a remote resource, such as a database • Reading configuration that is expected to change when the application is deployed • Starting a transaction Kernel start and feature runtime processing Process application Start application Accept requests afterAppStart beforeAppStart Later checkpoint == Faster restore time Later checkpoint == More complexity 300 – 2200ms 100 – 3000ms 0 – ???ms
- 58. 67 67 Demo
- 59. Summary Adapting and thriving in the cloud requires evolving your development, deployment and runtime for a cloud-native world Liberty is a versatile runtime for these purposes InstantOn is a key capability that makes Liberty a great runtime for serverless use cases – https://openliberty.io – https://github.com/eclipse-openj9/openj9 – http://ibm.biz/InstantOn_HowToBlog – https://openliberty.io/blog/2023/06/29/rapid- startup-instanton.html – https://openliberty.io/docs/latest/instanton.html – https://blog.openj9.org/2022/09/26/getting-started- with-openj9-criu-support – https://blog.openj9.org/2022/10/14/openj9-criu- support-a-look-under-the-hood 68
- 60. 69 69 Wrap-up
- 61. 70 Architecture Styles vs Liberty Architecture Solution Compatibility Monoliths Liberty + Semeru Runtimes/OpenJ9 Macroservices Liberty + Semeru Runtimes/OpenJ9 Microservices Liberty + Semeru Runtimes/OpenJ9 + Semeru Cloud Compiler Serverless/Functions Liberty + Semeru Runtimes/OpenJ9 + InstantOn
- 62. 71
- 63. Open Liberty © 2021 IBM Corporation 72 Useful Links Why choose Liberty for Microservices https://ibm.biz/6ReasonsW hyLiberty Choosing the right Java runtime https://ibm.biz/ChooseJava Runtime How to approach application modernization https://ibm.biz/ModernizeJa vaApps Open Liberty Site https://www.openliberty.io Open Liberty Guides https://www.openliberty.io/ guides https://openliberty.io
- 64. Workloads: Java EE ⎼ WebSphere Application Server ⎼ Oracle WebLogic ⎼ Red Hat JBoss ⎼ Apache Tomcat Integration ⎼ IBM MQ ⎼ App Connect Enterprise (IIB) Open Source SDK for other extenders: https://github.com/IBM/transfo rmation-advisor-sdk IBM Cloud Transformation Advisor © 2021 IBM Corporation 73 Accelerates the modernization journey by quickly discovering and analyzing on-premise Java EE and/or messaging workloads in the enterprise to help in determining and executing the optimum modernization steps for each. Optimum modernization depends on workload needs! https://ibm.biz/6ReasonsWhyLiberty Traditional WebSphere network deployment Traditional WebSphere ‘base’ Liberty OpenShift OpenShift OpenShift Monolith Modernize Operations Modernize Runtime Modernize Architecture Containers, Java EE, monoliths & microservices…
- 65. 74 a free virtual consultation with an expert Liberty References Read Watch Explore the latest on WebSphere and Liberty ibm.biz/LibertyTV View our recent Expert TV episode all about Liberty https://ibm.biz/BdPn8Y Learn more about Liberty in containers and Operator-based deployment ibm.biz/LibertyContainerOperator Experience Try Liberty as a beginner openliberty.io/guides/getting- started.html Learn Liberty, MicroProfile, Containers, Kubernetes, Hands-on openliberty.io/guides Try dev mode in Containers https://openliberty.io/guides/doc ker.html Why choose Liberty for Microservices ibm.biz/6ReasonsWhyLiberty Choosing the right Java runtime ibm.biz/ChooseJavaRuntime How to approach application modernization ibm.biz/ModernizeJavaApps
- 66. 75 75 Thank You