SlideShare a Scribd company logo

How to reduce cold starts for Java Serverless applications in AWS at JCON World 2023

V
Vadym Kazulkin

The document discusses strategies to reduce cold starts in Java serverless applications on AWS using GraalVM and SnapStart. It highlights the challenges posed by cold starts, best practices for optimizing startup time, and the benefits of using GraalVM native images and AWS SnapStart. It provides practical recommendations, including minimizing dependencies, using appropriate initialization techniques, and leveraging custom runtimes to enhance performance.

Technology
1 of 64
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
How to reduce cold starts for Java Serverless applications in AWS GraalVM, AWS SnapStart and Co Vadym Kazulkin, ip.labs, JCON Europe 22 November 2023
Contact Vadym Kazulkin ip.labs GmbH Bonn, Germany Co-Organizer of the Java User Group Bonn v.kazulkin@gmail.com @VKazulkin https://dev.to/vkazulkin https://github.com/Vadym79/ https://www.linkedin.com/in/vadymkazulkin https://www.iplabs.de/
ip.labs https://www.iplabs.de/
Java popularity
https://distantjob.com/blog/programming-languages-rank/ Vadym Kazulkin @VKazulkin , ip.labs GmbH
Life of the Java (Serverless) developer on AWS
AWS Java Versions Support • Corretto Java 8 • With extended long-term support until 2026 • Corretto Java 11 (since 2019) • Corretto Java 17 (April 2023) • Corretto Java 21(since November 17, 2023) • Only Long Term Support (LTS) by AWS Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://aws.amazon.com/de/corretto/
Java is very fast and mature programming language… Image: burst.shopify.com/photos/a-look-across-the-landscape-with-view-of-the-sea … but Serverless adoption of Java looks like this Vadym Kazulkin @VKazulkin , ip.labs GmbH
https://www.datadoghq.com/state-of-serverless-2021 https://www.datadoghq.com/state-of-serverless/ Vadym Kazulkin @VKazulkin , ip.labs GmbH Percent of AWS Lambda invocations by language 2021 vs 2023 2021 2023
Challenge Number 1 with Java is a big cold-start Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm
Function lifecycle- a full cold start Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
• Start Firecracker VM • AWS Lambda starts the JVM • Java runtime loads and initializes handler class • Static initializer block of the handler class is executed (i.e. AWS service client creation) • Init-phase has full CPU access up to 10 seconds for free for the managed execution environments • Lambda calls the handler method Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications Michael Hart: „Shave 99.93% off your Lambda bill with this one weird trick“ https://hichaelmart.medium.com/shave-99-93-off-your-lambda-bill-with-this-one-weird-trick-33c0acebb2ea
Lambda demo with common Java application frameworks Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples
Cold starts with pure Java Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500 3100 4000 Amazon Corretto Java 17 Quarkus 2532 3150 4100 Amazon Corretto Java 17 Micronaut 4480 4900 5450 Amazon Corretto Java 17 Spring Boot 5300 5710 6204
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples 0 1000 2000 3000 4000 5000 6000 Pure Java Cold starts p90 No Framework Quarkus Micronaut SpringBoot
Best Practices and Recommendations Using Tiered Compilation Achieve up to 60% faster startup times can use level 1 compilation with little risk of reducing warm start performance Mark Sailes: "Optimizing AWS Lambda function performance for Java” https://aws.amazon.com/de/blogs/compute/optimizing-aws-lambda-function-performance-for-java/
• Switch to the AWS SDK 2.0 for Java • Lower footprint and more modular • Allows to configure HTTP Client of your choice (i.e. Java own Basic HTTP Client or newly introduced AWS Common Runtime async HTTP Client) Source: https://aws.amazon.com/de/blogs/developer/announcing-availability-of-the-aws-crt-http-client-in-the-aws-sdk-for-java-2-x/ Vadym Kazulkin: https://dev.to/aws-builders/aws-sdk-for-java-2x-asynchronous-http-clients-and-their-impact-on-cold-start-times-and-memory-consumption-of-aws-lambda-366p S3AsyncClient.builder() .httpClientBuilder(AwsCrtAsyncHttpClient.builder() .maxConcurrency(50)) .build(); Best Practices and Recommendations
• Less (dependencies, classes) is more • Include only required dependencies (e.g. not the whole AWS SDK 2.0 for Java, but the dependencies to the clients to be used in Lambda) • Exclude dependencies, which you don‘t need at runtime e.g. test frameworks like Junit Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2 <dependency> <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter-api</artifactId> <version>5.4.2</version> <scope>test</scope> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>dynamodb</artifactId> <version>2.10.86</version> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>bom</artifactId> <version>2.10.86</version> <type>pom</type> <scope>import</scope> </dependency> Best Practices and Recommendations
Provide all known values (for building clients i.e. DynamoDB client) to avoid auto-discovery • credential provider, region, endpoint AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard() .withRegion(Regions.US_WEST_2) .withCredentials(new ProfileCredentialsProvider("myProfile")) .build(); Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ Best Practices and Recommendations
• Initialize dependencies during initialization phase • Use static initialization in the handler class, instead of in the handler method (e.g. handleRequest) to take the advantage of the access to the full CPU core for max 10 seconds • In case of DynamoDB client put the following code outside of the handler method: AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()...build(); DynamoDB dynamoDB = new DynamoDB(client); Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ Best Practices and Recommendations
Best Practices and Recommendations • Prime dependencies during initialization phase (when it worth doing) • „Fake“ the calls to pre-initalize „some other expensive stuff“ • In case of DynamoDB client put the following code outside of the handler method to pre- initialize the Jackson Marshaller: AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()...build(); DynamoDB dynamoDB = new DynamoDB(client); Table table = dynamoDB.getTable(„mytable"); Item item = table.getItem("Id", 0); Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ getItem() call forces Jackson Marshallers to initialize
Avoid: • reflection • runtime byte code generation • runtime generated proxies • dynamic class loading Use DI Frameworks which aren‘t reflection-based Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ Best Practices and Recommendations
GraalVM enters the scene Source: https://www.graalvm.org/
GraalVM Goals: Low footprint ahead-of-time mode for JVM-based languages High performance for all languages Convenient language interoperability and polyglot tooling Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
GraalVM Architecture Sources: Practical Partial Evaluation for High-Performance Dynamic Language Runtimes http://chrisseaton.com/rubytruffle/pldi17-truffle/pldi17-truffle.pdf „The LLVM Compiler Infrastructure“ https://llvm.org/
SubstrateVM Source: Oleg Šelajev, Thomas Wuerthinger, Oracle: “Deep dive into using GraalVM for Java and JavaScript” https://www.youtube.com/watch?v=a-XEZobXspo
GraalVM on SubstrateVM A game changer for Java & Serverless? Java Function compiled into a native executable using GraalVM on SubstrateVM reduces • “cold start” times • memory footprint by order of magnitude compared to running on JVM.
Prodviding GraalVM Native Image Environment • AWS doesn’t provide GraalVM (Native Image) as Java Runtime out of the box • AWS provides Custom Runtime Option
Custom Lambda Runtimes
Lambda Container Image Support
Amazon Linux 2023 runtime for AWS Lambda
Cold starts: Pure Java vs GraalVM Native Image Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433 3100/470 4000/531 Amazon Corretto Java 17 Quarkus 2532/467 3150/600 4100/802 Amazon Corretto Java 17 Micronaut 4480/638 4900/722 5450/961 Amazon Corretto Java 17 Spring Boot 5300/621 5710/685 6204/722
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image Cold starts p90 No Framework Quarkus Micronaut SpringBoot
Frameworks Ready for Graal VM Native Image Vadym Kazulkin @VKazulkin , ip.labs GmbH https://www.graalvm.org/native-image/libraries-and-frameworks/
Graal VM Conclusion • GraalVM and Frameworks are really powerful with a lot of potential • GraalVM Native Image improves cold starts and memory footprint significally • GraalVM Native Image is currently not without challenges • AWS Lambda Custom Runtime requires Linux executable only • Building Custom Runtime requires some additional effort • e.g. you need to scale CI pipeline to build memory-intensive native image yourself • Build time is a factor • You pay for the init-phase of the function packaged as AWS Lambda Custom and Docker Runtime • Init-phase is free for the managed runtimes like Java 8 , Java 11 and Java17 (Corretto)
Options to reduce cold start time • GraalVM (Native Image) • AWS SnapStart Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm
AWS SnapStart Vadym Kazulkin @VKazulkin , ip.labs GmbH
AWS SnapStart Deployment and Invocation Vadym Kazulkin @VKazulkin , ip.labs GmbH https://aws.amazon.com/de/blogs/compute/reducing-java-cold-starts-on-aws-lambda-functions-with-snapstart/ C Create Snapshot Firecracker microVM create & restore snapshot is based on CRIU
CRIU (Checkpoint/Restore in Userspace) • Linux CRIU available since 2012 allows a running application to be paused and restarted at some point later in time, potentially on a different machine. • The overall goal of the project is to support the migration of containers. • When performing a checkpoint, essentially, the full context of the process is saved: program counter, registers, stacks, memory-mapped and shared memory • To restore the application, all this data can be reloaded and (theoretically) it continues from the same point. • Challenges • open files • network connections • sudden change in the value of the system clock • time-based caches https://criu.org/Main_Page
AWS SnapStart Deployment and Invocation Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
AWS SnapStart Deployment Phase Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
AWS SnapStart Invocation Phase (before the end of September 2023) Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
AWS SnapStart Invocation Phase (after the end of September 2023) Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db
Cold starts: Pure Java vs GraalVM Native Image vs AWS SnapStart Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433/ 1514 3100/470/ 1649 4000/531/ 1860 Amazon Corretto Java 17 Quarkus 2532/467/ 1727 3150/600/ 1907 4100/802/ 2189 Amazon Corretto Java 17 Micronaut 4480/638/ 1852 4900/722/ 2125 5450/961/ 2401 Amazon Corretto Java 17 Spring Boot 5300/621/ 1920 5710/685/ 2321 6204/722/ 2538
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image AWS SnapStart Cold starts p90 No Framework Quarkus Micronaut SpringBoot
AWS SnapStart Deployment and Invocation Vadym Kazulkin @VKazulkin , ip.labs GmbH https://aws.amazon.com/de/blogs/compute/reducing-java-cold-starts-on-aws-lambda-functions-with-snapstart/ Lambda uses the CRaC APIs for runtime hooks C Create Snapshot Firecracker microVM create & restore snapshot is based on CRIU
• Speed up warmup time of the Java applications • The C2 compiler is used for very hot methods, which uses profiling data collected from the running application to optimize as much as possible. • Techniques like aggressive method inlining and speculative optimizations can easily lead to better performing code than generated ahead of time (AOT) using a static compiler. • JVM needs both time and compute resources to determine which methods to compile and compiling them. This same work has to happen every time we run an application • Ideally, we would like to run the application and then store all the state about the compiled methods, even the compiled code and state associated with the application and then we’d like to be able to restore it https://www.azul.com/blog/superfast-application-startup-java-on-crac/ https://github.com/CRaC/docs Ideas behind CRaC (Coordinated Restore at Checkpoint)
AWS SnapStart enabled with Pure Java Priming <groupId>io.github.crac</groupId> <artifactId>org-crac</artifactId> <version>0.1.3</version> Source: Vadym Kazulkin https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-5-priming-end-to-end-latency-and-deployment-time-jem
AWS SnapStart enabled with Micronaut Priming Source: Vadym Kazulkin https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-5-priming-end-to-end-latency-and-deployment-time-jem
Cold starts: Pure Java vs GraalVM Native Image vs SnapStart vs SnapStart with Priming Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433/ 1514/726 3100/470/ 1649/829 4000/531/ 1860/1067 Amazon Corretto Java 17 Quarkus 4000/467/ 1727/872 3150/600/ 1907/996 4100/802/ 2189/1090 Amazon Corretto Java 17 Micronaut 4480/638/ 1852/1007 4900/722/ 2125/1232 5450/961/ 2401/1332 Amazon Corretto Java 17 Spring Boot 5300/621/ 1920/1024 5710/685/ 2321/1525 6204/722/ 2538/1850
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image AWS SnapStart AWS SnapStart with invoke DynamoDB priming Cold starts p90 No Framework Quarkus Micronaut SpringBoot
AWS SnapStart with Quarkus extended Priming Source: Vadym Kazulkin https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-6-priming-the-request-invocation-30od
Cold starts: Pure Java vs GraalVM Native Image vs ASS vs ASS w. Priming vs ASS w ext. Priming Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433/ 1514/726/726 3100/470/ 1649/829/829 4000/531/ 1860/1067/1067 Amazon Corretto Java 17 Quarkus 4000/467/ 1727/872/827 3150/600/ 1907/996/961 4100/802/ 2189/1090/1080 Amazon Corretto Java 17 Micronaut 4250/638/ 1852/1007/935 4900/722/ 2125/1232/1012 5450/961/ 2401/1332/1102 Amazon Corretto Java 17 Spring Boot 5300/621/ 1920/1024/844 5750/685/ 2321/1525/1043 6204/722/ 2538/1850/1290
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image AWS SnapStartAWS SnapStart with invoke DynamoDB priming AWS SnapStart with framework web model priming Cold starts p90 No Framework Quarkus Micronaut SpringBoot
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 500 1000 1500 2000 2500 3000 3500 4000 4500 Pure Java GraalVM Native Image AWS SnapStartAWS SnapStart with invoke DynamoDB priming Cold starts of Lambda function with Java 17 runtime p50 p90 p99
Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm Lambda support for Java 21
Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 500 1000 1500 2000 2500 3000 3500 4000 4500 Pure Java 17 Pure Java 21 AWS SnapStart 17 AWS SnapStart 21 AWS SnapStart with invoke DynamoDB priming 17 AWS SnapStart with invoke DynamoDB priming 21 Cold starts of Lambda function with Java 17 vs 21 runtime p50 p90 p99
AWS SnapStart Challenges & Limitations • SnapStart supports the Java 11, 17 and 21 (all Amazon Corretto) managed runtime only • Deployment with SnapStart enabled takes more than 2-2,5 minutes additionally • Snapshot is deleted from cache if Lambda function is not invoked for 14 days • SnapStart currently does not support : • Provisioned concurrency • arm64 architecture (supports only x86) • Amazon Elastic File System (Amazon EFS) • Ephemeral storage greater than 512 MB Vadym Kazulkin @VKazulkin , ip.labs GmbH https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html
AWS SnapStart Possible Next Steps • Perform Priming out of the box without writing the logic on our own • Snapshot creation on first Lambda function invocation instead of during the deployment phase • Regular cold start as long as the snapshot hasn’t been fully taken • Trade off between duration of the deployment phase and several regular bigger cold starts until the snapshot is taken and SnapStart becomes effective • If snapshot not found after 14 days (due to the Lambda not being invoked), do regular cold start as long as new snapshot has been fully taken under the hood Vadym Kazulkin @VKazulkin , ip.labs GmbH
Lambda Provisioned Concurrency https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ • Requires manually managing start and end time when Provisioned concurrency should apply (can be tricky for spikey workloads) • You pay for unused capacity
NEW: Lambda Proactive initialization In June 2023 AWS updated the documentation for the Lambda Function lifecycle and included this new statement: for functions using unreserved (on-demand) concurrency, Lambda may proactively initialize a function instance, even if there's no invocation. When this happens, you can observe an unexpected time gap between your function's initialization and invocation phases. This gap can appear similar to what you would observe when using provisioned concurrency. https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html https://aaronstuyvenberg.com/posts/understanding-proactive-initialization Running this query over several days across multiple runtimes and invocation methods, between 50% and 75% of initializations were proactive (versus 50% to 25% which were true cold starts)
Project Leyden https://www.youtube.com/watch?v=lnth19Kf-x0 https://openjdk.org/projects/leyden/ The primary goal of this Project is to improve the startup time, time to peak performance, and footprint of Java programs.
How to reduce cold starts for Java Serverless applications in AWS at JCON World 2023
www.iplabs.de Accelerate Your Photo Business Get in Touch

More Related Content

Similar to How to reduce cold starts for Java Serverless applications in AWS at JCON World 2023 (20)

PPTX
Adopting Java for the Serverless World at JAX 2022
Vadym Kazulkin
 
PDF
Adapting Java for the Serverless World at JUG Barcelona
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS- JavaDays Lviv 2024
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless World at VoxxedDays Luxemburg
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS at Froscon 2024
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless World at JUG Darmstadt 2022
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless World at JUG Bonn 2022
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS- AWS Community Day Budapest 2024
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS at We Are Developers 2024
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS- Serverless Architecture Conference B...
Vadym Kazulkin
 
PDF
AWS Lambda SnapStart: Why, How and What AWS Serverless Meetup New York Boston...
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS- Serverless Architecture Javaland 2025
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless world at JUG London
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at I...
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless world at JUG Hamburg
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS- Serverless Meetup Toronto
Vadym Kazulkin
 
PDF
High performance Serverless Java on AWS at AWS Community Day Belfast 2024
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless world at AWS User Group Pretoria
Vadym Kazulkin
 
PDF
Adopting Java for the Serverless world at Serverless Meetup Singapore
Vadym Kazulkin
 
Adopting Java for the Serverless World at JAX 2022
Vadym Kazulkin
 
Adapting Java for the Serverless World at JUG Barcelona
Vadym Kazulkin
 
High performance Serverless Java on AWS- JavaDays Lviv 2024
Vadym Kazulkin
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
Adopting Java for the Serverless World at VoxxedDays Luxemburg
Vadym Kazulkin
 
High performance Serverless Java on AWS at Froscon 2024
Vadym Kazulkin
 
Adopting Java for the Serverless World at JUG Darmstadt 2022
Vadym Kazulkin
 
Adopting Java for the Serverless World at JUG Bonn 2022
Vadym Kazulkin
 
High performance Serverless Java on AWS- AWS Community Day Budapest 2024
Vadym Kazulkin
 
High performance Serverless Java on AWS at We Are Developers 2024
Vadym Kazulkin
 
High performance Serverless Java on AWS- Serverless Architecture Conference B...
Vadym Kazulkin
 
AWS Lambda SnapStart: Why, How and What AWS Serverless Meetup New York Boston...
Vadym Kazulkin
 
High performance Serverless Java on AWS- Serverless Architecture Javaland 2025
Vadym Kazulkin
 
Adopting Java for the Serverless world at JUG London
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at I...
Vadym Kazulkin
 
Adopting Java for the Serverless world at JUG Hamburg
Vadym Kazulkin
 
High performance Serverless Java on AWS- Serverless Meetup Toronto
Vadym Kazulkin
 
High performance Serverless Java on AWS at AWS Community Day Belfast 2024
Vadym Kazulkin
 
Adopting Java for the Serverless world at AWS User Group Pretoria
Vadym Kazulkin
 
Adopting Java for the Serverless world at Serverless Meetup Singapore
Vadym Kazulkin
 

More from Vadym Kazulkin (20)

PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda - Wa...
Vadym Kazulkin
 
PDF
Event-driven architecture patterns in highly scalable image storage solution-...
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda-OBI ...
Vadym Kazulkin
 
PPTX
Making sense of AWS Serverless operations- AWS User Group Nuremberg
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at V...
Vadym Kazulkin
 
PPTX
Making sense of AWS Serverless operations at Believe in Serverless community ...
Vadym Kazulkin
 
PDF
Making sense of AWS Serverless operations - Amarathon Geek China 2024
Vadym Kazulkin
 
PDF
Event-driven architecture patterns in highly scalable image storage solution-...
Vadym Kazulkin
 
PDF
Making sense of AWS Serverless operations- Serverless Architecture Conference...
Vadym Kazulkin
 
PDF
Detect operational anomalies in Serverless Applications with Amazon DevOps Gu...
Vadym Kazulkin
 
PDF
Detect operational anomalies in Serverless Applications with Amazon DevOps Gu...
Vadym Kazulkin
 
PDF
Making sense of AWS Serverless operations AWS Community Day NL 2024-
Vadym Kazulkin
 
PDF
Event-driven architecture patterns in highly scalable image storage solution ...
Vadym Kazulkin
 
PDF
Detect operational anomalies in Serverless Applications with Amazon DevOps Gu...
Vadym Kazulkin
 
PDF
Amazon DevOps Guru for Serverless Applications at JAWS Pankration 2024
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at J...
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at A...
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at J...
Vadym Kazulkin
 
PDF
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at ...
Vadym Kazulkin
 
PDF
Developing highly scalable image storage solution with AWS Serverless at GoTo...
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda - Wa...
Vadym Kazulkin
 
Event-driven architecture patterns in highly scalable image storage solution-...
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda-OBI ...
Vadym Kazulkin
 
Making sense of AWS Serverless operations- AWS User Group Nuremberg
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at V...
Vadym Kazulkin
 
Making sense of AWS Serverless operations at Believe in Serverless community ...
Vadym Kazulkin
 
Making sense of AWS Serverless operations - Amarathon Geek China 2024
Vadym Kazulkin
 
Event-driven architecture patterns in highly scalable image storage solution-...
Vadym Kazulkin
 
Making sense of AWS Serverless operations- Serverless Architecture Conference...
Vadym Kazulkin
 
Detect operational anomalies in Serverless Applications with Amazon DevOps Gu...
Vadym Kazulkin
 
Detect operational anomalies in Serverless Applications with Amazon DevOps Gu...
Vadym Kazulkin
 
Making sense of AWS Serverless operations AWS Community Day NL 2024-
Vadym Kazulkin
 
Event-driven architecture patterns in highly scalable image storage solution ...
Vadym Kazulkin
 
Detect operational anomalies in Serverless Applications with Amazon DevOps Gu...
Vadym Kazulkin
 
Amazon DevOps Guru for Serverless Applications at JAWS Pankration 2024
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at J...
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at A...
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at J...
Vadym Kazulkin
 
How to develop, run and optimize Spring Boot 3 application on AWS Lambda at ...
Vadym Kazulkin
 
Developing highly scalable image storage solution with AWS Serverless at GoTo...
Vadym Kazulkin
 
Ad

Recently uploaded (20)

PDF
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
PDF
Empowering Cloud Providers with Apache CloudStack and Stackbill
ShapeBlue
 
PPTX
Building a Production-Ready Barts Health Secure Data Environment Tooling, Acc...
Barts Health
 
PDF
Wojciech Ciemski for Top Cyber News MAGAZINE. June 2025
Dr. Ludmila Morozova-Buss
 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
PDF
CloudStack GPU Integration - Rohit Yadav
ShapeBlue
 
PPTX
UiPath Academic Alliance Educator Panels: Session 2 - Business Analyst Content
DianaGray10
 
PDF
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
PDF
Impact of IEEE Computer Society in Advancing Emerging Technologies including ...
Hironori Washizaki
 
PDF
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
PDF
Meetup Kickoff & Welcome - Rohit Yadav, CSIUG Chairman
ShapeBlue
 
PDF
CIFDAQ Token Spotlight for 9th July 2025
CIFDAQ
 
PDF
Why Orbit Edge Tech is a Top Next JS Development Company in 2025
mahendraalaska08
 
PPTX
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
PDF
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
PDF
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
PDF
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
PDF
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
PPTX
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
PDF
Français Patch Tuesday - Juillet
Ivanti
 
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
Empowering Cloud Providers with Apache CloudStack and Stackbill
ShapeBlue
 
Building a Production-Ready Barts Health Secure Data Environment Tooling, Acc...
Barts Health
 
Wojciech Ciemski for Top Cyber News MAGAZINE. June 2025
Dr. Ludmila Morozova-Buss
 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
CloudStack GPU Integration - Rohit Yadav
ShapeBlue
 
UiPath Academic Alliance Educator Panels: Session 2 - Business Analyst Content
DianaGray10
 
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
Impact of IEEE Computer Society in Advancing Emerging Technologies including ...
Hironori Washizaki
 
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
Meetup Kickoff & Welcome - Rohit Yadav, CSIUG Chairman
ShapeBlue
 
CIFDAQ Token Spotlight for 9th July 2025
CIFDAQ
 
Why Orbit Edge Tech is a Top Next JS Development Company in 2025
mahendraalaska08
 
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
Français Patch Tuesday - Juillet
Ivanti
 
Ad

How to reduce cold starts for Java Serverless applications in AWS at JCON World 2023

  • 1. How to reduce cold starts for Java Serverless applications in AWS GraalVM, AWS SnapStart and Co Vadym Kazulkin, ip.labs, JCON Europe 22 November 2023
  • 2. Contact Vadym Kazulkin ip.labs GmbH Bonn, Germany Co-Organizer of the Java User Group Bonn [email protected] @VKazulkin https://dev.to/vkazulkin https://github.com/Vadym79/ https://www.linkedin.com/in/vadymkazulkin https://www.iplabs.de/
  • 6. Life of the Java (Serverless) developer on AWS
  • 7. AWS Java Versions Support • Corretto Java 8 • With extended long-term support until 2026 • Corretto Java 11 (since 2019) • Corretto Java 17 (April 2023) • Corretto Java 21(since November 17, 2023) • Only Long Term Support (LTS) by AWS Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://aws.amazon.com/de/corretto/
  • 8. Java is very fast and mature programming language… Image: burst.shopify.com/photos/a-look-across-the-landscape-with-view-of-the-sea … but Serverless adoption of Java looks like this Vadym Kazulkin @VKazulkin , ip.labs GmbH
  • 10. Challenge Number 1 with Java is a big cold-start Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm
  • 11. Function lifecycle- a full cold start Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications
  • 12. • Start Firecracker VM • AWS Lambda starts the JVM • Java runtime loads and initializes handler class • Static initializer block of the handler class is executed (i.e. AWS service client creation) • Init-phase has full CPU access up to 10 seconds for free for the managed execution environments • Lambda calls the handler method Sources: Ajay Nair „Become a Serverless Black Belt” https://www.youtube.com/watch?v=oQFORsso2go Tomasz Łakomy "Notes from Optimizing Lambda Performance for Your Serverless Applications“ https://tlakomy.com/optimizing-lambda-performance-for-serverless-applications Michael Hart: „Shave 99.93% off your Lambda bill with this one weird trick“ https://hichaelmart.medium.com/shave-99-93-off-your-lambda-bill-with-this-one-weird-trick-33c0acebb2ea
  • 13. Lambda demo with common Java application frameworks Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples
  • 14. Cold starts with pure Java Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500 3100 4000 Amazon Corretto Java 17 Quarkus 2532 3150 4100 Amazon Corretto Java 17 Micronaut 4480 4900 5450 Amazon Corretto Java 17 Spring Boot 5300 5710 6204
  • 15. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples 0 1000 2000 3000 4000 5000 6000 Pure Java Cold starts p90 No Framework Quarkus Micronaut SpringBoot
  • 16. Best Practices and Recommendations Using Tiered Compilation Achieve up to 60% faster startup times can use level 1 compilation with little risk of reducing warm start performance Mark Sailes: "Optimizing AWS Lambda function performance for Java” https://aws.amazon.com/de/blogs/compute/optimizing-aws-lambda-function-performance-for-java/
  • 17. • Switch to the AWS SDK 2.0 for Java • Lower footprint and more modular • Allows to configure HTTP Client of your choice (i.e. Java own Basic HTTP Client or newly introduced AWS Common Runtime async HTTP Client) Source: https://aws.amazon.com/de/blogs/developer/announcing-availability-of-the-aws-crt-http-client-in-the-aws-sdk-for-java-2-x/ Vadym Kazulkin: https://dev.to/aws-builders/aws-sdk-for-java-2x-asynchronous-http-clients-and-their-impact-on-cold-start-times-and-memory-consumption-of-aws-lambda-366p S3AsyncClient.builder() .httpClientBuilder(AwsCrtAsyncHttpClient.builder() .maxConcurrency(50)) .build(); Best Practices and Recommendations
  • 18. • Less (dependencies, classes) is more • Include only required dependencies (e.g. not the whole AWS SDK 2.0 for Java, but the dependencies to the clients to be used in Lambda) • Exclude dependencies, which you don‘t need at runtime e.g. test frameworks like Junit Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2 <dependency> <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter-api</artifactId> <version>5.4.2</version> <scope>test</scope> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>dynamodb</artifactId> <version>2.10.86</version> </dependency> <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>bom</artifactId> <version>2.10.86</version> <type>pom</type> <scope>import</scope> </dependency> Best Practices and Recommendations
  • 19. Provide all known values (for building clients i.e. DynamoDB client) to avoid auto-discovery • credential provider, region, endpoint AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard() .withRegion(Regions.US_WEST_2) .withCredentials(new ProfileCredentialsProvider("myProfile")) .build(); Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ Best Practices and Recommendations
  • 20. • Initialize dependencies during initialization phase • Use static initialization in the handler class, instead of in the handler method (e.g. handleRequest) to take the advantage of the access to the full CPU core for max 10 seconds • In case of DynamoDB client put the following code outside of the handler method: AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()...build(); DynamoDB dynamoDB = new DynamoDB(client); Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ Best Practices and Recommendations
  • 21. Best Practices and Recommendations • Prime dependencies during initialization phase (when it worth doing) • „Fake“ the calls to pre-initalize „some other expensive stuff“ • In case of DynamoDB client put the following code outside of the handler method to pre- initialize the Jackson Marshaller: AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()...build(); DynamoDB dynamoDB = new DynamoDB(client); Table table = dynamoDB.getTable(„mytable"); Item item = table.getItem("Id", 0); Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ getItem() call forces Jackson Marshallers to initialize
  • 22. Avoid: • reflection • runtime byte code generation • runtime generated proxies • dynamic class loading Use DI Frameworks which aren‘t reflection-based Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: Stefano Buliani : "Best practices for AWS Lambda and Java„ https://www.youtube.com/watch?v=ddg1u5HLwg8 Sean O‘Toole „AWS Lambda Java Tutorial: Best Practices to Lower Cold Starts” https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/ Best Practices and Recommendations
  • 23. GraalVM enters the scene Source: https://www.graalvm.org/
  • 24. GraalVM Goals: Low footprint ahead-of-time mode for JVM-based languages High performance for all languages Convenient language interoperability and polyglot tooling Source: „Everything you need to know about GraalVM by Oleg Šelajev & Thomas Wuerthinger” https://www.youtube.com/watch?v=ANN9rxYo5Hg
  • 25. GraalVM Architecture Sources: Practical Partial Evaluation for High-Performance Dynamic Language Runtimes http://chrisseaton.com/rubytruffle/pldi17-truffle/pldi17-truffle.pdf „The LLVM Compiler Infrastructure“ https://llvm.org/
  • 26. SubstrateVM Source: Oleg Šelajev, Thomas Wuerthinger, Oracle: “Deep dive into using GraalVM for Java and JavaScript” https://www.youtube.com/watch?v=a-XEZobXspo
  • 27. GraalVM on SubstrateVM A game changer for Java & Serverless? Java Function compiled into a native executable using GraalVM on SubstrateVM reduces • “cold start” times • memory footprint by order of magnitude compared to running on JVM.
  • 28. Prodviding GraalVM Native Image Environment • AWS doesn’t provide GraalVM (Native Image) as Java Runtime out of the box • AWS provides Custom Runtime Option
  • 31. Amazon Linux 2023 runtime for AWS Lambda
  • 32. Cold starts: Pure Java vs GraalVM Native Image Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433 3100/470 4000/531 Amazon Corretto Java 17 Quarkus 2532/467 3150/600 4100/802 Amazon Corretto Java 17 Micronaut 4480/638 4900/722 5450/961 Amazon Corretto Java 17 Spring Boot 5300/621 5710/685 6204/722
  • 33. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://github.com/aws-samples/serverless-java-frameworks-samples 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image Cold starts p90 No Framework Quarkus Micronaut SpringBoot
  • 34. Frameworks Ready for Graal VM Native Image Vadym Kazulkin @VKazulkin , ip.labs GmbH https://www.graalvm.org/native-image/libraries-and-frameworks/
  • 35. Graal VM Conclusion • GraalVM and Frameworks are really powerful with a lot of potential • GraalVM Native Image improves cold starts and memory footprint significally • GraalVM Native Image is currently not without challenges • AWS Lambda Custom Runtime requires Linux executable only • Building Custom Runtime requires some additional effort • e.g. you need to scale CI pipeline to build memory-intensive native image yourself • Build time is a factor • You pay for the init-phase of the function packaged as AWS Lambda Custom and Docker Runtime • Init-phase is free for the managed runtimes like Java 8 , Java 11 and Java17 (Corretto)
  • 36. Options to reduce cold start time • GraalVM (Native Image) • AWS SnapStart Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm
  • 37. AWS SnapStart Vadym Kazulkin @VKazulkin , ip.labs GmbH
  • 38. AWS SnapStart Deployment and Invocation Vadym Kazulkin @VKazulkin , ip.labs GmbH https://aws.amazon.com/de/blogs/compute/reducing-java-cold-starts-on-aws-lambda-functions-with-snapstart/ C Create Snapshot Firecracker microVM create & restore snapshot is based on CRIU
  • 39. CRIU (Checkpoint/Restore in Userspace) • Linux CRIU available since 2012 allows a running application to be paused and restarted at some point later in time, potentially on a different machine. • The overall goal of the project is to support the migration of containers. • When performing a checkpoint, essentially, the full context of the process is saved: program counter, registers, stacks, memory-mapped and shared memory • To restore the application, all this data can be reloaded and (theoretically) it continues from the same point. • Challenges • open files • network connections • sudden change in the value of the system clock • time-based caches https://criu.org/Main_Page
  • 40. AWS SnapStart Deployment and Invocation Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
  • 41. AWS SnapStart Deployment Phase Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
  • 42. AWS SnapStart Invocation Phase (before the end of September 2023) Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/vkazulkin/measuring-java-11-lambda-cold-starts-with-snapstart-part-1-first-impressions-30a4
  • 43. AWS SnapStart Invocation Phase (after the end of September 2023) Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db
  • 44. Cold starts: Pure Java vs GraalVM Native Image vs AWS SnapStart Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433/ 1514 3100/470/ 1649 4000/531/ 1860 Amazon Corretto Java 17 Quarkus 2532/467/ 1727 3150/600/ 1907 4100/802/ 2189 Amazon Corretto Java 17 Micronaut 4480/638/ 1852 4900/722/ 2125 5450/961/ 2401 Amazon Corretto Java 17 Spring Boot 5300/621/ 1920 5710/685/ 2321 6204/722/ 2538
  • 45. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image AWS SnapStart Cold starts p90 No Framework Quarkus Micronaut SpringBoot
  • 46. AWS SnapStart Deployment and Invocation Vadym Kazulkin @VKazulkin , ip.labs GmbH https://aws.amazon.com/de/blogs/compute/reducing-java-cold-starts-on-aws-lambda-functions-with-snapstart/ Lambda uses the CRaC APIs for runtime hooks C Create Snapshot Firecracker microVM create & restore snapshot is based on CRIU
  • 47. • Speed up warmup time of the Java applications • The C2 compiler is used for very hot methods, which uses profiling data collected from the running application to optimize as much as possible. • Techniques like aggressive method inlining and speculative optimizations can easily lead to better performing code than generated ahead of time (AOT) using a static compiler. • JVM needs both time and compute resources to determine which methods to compile and compiling them. This same work has to happen every time we run an application • Ideally, we would like to run the application and then store all the state about the compiled methods, even the compiled code and state associated with the application and then we’d like to be able to restore it https://www.azul.com/blog/superfast-application-startup-java-on-crac/ https://github.com/CRaC/docs Ideas behind CRaC (Coordinated Restore at Checkpoint)
  • 48. AWS SnapStart enabled with Pure Java Priming <groupId>io.github.crac</groupId> <artifactId>org-crac</artifactId> <version>0.1.3</version> Source: Vadym Kazulkin https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-5-priming-end-to-end-latency-and-deployment-time-jem
  • 49. AWS SnapStart enabled with Micronaut Priming Source: Vadym Kazulkin https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-5-priming-end-to-end-latency-and-deployment-time-jem
  • 50. Cold starts: Pure Java vs GraalVM Native Image vs SnapStart vs SnapStart with Priming Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433/ 1514/726 3100/470/ 1649/829 4000/531/ 1860/1067 Amazon Corretto Java 17 Quarkus 4000/467/ 1727/872 3150/600/ 1907/996 4100/802/ 2189/1090 Amazon Corretto Java 17 Micronaut 4480/638/ 1852/1007 4900/722/ 2125/1232 5450/961/ 2401/1332 Amazon Corretto Java 17 Spring Boot 5300/621/ 1920/1024 5710/685/ 2321/1525 6204/722/ 2538/1850
  • 51. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image AWS SnapStart AWS SnapStart with invoke DynamoDB priming Cold starts p90 No Framework Quarkus Micronaut SpringBoot
  • 52. AWS SnapStart with Quarkus extended Priming Source: Vadym Kazulkin https://dev.to/aws-builders/measuring-java-11-lambda-cold-starts-with-snapstart-part-6-priming-the-request-invocation-30od
  • 53. Cold starts: Pure Java vs GraalVM Native Image vs ASS vs ASS w. Priming vs ASS w ext. Priming Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db Runtime/Optimization Frame- work used p50 ms p90 ms p99 ms Amazon Corretto Java 17 No 2500/433/ 1514/726/726 3100/470/ 1649/829/829 4000/531/ 1860/1067/1067 Amazon Corretto Java 17 Quarkus 4000/467/ 1727/872/827 3150/600/ 1907/996/961 4100/802/ 2189/1090/1080 Amazon Corretto Java 17 Micronaut 4250/638/ 1852/1007/935 4900/722/ 2125/1232/1012 5450/961/ 2401/1332/1102 Amazon Corretto Java 17 Spring Boot 5300/621/ 1920/1024/844 5750/685/ 2321/1525/1043 6204/722/ 2538/1850/1290
  • 54. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 1000 2000 3000 4000 5000 6000 Pure Java GraalVM Native Image AWS SnapStartAWS SnapStart with invoke DynamoDB priming AWS SnapStart with framework web model priming Cold starts p90 No Framework Quarkus Micronaut SpringBoot
  • 55. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 500 1000 1500 2000 2500 3000 3500 4000 4500 Pure Java GraalVM Native Image AWS SnapStartAWS SnapStart with invoke DynamoDB priming Cold starts of Lambda function with Java 17 runtime p50 p90 p99
  • 56. Vadym Kazulkin @VKazulkin , ip.labs GmbH Source: https://www.serverless.com/blog/keep-your-lambdas-warm Lambda support for Java 21
  • 57. Vadym Kazulkin @VKazulkin , ip.labs GmbH https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-8-measuring-with-java-17-21db 0 500 1000 1500 2000 2500 3000 3500 4000 4500 Pure Java 17 Pure Java 21 AWS SnapStart 17 AWS SnapStart 21 AWS SnapStart with invoke DynamoDB priming 17 AWS SnapStart with invoke DynamoDB priming 21 Cold starts of Lambda function with Java 17 vs 21 runtime p50 p90 p99
  • 58. AWS SnapStart Challenges & Limitations • SnapStart supports the Java 11, 17 and 21 (all Amazon Corretto) managed runtime only • Deployment with SnapStart enabled takes more than 2-2,5 minutes additionally • Snapshot is deleted from cache if Lambda function is not invoked for 14 days • SnapStart currently does not support : • Provisioned concurrency • arm64 architecture (supports only x86) • Amazon Elastic File System (Amazon EFS) • Ephemeral storage greater than 512 MB Vadym Kazulkin @VKazulkin , ip.labs GmbH https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html
  • 59. AWS SnapStart Possible Next Steps • Perform Priming out of the box without writing the logic on our own • Snapshot creation on first Lambda function invocation instead of during the deployment phase • Regular cold start as long as the snapshot hasn’t been fully taken • Trade off between duration of the deployment phase and several regular bigger cold starts until the snapshot is taken and SnapStart becomes effective • If snapshot not found after 14 days (due to the Lambda not being invoked), do regular cold start as long as new snapshot has been fully taken under the hood Vadym Kazulkin @VKazulkin , ip.labs GmbH
  • 60. Lambda Provisioned Concurrency https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ • Requires manually managing start and end time when Provisioned concurrency should apply (can be tricky for spikey workloads) • You pay for unused capacity
  • 61. NEW: Lambda Proactive initialization In June 2023 AWS updated the documentation for the Lambda Function lifecycle and included this new statement: for functions using unreserved (on-demand) concurrency, Lambda may proactively initialize a function instance, even if there's no invocation. When this happens, you can observe an unexpected time gap between your function's initialization and invocation phases. This gap can appear similar to what you would observe when using provisioned concurrency. https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html https://aaronstuyvenberg.com/posts/understanding-proactive-initialization Running this query over several days across multiple runtimes and invocation methods, between 50% and 75% of initializations were proactive (versus 50% to 25% which were true cold starts)
  • 64. www.iplabs.de Accelerate Your Photo Business Get in Touch