SlideShare a Scribd company logo

Tech Webinar: AUMENTARE LA SCALABILITÀ DELLE WEB APP CON SERVLET 3.1 ASYNC I/O

Codemotion
Codemotion

The document discusses the benefits and implementations of asynchronous I/O in web applications using Servlet 3.1. It addresses why blocking waits are inefficient, highlights the advantages of async processing for improved resource utilization and performance, and introduces the async I/O APIs introduced in Servlet 3.1 for handling operations more effectively. Additionally, it covers common pitfalls and bugs when implementing async I/O and provides examples of async servlets.

Software
1 of 56
Downloaded 20 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
Scale your Web Applications with Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com
Who Am I Simone Bordet sbordet@intalio.com  Simone Bordet  sbordet@intalio.com  @simonebordet  Open Source Contributor  Jetty, CometD, MX4J, Foxtrot, LiveTribe, JBoss, Larex  Lead Architect at Intalio/Webtide  Jetty's SPDY and HTTP client maintainer  CometD project leader  Web messaging framework  JVM tuning expert
Agenda Simone Bordet sbordet@intalio.com Why Async ?  Async Everywhere  Async I/O API in the Servlet 3.1 Specification  Correctly use the new Servlet Async I/O APIs
Why Async ? Simone Bordet sbordet@intalio.com
Why Async ? Client Server RDBMS Simone Bordet sbordet@intalio.com
Why Async ? Client Server RDBMS Blocking Wait Simone Bordet sbordet@intalio.com
Why Async ? Why blocking waits are bad ?  They consume resources  Native Thread  Native Memory (thread stack: 1 MiB per thread)  OS scheduler data structures  ThreadLocals  Local variables in the stack frames  GC has to walk the stack frames  Blocking waits == $$$ Simone Bordet sbordet@intalio.com
Why Async ? Client Server RDBMS Async Wait Simone Bordet sbordet@intalio.com
Simone Bordet sbordet@intalio.com REST Why Async ? Client Server RDBMS Multiple operations
Why Async ? CliCelnietnt Server RDBMS Client Simone Bordet sbordet@intalio.com Multiple clients
Why Async ? Simone Bordet sbordet@intalio.com  Improved latency  By performing tasks concurrently with less resources  Better resource utilization  The SAME thread can serve multiple clients  Async waits == $  Async == Increased performance  Increased performance == $$$
Why Async ? Simone Bordet sbordet@intalio.com
Demo https://webtide.com/async-rest/ Simone Bordet sbordet@intalio.com
Async Everywhere Simone Bordet sbordet@intalio.com
Async Everywhere  Until few years ago parallelism was not widespread Simone Bordet sbordet@intalio.com  Single core CPUs Modern hardware has gone multi-core  We want to be able to use those cores  Future hardware will have even more cores We need to leverage hw parallelism in software
Async Everywhere  Async programming is not new  Spawning a thread is a primitive in OS since day one  Async programming is difficult...  … to write: callback hell, synchronization  … to read: our brain reads from top to bottom Simone Bordet sbordet@intalio.com  New APIs are born  epoll, promises, actors, channels, etc.  New products are born  Nginx vs Apache, etc.
Async Everywhere  Example: Java 8 CompletableFuture CompletableFuture .supplyAsync(() -> fetchSQLFromREST(uri)) .thenApplyAsync(sql -> fetchDataFromRBDMS(sql)) .thenAccept(data -> updateUI(data)); We want a way to read code as if it is sequential  But in fact it is asynchronous  You have to marry the API model  Always use the model, no exceptions Simone Bordet sbordet@intalio.com
Async Everywhere  Servlet 3.0: async processing of response void doGet(request, response) { OutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); doAsyncREST(request).thenAccept(json -> { out.write(json); ctx.complete(); Simone Bordet sbordet@intalio.com }); }
Async Everywhere  Two requirements for an async Servlet  First requirement: doAsyncREST()  Must use async libraries to do its work  Must return a CompletableFuture  Or take a callback parameter (or something similar)  Second requirement: thenAccept() lambda  Must use async libraries Simone Bordet sbordet@intalio.com
Async Everywhere  ✗ JDBC  ✗ InputStream – OutputStream  ✔ Asynchronous I/O (NIO2: files, sockets)  ✔ REST  ✔ Call other Servlets Simone Bordet sbordet@intalio.com
Async Everywhere  Servlet 3.0: no async I/O void doGet(request, response) { OutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); doAsyncREST(request).thenAccept(json -> { out.write(json); Blocking! ctx.complete(); Simone Bordet sbordet@intalio.com }); }
Async Everywhere  Finally! That's why Servlet 3.1 introduced async I/O What API to use then ?  Use java.util.concurrent.Future ? Future<Integer> f = out.write(json); int written = f.get().intValue(); Simone Bordet sbordet@intalio.com
Async Everywhere  Finally! That's why Servlet 3.1 introduced async I/O ! What API to use then ?  Use java.util.concurrent.Future ? Future<Integer> f = channel.write(json); int written = f.get().intValue(); This is just delayed blocking, not async Simone Bordet sbordet@intalio.com
Async Everywhere Simone Bordet sbordet@intalio.com  Use NIO2 ? out.write(json, null, new CompletionHandler() { void completed(Integer written, Object obj) { ctx.complete(); } void failed(Throwable x, Object obj) { } }
Simone Bordet sbordet@intalio.com  Use NIO2 ? out.write(json, null, new CompletionHandler() { void completed(Integer written, Object obj) { ctx.complete(); } void failed(Throwable x, Object obj) { } } Allocation Not a lambda Not thread efficient Prone to stack overflow (resolved by further thread dispatching)
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com
Servlet 3.1 Async I/O  Servlet 3.1 async write I/O APIs void doGet(request, response) { ServletOutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); out.setWriteListener(new WriteListener() { void onWritePossible() { while (out.isReady()) { byte[] buffer = readFromSomeWhere(); if (buffer != null) out.write(buffer); Simone Bordet sbordet@intalio.com else ctx.complete(); }}});}
Servlet 3.1 Async I/O  Servlet 3.1 async write I/O APIs void doGet(request, response) { ServletOutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); out.setWriteListener(new WriteListener() { void onWritePossible() { while (out.isReady()) { byte[] buffer = readFromSomeWhere(); if (buffer != null) out.write(buffer); Simone Bordet sbordet@intalio.com else ctx.complete(); break; Iterative }}});} Async write
Servlet 3.1 Async I/O  Normal writes (fast connection)  Iterative writes in the same thread  No thread dispatch  No stack overflow  No allocation  Incomplete writes (slow connection)  Some iterative write, then isReady() returns false  When ready again, new thread calls onWritePossible()  Got it ! Is it really this simple ? Simone Bordet sbordet@intalio.com
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #1 void onWritePossible() { out.write(allContent); logger.debug("written: {}", out.isReady()); }
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #1 void onWritePossible() { out.write(allContent); logger.debug("written: {}", out.isReady()); }  Content is written twice ! When isReady() == false onWritePossible() will be called again !
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #2 void onWritePossible() { if (dataAvailable() && out.isReady()) out.write(getData()); }
Servlet 3.1 Async I/O The order of the expressions in the if statement is significant ! Simone Bordet sbordet@intalio.com  Bug #2 void onWritePossible() { if (dataAvailable() && out.isReady()) out.write(getData()); }  If dataAvailable() returns false, isReady() is not called and onWritePossible() will never be called  Switching the if expressions changes the semantic
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #3 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); out.write(“<p>Content</p>”); } }
Servlet 3.1 Async I/O The first write may be pending ! Simone Bordet sbordet@intalio.com  Bug #3 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); out.write(“<p>Content</p>”); } }  You HAVE to use isReady() after each write
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #4 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); if (out.isReady()) out.write(“<p>Content</p>”); } }
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #4 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); if (out.isReady()) out.write(“<p>Content</p>”); } }  Don't misuse isReady() Content written multiple times !
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #5 void onWritePossible() { int read = file.read(buffer); while (out.isReady()) { out.write(buffer); if (file.read(buffer) < 0) ctx.complete(); return; } }
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #5 void onWritePossible() { int read = file.read(buffer); while (out.isReady()) { out.write(buffer); if (file.read(buffer) < 0) ctx.complete(); return; } Write may be pending: you don't own the buffer }  The buffer is overwritten by the new read
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #6 void onWritePossible() { try { ... } catch (Exception x) { response.sendError(500); } }
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #6 void onWritePossible() { try { ... } catch (Exception x) { response.sendError(500); } } sendError() is a blocking API !  Cannot mix async I/O and blocking I/O
Servlet 3.1 Async I/O When ServletOutputStream is put into async mode:  Quacks like a stream  Walks like a stream  But it's NOT a stream ! Simone Bordet sbordet@intalio.com  Broken semantic:  OutputStream.write() is expected to be blocking !  ServletOutputStream.isReady(): a query with side effects !  Breaking semantic consequences:  Filters and wrappers are broken !  Cannot tell whether the stream is async or not
Servlet 3.1 Async I/O  Servlet 3.1 async read I/O APIs void doPost(request, response) { AsyncContext ctx = request.startAsync(); ServletInputStream in = request.getInputStream(); in.setReadListener(new ReadListener() { void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int len = in.read(buffer); if (len > 0) store(buffer); // Blocking API if (in.isFinished()) ctx.complete(); return; Simone Bordet sbordet@intalio.com }}});}
Servlet 3.1 Async I/O  Servlet 3.1 async read I/O APIs void doPost(request, response) { AsyncContext ctx = request.startAsync(); ServletInputStream in = request.getInputStream(); in.setReadListener(new ReadListener() { void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int len = input.read(buffer); if (len > 0) store(buffer); // Blocking API if (in.isFinished()) ctx.complete(); return; Simone Bordet sbordet@intalio.com Iterative }}});} Async read
Servlet 3.1 Async I/O  Normal reads (fast connection)  Iterative reads in the same thread  No thread dispatch  No stack overflow  No allocation  Incomplete reads (slow connection)  Some iterative reads, then isReady() returns false  When ready again, new thread calls onDataAvailable()  Got it ! Is it really this simple ? Simone Bordet sbordet@intalio.com
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #1 void onDataAvailable() { while (in.isReady()) { in.read(buffer); store(buffer); } }
Servlet 3.1 Async I/O isReady() returns Simone Bordet sbordet@intalio.com  Bug #1 void onDataAvailable() { while (in.isReady()) { in.read(buffer); store(buffer); } }  Infinite loop true at EOF  The loop must call also in.isFinished()
Servlet 3.1 Async I/O Simone Bordet sbordet@intalio.com  Bug #2 void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int read = in.read(buffer); store(buffer, new Callback() { void done() { onDataAvailable(); } }); } }
Servlet 3.1 Async I/O Stack overflow ! Simone Bordet sbordet@intalio.com  Bug #2 void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int read = in.read(buffer); store(buffer, new Callback() { void done() { onDataAvailable(); } }); } }  Stack overflows for fast connections Mixing different async models requires more careful coding  Jetty's IteratingCallback is one solution
Servlet 3.1 Async I/O Echo Servlet Blocking I/O Echo Servlet Async I/O Simone Bordet sbordet@intalio.com public class SyncServlet extends HttpServlet { protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { byte[] buffer = new byte[BUFFER_SIZE]; while (true) { int read = request.getInputStream().read(buffer, 0, BUFFER_SIZE); if (read < 0) break; response.getOutputStream().write(buffer, 0, read); } } } public static class AsyncServlet extends HttpServlet { @Override protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { AsyncContext asyncContext = request.startAsync(request, response); asyncContext.setTimeout(0); Echoer echoer = new Echoer(asyncContext); request.getInputStream().setReadListener(echoer); response.getOutputStream().setWriteListener(echoer); } private class Echoer implements ReadListener, WriteListener { private final byte[] buffer = new byte[BUFFER_SIZE]; private final AsyncContext asyncContext; private final ServletInputStream input; private final ServletOutputStream output; private boolean complete; private Echoer(AsyncContext asyncContext) throws IOException { this.asyncContext = asyncContext; this.input = asyncContext.getRequest().getInputStream(); this.output = asyncContext.getResponse().getOutputStream(); } @Override public void onDataAvailable() throws IOException { while (input.isReady()) { int read = input.read(buffer); output.write(buffer, 0, read); if (!output.isReady()) return; } if (input.isFinished()) { complete = true; asyncContext.complete(); } } @Override public void onAllDataRead() throws IOException { } @Override public void onWritePossible() throws IOException { if (input.isFinished()) { if (!complete) asyncContext.complete(); } else { onDataAvailable(); } } @Override public void onError(Throwable failure) { failure.printStackTrace(); } } }
Servlet 3.1 Async I/O  Async I/O Echo Servlet Simone Bordet sbordet@intalio.com  Pros  Async  Cons  4x-6x more code  Requires a state machine (a simple boolean, but still)  More difficult buffer handling  Complexity, maintainability, etc.
Conclusions Simone Bordet sbordet@intalio.com
Servlet 3.1 Async I/O  Blocking code has a reason to exist  And will continue to exist  Is the async I/O complexity worth it ?  For a class of applications, YES !  Improved scalability  Decreased latency  Makes you more money Simone Bordet sbordet@intalio.com
Demo https://webtide.com/async-rest/ Simone Bordet sbordet@intalio.com
Why Async ? Simone Bordet sbordet@intalio.com
Questions & Answers Simone Bordet sbordet@intalio.com

More Related Content

What's hot (20)

PDF
effective_r27
Hiroshi Ono
 
PDF
Erlang and Elixir
Krzysztof Marciniak
 
KEY
Ruby Concurrency and EventMachine
Christopher Spring
 
PDF
Introduction to Elixir
Diacode
 
PDF
PyPy's approach to construct domain-specific language runtime
National Cheng Kung University
 
PPT
Server side JavaScript: going all the way
Oleg Podsechin
 
PDF
Elixir Into Production
Jamie Winsor
 
PDF
Sed Introduction
Anthony Magee
 
PDF
Lean way write asynchronous code with Kotlin’s coroutines - Ronen Sabag, Gett
DroidConTLV
 
PPTX
Oop object oriented programing topics
(•̮̮̃•̃) Prince Do Not Work
 
PDF
50 new things we can do with Java 8
José Paumard
 
PDF
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Tom Croucher
 
PDF
Concurrency in Elixir with OTP
Justin Reese
 
PDF
Ruby HTTP clients comparison
Hiroshi Nakamura
 
KEY
Puppet NBLUG 2008-09
Eric Eisenhart
 
PPTX
Coroutines talk ppt
Shahroz Khan
 
PDF
Awk Introduction
Anthony Magee
 
PDF
Ruby thread safety first
Emily Stolfo
 
PDF
ElixirConf Lightning Talk: Elixir |> Production
Jeff Weiss
 
PDF
JVMLS 2016. Coroutines in Kotlin
Andrey Breslav
 
effective_r27
Hiroshi Ono
 
Erlang and Elixir
Krzysztof Marciniak
 
Ruby Concurrency and EventMachine
Christopher Spring
 
Introduction to Elixir
Diacode
 
PyPy's approach to construct domain-specific language runtime
National Cheng Kung University
 
Server side JavaScript: going all the way
Oleg Podsechin
 
Elixir Into Production
Jamie Winsor
 
Sed Introduction
Anthony Magee
 
Lean way write asynchronous code with Kotlin’s coroutines - Ronen Sabag, Gett
DroidConTLV
 
Oop object oriented programing topics
(•̮̮̃•̃) Prince Do Not Work
 
50 new things we can do with Java 8
José Paumard
 
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
Tom Croucher
 
Concurrency in Elixir with OTP
Justin Reese
 
Ruby HTTP clients comparison
Hiroshi Nakamura
 
Puppet NBLUG 2008-09
Eric Eisenhart
 
Coroutines talk ppt
Shahroz Khan
 
Awk Introduction
Anthony Magee
 
Ruby thread safety first
Emily Stolfo
 
ElixirConf Lightning Talk: Elixir |> Production
Jeff Weiss
 
JVMLS 2016. Coroutines in Kotlin
Andrey Breslav
 

Similar to Tech Webinar: AUMENTARE LA SCALABILITÀ DELLE WEB APP CON SERVLET 3.1 ASYNC I/O (20)

ODP
Servlet 3.1 Async I/O
Simone Bordet
 
PPTX
Think async
Bhakti Mehta
 
PDF
Web Component Development Using Servlet & JSP Technologies (EE6) - Chapter 1...
WebStackAcademy
 
PDF
Introduction tomcat7 servlet3
JavaEE Trainers
 
PPTX
Don't Wait! Develop Responsive Applications with Java EE7 Instead
WASdev Community
 
PPTX
Advance java session 20
Smita B Kumar
 
PDF
CON 2107- Think Async: Embrace and Get Addicted to the Asynchronicity of EE
Masoud Kalali
 
PDF
JavaOne San Francisco 2013 - Servlet 3.1 (JSR 340)
Shing Wai Chan
 
PPTX
The Road To Reactive with RxJava JEEConf 2016
Frank Lyaruu
 
PDF
An opinionated intro to Node.js - devrupt hospitality hackathon
Luciano Mammino
 
PDF
A java servers
vibrantuser
 
PPT
A java servers
vibrantuser
 
PDF
JavaCro'15 - Spring @Async - Dragan Juričić
HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
 
PDF
Asynchronous web apps with the Play Framework 2.0
Oscar Renalias
 
PDF
Servlet 3.1
Arun Gupta
 
PDF
Asynchronous web-development with Python
Anton Caceres
 
PPTX
J2ee servlet
vinoth ponnurangam
 
PDF
The art of messaging tune (Joker 2015 edition)
Vyacheslav Lapin
 
PDF
NodeJS ecosystem
Yukti Kaura
 
PPT
Don't Wait! Develop responsive applications with Java EE7 instead
Erin Schnabel
 
Servlet 3.1 Async I/O
Simone Bordet
 
Think async
Bhakti Mehta
 
Web Component Development Using Servlet & JSP Technologies (EE6) - Chapter 1...
WebStackAcademy
 
Introduction tomcat7 servlet3
JavaEE Trainers
 
Don't Wait! Develop Responsive Applications with Java EE7 Instead
WASdev Community
 
Advance java session 20
Smita B Kumar
 
CON 2107- Think Async: Embrace and Get Addicted to the Asynchronicity of EE
Masoud Kalali
 
JavaOne San Francisco 2013 - Servlet 3.1 (JSR 340)
Shing Wai Chan
 
The Road To Reactive with RxJava JEEConf 2016
Frank Lyaruu
 
An opinionated intro to Node.js - devrupt hospitality hackathon
Luciano Mammino
 
A java servers
vibrantuser
 
A java servers
vibrantuser
 
JavaCro'15 - Spring @Async - Dragan Juričić
HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
 
Asynchronous web apps with the Play Framework 2.0
Oscar Renalias
 
Servlet 3.1
Arun Gupta
 
Asynchronous web-development with Python
Anton Caceres
 
J2ee servlet
vinoth ponnurangam
 
The art of messaging tune (Joker 2015 edition)
Vyacheslav Lapin
 
NodeJS ecosystem
Yukti Kaura
 
Don't Wait! Develop responsive applications with Java EE7 instead
Erin Schnabel
 
Ad

More from Codemotion (20)

PDF
Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...
Codemotion
 
PDF
Pompili - From hero to_zero: The FatalNoise neverending story
Codemotion
 
PPTX
Pastore - Commodore 65 - La storia
Codemotion
 
PPTX
Pennisi - Essere Richard Altwasser
Codemotion
 
PPTX
Michel Schudel - Let's build a blockchain... in 40 minutes! - Codemotion Amst...
Codemotion
 
PPTX
Richard Süselbeck - Building your own ride share app - Codemotion Amsterdam 2019
Codemotion
 
PPTX
Eward Driehuis - What we learned from 20.000 attacks - Codemotion Amsterdam 2019
Codemotion
 
PPTX
Francesco Baldassarri - Deliver Data at Scale - Codemotion Amsterdam 2019 -
Codemotion
 
PDF
Martin Förtsch, Thomas Endres - Stereoscopic Style Transfer AI - Codemotion A...
Codemotion
 
PDF
Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
Codemotion
 
PDF
Angelo van der Sijpt - How well do you know your network stack? - Codemotion ...
Codemotion
 
PDF
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Codemotion
 
PDF
Sascha Wolter - Conversational AI Demystified - Codemotion Amsterdam 2019
Codemotion
 
PDF
Michele Tonutti - Scaling is caring - Codemotion Amsterdam 2019
Codemotion
 
PPTX
Pat Hermens - From 100 to 1,000+ deployments a day - Codemotion Amsterdam 2019
Codemotion
 
PPTX
James Birnie - Using Many Worlds of Compute Power with Quantum - Codemotion A...
Codemotion
 
PDF
Don Goodman-Wilson - Chinese food, motor scooters, and open source developmen...
Codemotion
 
PDF
Pieter Omvlee - The story behind Sketch - Codemotion Amsterdam 2019
Codemotion
 
PDF
Dave Farley - Taking Back “Software Engineering” - Codemotion Amsterdam 2019
Codemotion
 
PDF
Joshua Hoffman - Should the CTO be Coding? - Codemotion Amsterdam 2019
Codemotion
 
Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...
Codemotion
 
Pompili - From hero to_zero: The FatalNoise neverending story
Codemotion
 
Pastore - Commodore 65 - La storia
Codemotion
 
Pennisi - Essere Richard Altwasser
Codemotion
 
Michel Schudel - Let's build a blockchain... in 40 minutes! - Codemotion Amst...
Codemotion
 
Richard Süselbeck - Building your own ride share app - Codemotion Amsterdam 2019
Codemotion
 
Eward Driehuis - What we learned from 20.000 attacks - Codemotion Amsterdam 2019
Codemotion
 
Francesco Baldassarri - Deliver Data at Scale - Codemotion Amsterdam 2019 -
Codemotion
 
Martin Förtsch, Thomas Endres - Stereoscopic Style Transfer AI - Codemotion A...
Codemotion
 
Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
Codemotion
 
Angelo van der Sijpt - How well do you know your network stack? - Codemotion ...
Codemotion
 
Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...
Codemotion
 
Sascha Wolter - Conversational AI Demystified - Codemotion Amsterdam 2019
Codemotion
 
Michele Tonutti - Scaling is caring - Codemotion Amsterdam 2019
Codemotion
 
Pat Hermens - From 100 to 1,000+ deployments a day - Codemotion Amsterdam 2019
Codemotion
 
James Birnie - Using Many Worlds of Compute Power with Quantum - Codemotion A...
Codemotion
 
Don Goodman-Wilson - Chinese food, motor scooters, and open source developmen...
Codemotion
 
Pieter Omvlee - The story behind Sketch - Codemotion Amsterdam 2019
Codemotion
 
Dave Farley - Taking Back “Software Engineering” - Codemotion Amsterdam 2019
Codemotion
 
Joshua Hoffman - Should the CTO be Coding? - Codemotion Amsterdam 2019
Codemotion
 
Ad

Recently uploaded (20)

PPTX
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
PDF
Download Canva Pro 2025 PC Crack Full Latest Version
bashirkhan333g
 
PDF
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
PDF
Empower Your Tech Vision- Why Businesses Prefer to Hire Remote Developers fro...
logixshapers59
 
PDF
SAP Firmaya İade ABAB Kodları - ABAB ile yazılmıl hazır kod örneği
Salih Küçük
 
PDF
Unlock Efficiency with Insurance Policy Administration Systems
Insurance Tech Services
 
PPTX
Homogeneity of Variance Test Options IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
Linux Certificate of Completion - LabEx Certificate
VICTOR MAESTRE RAMIREZ
 
PPTX
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
PDF
Driver Easy Pro 6.1.1 Crack Licensce key 2025 FREE
utfefguu
 
PDF
Generic or Specific? Making sensible software design decisions
Bert Jan Schrijver
 
PPTX
Finding Your License Details in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
vMix Pro 28.0.0.42 Download vMix Registration key Bundle
kulindacore
 
PDF
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
PDF
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
PPTX
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
PPTX
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
PPTX
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
How to Hire AI Developers_ Step-by-Step Guide in 2025.pdf
DianApps Technologies
 
PDF
HiHelloHR – Simplify HR Operations for Modern Workplaces
HiHelloHR
 
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
Download Canva Pro 2025 PC Crack Full Latest Version
bashirkhan333g
 
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
Empower Your Tech Vision- Why Businesses Prefer to Hire Remote Developers fro...
logixshapers59
 
SAP Firmaya İade ABAB Kodları - ABAB ile yazılmıl hazır kod örneği
Salih Küçük
 
Unlock Efficiency with Insurance Policy Administration Systems
Insurance Tech Services
 
Homogeneity of Variance Test Options IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
Linux Certificate of Completion - LabEx Certificate
VICTOR MAESTRE RAMIREZ
 
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
Driver Easy Pro 6.1.1 Crack Licensce key 2025 FREE
utfefguu
 
Generic or Specific? Making sensible software design decisions
Bert Jan Schrijver
 
Finding Your License Details in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
vMix Pro 28.0.0.42 Download vMix Registration key Bundle
kulindacore
 
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
How to Hire AI Developers_ Step-by-Step Guide in 2025.pdf
DianApps Technologies
 
HiHelloHR – Simplify HR Operations for Modern Workplaces
HiHelloHR
 

Tech Webinar: AUMENTARE LA SCALABILITÀ DELLE WEB APP CON SERVLET 3.1 ASYNC I/O

  • 1. Scale your Web Applications with Servlet 3.1 Async I/O Simone Bordet [email protected]
  • 2. Who Am I Simone Bordet [email protected]  Simone Bordet  [email protected]  @simonebordet  Open Source Contributor  Jetty, CometD, MX4J, Foxtrot, LiveTribe, JBoss, Larex  Lead Architect at Intalio/Webtide  Jetty's SPDY and HTTP client maintainer  CometD project leader  Web messaging framework  JVM tuning expert
  • 3. Agenda Simone Bordet [email protected] Why Async ?  Async Everywhere  Async I/O API in the Servlet 3.1 Specification  Correctly use the new Servlet Async I/O APIs
  • 5. Why Async ? Client Server RDBMS Simone Bordet [email protected]
  • 6. Why Async ? Client Server RDBMS Blocking Wait Simone Bordet [email protected]
  • 7. Why Async ? Why blocking waits are bad ?  They consume resources  Native Thread  Native Memory (thread stack: 1 MiB per thread)  OS scheduler data structures  ThreadLocals  Local variables in the stack frames  GC has to walk the stack frames  Blocking waits == $$$ Simone Bordet [email protected]
  • 8. Why Async ? Client Server RDBMS Async Wait Simone Bordet [email protected]
  • 9. Simone Bordet [email protected] REST Why Async ? Client Server RDBMS Multiple operations
  • 10. Why Async ? CliCelnietnt Server RDBMS Client Simone Bordet [email protected] Multiple clients
  • 11. Why Async ? Simone Bordet [email protected]  Improved latency  By performing tasks concurrently with less resources  Better resource utilization  The SAME thread can serve multiple clients  Async waits == $  Async == Increased performance  Increased performance == $$$
  • 15. Async Everywhere  Until few years ago parallelism was not widespread Simone Bordet [email protected]  Single core CPUs Modern hardware has gone multi-core  We want to be able to use those cores  Future hardware will have even more cores We need to leverage hw parallelism in software
  • 16. Async Everywhere  Async programming is not new  Spawning a thread is a primitive in OS since day one  Async programming is difficult...  … to write: callback hell, synchronization  … to read: our brain reads from top to bottom Simone Bordet [email protected]  New APIs are born  epoll, promises, actors, channels, etc.  New products are born  Nginx vs Apache, etc.
  • 17. Async Everywhere  Example: Java 8 CompletableFuture CompletableFuture .supplyAsync(() -> fetchSQLFromREST(uri)) .thenApplyAsync(sql -> fetchDataFromRBDMS(sql)) .thenAccept(data -> updateUI(data)); We want a way to read code as if it is sequential  But in fact it is asynchronous  You have to marry the API model  Always use the model, no exceptions Simone Bordet [email protected]
  • 18. Async Everywhere  Servlet 3.0: async processing of response void doGet(request, response) { OutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); doAsyncREST(request).thenAccept(json -> { out.write(json); ctx.complete(); Simone Bordet [email protected] }); }
  • 19. Async Everywhere  Two requirements for an async Servlet  First requirement: doAsyncREST()  Must use async libraries to do its work  Must return a CompletableFuture  Or take a callback parameter (or something similar)  Second requirement: thenAccept() lambda  Must use async libraries Simone Bordet [email protected]
  • 20. Async Everywhere  ✗ JDBC  ✗ InputStream – OutputStream  ✔ Asynchronous I/O (NIO2: files, sockets)  ✔ REST  ✔ Call other Servlets Simone Bordet [email protected]
  • 21. Async Everywhere  Servlet 3.0: no async I/O void doGet(request, response) { OutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); doAsyncREST(request).thenAccept(json -> { out.write(json); Blocking! ctx.complete(); Simone Bordet [email protected] }); }
  • 22. Async Everywhere  Finally! That's why Servlet 3.1 introduced async I/O What API to use then ?  Use java.util.concurrent.Future ? Future<Integer> f = out.write(json); int written = f.get().intValue(); Simone Bordet [email protected]
  • 23. Async Everywhere  Finally! That's why Servlet 3.1 introduced async I/O ! What API to use then ?  Use java.util.concurrent.Future ? Future<Integer> f = channel.write(json); int written = f.get().intValue(); This is just delayed blocking, not async Simone Bordet [email protected]
  • 24. Async Everywhere Simone Bordet [email protected]  Use NIO2 ? out.write(json, null, new CompletionHandler() { void completed(Integer written, Object obj) { ctx.complete(); } void failed(Throwable x, Object obj) { } }
  • 25. Simone Bordet [email protected]  Use NIO2 ? out.write(json, null, new CompletionHandler() { void completed(Integer written, Object obj) { ctx.complete(); } void failed(Throwable x, Object obj) { } } Allocation Not a lambda Not thread efficient Prone to stack overflow (resolved by further thread dispatching)
  • 27. Servlet 3.1 Async I/O  Servlet 3.1 async write I/O APIs void doGet(request, response) { ServletOutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); out.setWriteListener(new WriteListener() { void onWritePossible() { while (out.isReady()) { byte[] buffer = readFromSomeWhere(); if (buffer != null) out.write(buffer); Simone Bordet [email protected] else ctx.complete(); }}});}
  • 28. Servlet 3.1 Async I/O  Servlet 3.1 async write I/O APIs void doGet(request, response) { ServletOutputStream out = response.getOutputStream(); AsyncContext ctx = request.startAsync(); out.setWriteListener(new WriteListener() { void onWritePossible() { while (out.isReady()) { byte[] buffer = readFromSomeWhere(); if (buffer != null) out.write(buffer); Simone Bordet [email protected] else ctx.complete(); break; Iterative }}});} Async write
  • 29. Servlet 3.1 Async I/O  Normal writes (fast connection)  Iterative writes in the same thread  No thread dispatch  No stack overflow  No allocation  Incomplete writes (slow connection)  Some iterative write, then isReady() returns false  When ready again, new thread calls onWritePossible()  Got it ! Is it really this simple ? Simone Bordet [email protected]
  • 30. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #1 void onWritePossible() { out.write(allContent); logger.debug("written: {}", out.isReady()); }
  • 31. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #1 void onWritePossible() { out.write(allContent); logger.debug("written: {}", out.isReady()); }  Content is written twice ! When isReady() == false onWritePossible() will be called again !
  • 32. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #2 void onWritePossible() { if (dataAvailable() && out.isReady()) out.write(getData()); }
  • 33. Servlet 3.1 Async I/O The order of the expressions in the if statement is significant ! Simone Bordet [email protected]  Bug #2 void onWritePossible() { if (dataAvailable() && out.isReady()) out.write(getData()); }  If dataAvailable() returns false, isReady() is not called and onWritePossible() will never be called  Switching the if expressions changes the semantic
  • 34. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #3 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); out.write(“<p>Content</p>”); } }
  • 35. Servlet 3.1 Async I/O The first write may be pending ! Simone Bordet [email protected]  Bug #3 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); out.write(“<p>Content</p>”); } }  You HAVE to use isReady() after each write
  • 36. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #4 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); if (out.isReady()) out.write(“<p>Content</p>”); } }
  • 37. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #4 void onWritePossible() { if (out.isReady()) { out.write(“<h1>Async</h1>”); if (out.isReady()) out.write(“<p>Content</p>”); } }  Don't misuse isReady() Content written multiple times !
  • 38. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #5 void onWritePossible() { int read = file.read(buffer); while (out.isReady()) { out.write(buffer); if (file.read(buffer) < 0) ctx.complete(); return; } }
  • 39. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #5 void onWritePossible() { int read = file.read(buffer); while (out.isReady()) { out.write(buffer); if (file.read(buffer) < 0) ctx.complete(); return; } Write may be pending: you don't own the buffer }  The buffer is overwritten by the new read
  • 40. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #6 void onWritePossible() { try { ... } catch (Exception x) { response.sendError(500); } }
  • 41. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #6 void onWritePossible() { try { ... } catch (Exception x) { response.sendError(500); } } sendError() is a blocking API !  Cannot mix async I/O and blocking I/O
  • 42. Servlet 3.1 Async I/O When ServletOutputStream is put into async mode:  Quacks like a stream  Walks like a stream  But it's NOT a stream ! Simone Bordet [email protected]  Broken semantic:  OutputStream.write() is expected to be blocking !  ServletOutputStream.isReady(): a query with side effects !  Breaking semantic consequences:  Filters and wrappers are broken !  Cannot tell whether the stream is async or not
  • 43. Servlet 3.1 Async I/O  Servlet 3.1 async read I/O APIs void doPost(request, response) { AsyncContext ctx = request.startAsync(); ServletInputStream in = request.getInputStream(); in.setReadListener(new ReadListener() { void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int len = in.read(buffer); if (len > 0) store(buffer); // Blocking API if (in.isFinished()) ctx.complete(); return; Simone Bordet [email protected] }}});}
  • 44. Servlet 3.1 Async I/O  Servlet 3.1 async read I/O APIs void doPost(request, response) { AsyncContext ctx = request.startAsync(); ServletInputStream in = request.getInputStream(); in.setReadListener(new ReadListener() { void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int len = input.read(buffer); if (len > 0) store(buffer); // Blocking API if (in.isFinished()) ctx.complete(); return; Simone Bordet [email protected] Iterative }}});} Async read
  • 45. Servlet 3.1 Async I/O  Normal reads (fast connection)  Iterative reads in the same thread  No thread dispatch  No stack overflow  No allocation  Incomplete reads (slow connection)  Some iterative reads, then isReady() returns false  When ready again, new thread calls onDataAvailable()  Got it ! Is it really this simple ? Simone Bordet [email protected]
  • 46. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #1 void onDataAvailable() { while (in.isReady()) { in.read(buffer); store(buffer); } }
  • 47. Servlet 3.1 Async I/O isReady() returns Simone Bordet [email protected]  Bug #1 void onDataAvailable() { while (in.isReady()) { in.read(buffer); store(buffer); } }  Infinite loop true at EOF  The loop must call also in.isFinished()
  • 48. Servlet 3.1 Async I/O Simone Bordet [email protected]  Bug #2 void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int read = in.read(buffer); store(buffer, new Callback() { void done() { onDataAvailable(); } }); } }
  • 49. Servlet 3.1 Async I/O Stack overflow ! Simone Bordet [email protected]  Bug #2 void onDataAvailable() { while (in.isReady() && !in.isFinished()) { int read = in.read(buffer); store(buffer, new Callback() { void done() { onDataAvailable(); } }); } }  Stack overflows for fast connections Mixing different async models requires more careful coding  Jetty's IteratingCallback is one solution
  • 50. Servlet 3.1 Async I/O Echo Servlet Blocking I/O Echo Servlet Async I/O Simone Bordet [email protected] public class SyncServlet extends HttpServlet { protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { byte[] buffer = new byte[BUFFER_SIZE]; while (true) { int read = request.getInputStream().read(buffer, 0, BUFFER_SIZE); if (read < 0) break; response.getOutputStream().write(buffer, 0, read); } } } public static class AsyncServlet extends HttpServlet { @Override protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { AsyncContext asyncContext = request.startAsync(request, response); asyncContext.setTimeout(0); Echoer echoer = new Echoer(asyncContext); request.getInputStream().setReadListener(echoer); response.getOutputStream().setWriteListener(echoer); } private class Echoer implements ReadListener, WriteListener { private final byte[] buffer = new byte[BUFFER_SIZE]; private final AsyncContext asyncContext; private final ServletInputStream input; private final ServletOutputStream output; private boolean complete; private Echoer(AsyncContext asyncContext) throws IOException { this.asyncContext = asyncContext; this.input = asyncContext.getRequest().getInputStream(); this.output = asyncContext.getResponse().getOutputStream(); } @Override public void onDataAvailable() throws IOException { while (input.isReady()) { int read = input.read(buffer); output.write(buffer, 0, read); if (!output.isReady()) return; } if (input.isFinished()) { complete = true; asyncContext.complete(); } } @Override public void onAllDataRead() throws IOException { } @Override public void onWritePossible() throws IOException { if (input.isFinished()) { if (!complete) asyncContext.complete(); } else { onDataAvailable(); } } @Override public void onError(Throwable failure) { failure.printStackTrace(); } } }
  • 51. Servlet 3.1 Async I/O  Async I/O Echo Servlet Simone Bordet [email protected]  Pros  Async  Cons  4x-6x more code  Requires a state machine (a simple boolean, but still)  More difficult buffer handling  Complexity, maintainability, etc.
  • 53. Servlet 3.1 Async I/O  Blocking code has a reason to exist  And will continue to exist  Is the async I/O complexity worth it ?  For a class of applications, YES !  Improved scalability  Decreased latency  Makes you more money Simone Bordet [email protected]