Reactive Java: Promises and Streams with Reakt (JavaOne talk 2016)

Editor's Notes

• #2: These slides are from a talk we gave at JavaOne 2016 for work we did in the past, and open-source projects that we are working on.
• #6: 10 seconds
• #7: 10 seconds
• #9: Give example of not small Looked at RXJava, Reactive Streams, Actors Used Vert.x extensively Not tied to QBit.
• #11: The headline is the slide, briefly cover the concepts. Results can comeback on foreign threadsIf you need to call several services, what if one of them fails or times out?What if you need to combine the results of several calls before you can respond to the caller?What if a downstream service goes down?What if your async handler throws an exception?If you use a blocking future, you can tie up the event handling threadHow do you test async services with a unit test framework?
• #13: The best way to improve a lib is to use it. Many methods were added to simplify daily dev life. Scratching our own itches.
• #14: Each slide in this section is to be covered quickly. No more than 30 seconds per slide.
• #16: Geoff tells anecdotes about experience with ES6. Breaking a large monolithic Node.js app into microservices calling into services written in Java/Vert.x. Picked Vert.x to handle large amount of traffic on fewer boxes for in-memory service Vert.x + early QBit handled more load on 13 boxes than 2,000 boxes did for similar system Created stream/batch, service actor system to simplify Vert.x dev (circa 2012/2013) maximize throughput by minimizing IO and thread hand-off (QBit Java Microservices Lib) Trial by fire, Callback coordination rough Needed high-speed call coordination for OAuth rate limiter fronting many backend services Worked on QBit Reactor but interface was better than before but still too complicated 2016 - Worked on Vert.x / QBit project / Node.js project Started using Node.js / JavaScript promises for client libs Nice abstraction for dealing with async service calls JS Promises were just right We have been doing async callback coordination for some time. Using messaging, and handler ids, and streams, and reactor pattern, and all forms of async programming models. It is clunky. We tend to think about things as Future. We tend to think in a MT world not a reactor, async, streaming world. Promises are a nice abstraction to not just reactor pattern but many forms of Java async programming. Having written three or four async callback handlers systems and not really liking any of them per se, and seeing the similarities to JS Promise but also seeing the eloquence of async promises, it seemed to make sense to adopt the terminology of JavaScript promises and the simplicity of it. Picked Vert.x as our server to handle large amount of traffic on fewer boxes Vert.x + early QBit allowed us to handle more load on 13 boxes than 2000 boxes on similar system at same company Created stream/batch, service, actor system to simplify Vert.x dev (circa 2012/2013) and to maximize throughput by minimizing IO and thread hand-off (QBit Java Microservices Lib) - Trial by fire, Callback coordination rough. Needed to do high-speed call coordination for OAuth rate limiter fronting many backend services - Worked on QBit Reactor but interface was better than before but clunky 2016 - Worked on Vert.x / QBit project and started writing integration libs Started using Node.js / JavaScript promises Nice abstraction for dealing with async service calls
• #17: The point of this slide and the next is to make the claim that streaming is not the only way to do reactive programming. The most common form of reactive programming is the reactor pattern / event loop, i.e., Browser DOM, Node.JS, Twisted and Vert.x. Streaming fits many problem domains but so do service calls which are also more common. Also streaming can be an implementation detail (as you can stream calls and stream responses). If streaming is not the only way to handle async programming and not even the most common, what / how to people do service style programming and what tools do they use. Node.js and Browser DOM JS are the two most common forms of async programming and they use Promises. Akka has promises, Netty has promises, Vert.x has async result which is similar and QBit had something like promises before Reakt. Reakt attempts to be separate from any particular implementation to focus on being a good promise lib for any sort of async programming in Java. From Microservices paper by Martin Fowler et al: “Synchronous calls considered harmful: Any time you have a number of synchronous calls between services you will encounter the multiplicative effect of downtime. Simply, this is when the downtime of your system becomes the product of the downtimes of the individual components. ...” http://martinfowler.com/articles/microservices.html
• #19: “The reactor design pattern is an event handling pattern for handling service requests delivered concurrently to a service handler by one or more inputs. The service handler then demultiplexes the incoming requests and dispatches them synchronously to the associated request handlers.” https://en.wikipedia.org/wiki/Reactor_pattern “The Reactor pattern has been introduced in [Schmidt95] as a general architecture for event-driven systems. It explains how to register handlers for particular event types, and how to activate handlers when events occur, even when events come from multiple sources, in a single-threaded environment. In other words, the reactor allows for the combination of multiple event-loops, without introducing additional threads.” http://www.cs.vu.nl/~eliens/online/oo/I/2/reactor.html
• #21: A good anecdote would be good here. Having multiple projects open. Loading the wrong one. Thinking it is the Java client when it was the JavaScript client. Wrote client libs in Java with Reakt Promises and ES6 (JavaScript) with Promises Code looked very similar Semantics were same Slight syntactic differences We were both “Wow!” That is clean It became hard to look at old way
• #23: 10 seconds.. We cover this again in detail
• #25: This has been adapted from this article on ES6 promises. A promise can be: fulfilled The callback/action relating to the promise succeeded rejected The callback/action relating to the promise failed pending The callback/action has not been fulfilled or rejected yet completed The callback/action has been fulfilled/resolved or rejected Java is not single threaded, meaning that two bits of code can run at the same time, so the design of this promise and streaming library takes that into account.
• #27: JavaScript is single-threaded - You can make call, then register for callbacks and that is ok because call won’t be made until event loop moves on. Not so in Java! Java is MT to the core. Java is not single threaded, meaning that two bits of code can run at the same time, so the design of this promise and streaming library takes that into account. There are three types of promises: Callback promises Blocking promises (for testing and legacy integration) Replay promises (allow promises to be handled on the same thread as caller) Replay promises are the most like their JS cousins. Replay promises are usually managed by the Reakt Reactor and supports environments like Vert.x and QBit. See the wiki for more details on Replay promises. It is common to make async calls to store data in a NoSQL store or to call a remote REST interface or deal with a distributed cache or queue. Also Java is strongly typed so the library that mimics JS promises is going to look a bit different. We tried to use similar terminology where it makes sense. Events and Streams are great for things that can happen multiple times on the same object — keyup, touchstart, or event a user action stream from Kafka, etc. With those events you don't really care about what happened before when you attached the listener. But often times when dealing with services and data repositories, you want to handle a response with a specific next action, and a different action if there was an error or timeout from the responses. You essentially want to call and handle a response asynchronously and that is what promises allow. This is not our first time to bat with Promises. QBit has had Promises for a few years now. We just called them CallbackBuilders instead. We wanted to use more standard terminology and wanted to use the same terminology and modeling on projects that do not use QBit like Conekt, Vert.x, RxJava, and reactive streams. At their most basic level, promises are like event listeners except: A promise can only succeed or fail once. A promise cannot succeed or fail twice, neither can it switch from success to failure. Once it enters its completed state, then it is done.
• #29: thenExpect and thenSafeExpect The handlers thenExpect and thenSafeExpect return a Reakt Expected instance. Expected is like Option in Java 8, it has methods like map, filter, etc. and adds methods ifEmpty, isEmpty. This gives a nice fluent API when you don't know if a successful return is null or not. then and thenSafeExpect The methods then and thenSafe async return the result that is not wrapped in an Expected object, i.e., the raw result. Use then and thenSafe when you know the async return will not be null. Use thenExpect and thenSafeExpect if the value could be null or if you want to map or filter the result. thenMap Use thenMap when a promise returns for example a List<Employee>, but you only want the first Employee. See Promise.thenMap for more details. safe thenSafe thenSafeExpect Unless you are using a reactor, custom Promises or blocking promises, the then* handlers will typically run in a foreign thread and if they throw an exception depending on the library, they could get logged in an odd way. If you think your handler could throw an exception (not the service you are calling but your handlers), then you might want to use thenSafe or thenSafeExpect. These will wrap your async then* handler code in a try/catch and pass the thrown exception to a ThenHandlerException to catchError. If your code ever hangs when making an async call, try using a thenSafe or thenSafeExpect. They ensure that any exceptions thrown in your handler don't get dropped by the system you are using, which could indicate a lack of understanding of the async lib you are using or that you are using it wrong. If it hangs, try thenSafe or thenSafeExpect. They help you debug async problems.
• #30: We have all functionality with promises. You can create a promise that waits (async triggers) on all promises passed to it to be async returned. The Promise.all(list or array) method returns a promise that resolves when all of the promises have resolved, or rejects with the reason of the first passed promise that rejects.
• #31: This example is from the lab. It saves a Todo item into two tables into cassandra. If either promise fails, the all promise fails. If they both succeed, the all promise succeeds. If every promise you pass to a promise is invokeable then the all promise is invokeable as well, and calling invoke on it will invoke all of the children promises.
• #32: Promise.any(promises...) method returns a promise that resolves as soon as one of promises resolves. It will ignore errors from any of the other promises as long as they don’t all error. It is similar but different than Promises.race from JavaScript. Many JS libs have an any equiv. Promise.race does not seem as useful as Promsie.any, but we might add Promise.race as well.
• #34: This is an example of adapting a Guava style future handling to Reakt so you can use fluent promises with Cassandra. Guava gets used by many libraries for its async support. Many NoSQL drivers use Guava, e.g., Cassandra. Guava is JDK 1.6 backwards compatible. Reakt provides composable promises that support lambda expressions, and a fluent API. This bridge allows you to use Reakt's promises, reactive streams and callbacks to have a more modern Java experience with libs like Cassandra and other libs that use Guava.
• #35: The above shows adapting a Vert.x async handler to a Reakt promise. We used this quite a bit.
• #37: Reactor is similar to QBit Reactor (but cleaner, 2nd attempt) or Vert.x Verticle context. The Reactor is a class that enables callbacks that execute in caller's thread (thread safe, async callbacks) tasks that run in the caller's thread repeating tasks that run in a caller's thread one shot after time period tasks that run in the caller's thread The reakt Reactor is a lot like the QBit Reactor or the Vert.x context. It allows you to enable tasks that run in that actors or verticles thread. The reakt Reactor creates replay promises. Replay promises execute in the same thread as the caller. They are "replayed" in the callers thread. QBit implements a service actor model (similar to Akka type actors), and Vert.x implements a Reactor model (like Node.js). QBit, for example ensures that all method calls are queued and handled by the service/actor thread. You can also use the QBit Reactor to ensure that callbacks happen on the same thread as the caller. This allows you callbacks to be thread safe. The Reakt Reactor is a drop in replacement for QBit Reactor except that the Reakt Reactor uses Reakt Promises, asyncResults and Callbacks. QBit 2 and Conekt will use Reakt's API and not its own. You can use the Reakt Reactor with RxJava, Vert.x, or Spring Reactor and other similar minded projects to manage repeating tasks, tasks, and callbacks on the same thread as the caller (which you do not always need to do). The Reactor is just an interface so you could replace it with an optimized version.
• #38: Reactor Methods of note Here is a high level list of Reactor methods. addRepeatingTask(interval, runnable) add a task that repeats every interval runTaskAfter(afterInterval, runnable) run a task after an interval expires deferRun(Runnable runnable) run a task on this thread as soon as you can static reactor(...) create a reactor all(...) create a promise that does not async return until all promises async return. (you can pass a timeout) any(...) create a promise that does not async return until one of the promises async return. (you can pass a timeout) process process all tasks, callbacks. Here is the Reactor interface that can be implemented by anyone. Promise invokeWithReactor (shorthand) runas replay on reactor
• #39: This might be a good slide to actually pull up the invokeWithReactor code and show what it is doing.
• #40: This slide might make more sense now.
• #42: A Breaker is short for Circuit Breaker. The idea behind the breaker is to wrap access to a service so that errors can be tracked and the circuit breaker can open if errors are exceeded. Like all things in Reakt there is an interface for Breaker that defines a contract but other implementations can get creative on how they detect the Breaker has been thrown.
• #43: In this example we are using this with a database connect, but imagine this could be a downstream REST client or a downstream WebSocket client. It has been our experience, especially in cloud environments. That downstream clients can be restarted and they not only have to be reconnected to, but they may have to be relooked up in DNS or some other form of client discovery. They may have moved addresses entirely. @Override public Promise<Boolean> connect() { return invokablePromise(promise -> { serviceMgmt.increment("connect.called"); discoveryService.lookupService(cassandraURI).thenSafe(cassandraUris -> { serviceMgmt.increment("discovery.service.success"); final Builder builder = builder(); cassandraUris.forEach(cassandraURI1 -> builder.withPort(cassandraURI1.getPort()) .addContactPoints(cassandraURI1.getHost()).build()); futureToPromise(builder.build().connectAsync()) //Cassandra / Guava Reakt bridge. .catchError(error -> promise.reject("Unable to load initial session", error)) .then(sessionToInitialize -> buildDBIfNeeded(sessionToInitialize) .thenSafe(session -> { cassandraErrors.set(0); sessionBreaker = Breaker.operational(session, 10, theSession -> !theSession.isClosed() && cassandraErrors.incrementAndGet() > 25 ); promise.resolve(true); }) .catchError(error -> promise.reject( "Unable to create or initialize session", error) ).invokeWithReactor(reactor) ).invokeWithReactor(reactor); }).catchError(error -> serviceMgmt.increment("discovery.service.fail")).invokeWithReactor(reactor); }); }
• #46: A BlockingPromise is very much like a Java Future. It is blocking. This is useful for unit testing and for legacy integration. Promises returns a blocking promise as follows: Blocking Promise /** * Create a blocking promise. * NOTE BLOCKING PROMISES ARE FOR LEGACY INTEGRATION AND TESTING ONLY!!! * After you create a promise you register its then and catchError and then you use it to * handle a callback. * * @param <T> type of result * @return new promise */ static <T> Promise<T> blockingPromise() { return new BlockingPromise<>(); }
• #47: You could show what invokeAsBlockingProimse looks like. Show the actual code. The second code listing shows how to use an expected as well.
• #49: Stream is a generic event handler for N results, i.e., a stream of results. This is a like a type of Callback for streaming results. While Callback can be considered for scalar results, a Stream is more appropriate for non-scalar results, i.e., Stream.onNext will get called many times. StreamResult is a result of an async operations with optional methods for cancel and request more items.
• #51: Break it down
• #65: Recently, Rick help develop many resilient, fault-tolerant microservices running in a Heroku clone based on Mesos in Scala and Java. He adapted QBit, microservice lib, so that when you drop in a service, it hooks into all of the 12 factor features that Orchard supports. QBit plugs into Orchard, port binding, DNS discovery, KPI stats, monitoring/distributed logging, health system, etc. Rick also wrote support for API gateway services via built-in Swagger support. This allows generation of service docs as well as REST clients for Python, Ruby, Java, Scala, etc. For the JVM, you also get WebSocket client which are auto generated runtime stubs which can invoke 500K method calls (request response) over WebSocket a second (per thread). Rick also wrote the OAuth rate limiter microservice which does OAuth rate limiting by application id and service load balancing for another online media services. Before that Rick wrote a 100 million-user in-memory content preference engine microservice for a large media company with custom NoSQL service store (2014) as part of this effort he wrote high speed JSON REST/WebSocket framework for reactive computing model based on Boon and Vert.x (and a disk batcher capable of writing 720 MB per second—the disk batcher was later used by Beats).Rick wrote over 150,000+ lines of open source code in 2013 – 2016. Rick also contributed to the reference implementations of Grid Computing and enterprise caches as well as being a member of several spec. committees (JSR-347, JSR-107, etc.) Rick is the primary author of Boon, SlumberDB and QBit.Rick is author of the best-selling book Java Tools for Extreme Programming (#1 SW development book on Amazon for 3 months) and other books. Rick also wrote a book on Python that covered programming and OO basics which was used as college text for introduction to programming and software development. Rick setup computer science programs at an elementary school and taught classes for three years. "Rick has the distinction of writing the single most popular article/series ever published on the Java technology zone." --Jenni Aloi, IBM DeveloperWorks. Rick also wrote a book on Java Web Development, which is the number one download on TheServerSide.com, and he wrote about NoSQL and scalability on InfoQ, and was the NoSQL editor for a short period of time. Prior to becoming a consultant, Rick helped create a startup that hosted 2,000 online stores on commodity hardware. 
• #75: The point of this slide and the previous is to make the claim that streaming is not the only way to do reactive programming. The most common form of reactive programming is the reactor pattern / event loop, i.e., Browser DOM, Node.JS, Twisted and Vert.x. Streaming fits many problem domains but so do service calls which are also more common. Also streaming can be an implementation detail (as you can stream calls and stream responses). If streaming is not the only way to handle async programming and not even the most common, what / how to people do service style programming and what tools do they use. Node.js and Browser DOM JS are the two most common forms of async programming and they use Promises. Akka has promises, Netty has promises, Vert.x has async result which is similar and QBit had something like promises before Reakt. Reakt attempts to be separate from any particular implementation to focus on being a good promise lib for any sort of async programming in Java.
• #79: Works well with Reactor Architecture (Vert.x, Spring Reactor, ) Works well with Actor model and Active Objects (Akka actors, Akka typed actor, QBit, etc.) ReplayPromises need a Reactor (or something that fulfills that role) Reactor is just an interface so you can replace it with one optimized for your environment
• #80: You can talk about changes in 4.0 and how we are simplifying the interface.