Webinar Slides: Real time Recommendations with Redis, Java and Websockets

Editor's Notes

• #2: Hello everybody, thank you for coming and welcome to my presentation. My name is Ninad Divadkar, and I’m going to talk about Real time recommendations using Web Sockets and Redis.
• #3: A little bit about my self. I work for Intuit Inc in the QuickBooks Self Employed division. I’m a full stack developer, and although I have used quite a few different programming languages over the years, I’d say my backend expertise is in JAVA and .NET, and front end in AngularJS. I’m also a Spring contributor. My latest contribution is to the spring-data-redis project. Redis is introduced Geo commands in the latest version, and I have added support for those in spring-data-redis. In addition, I am active on stackoverflow, mainly in JavaScript, Java and Spring topics.
• #4: Before we dive into the presentation, let me tell you about QuickBooks Self Employed, because it will help us understand our use case better. QuickBooks Self Employed helps self employed people – like artists, dancers, uber drivers, etc find tax deductions and be complaint with IRS rules and regulations. We do that by connecting to their bank and importing the bank transactions. Users then categorize these transactions as business or personal. Anything transaction categorized as a business expense can be a tax deduction. We have an interface so that users can pay the IRS their estimated taxes directly through our software. And we integrate with TurboTax for hassle free filing. In addition to our web presence, our iOS and Android apps help self employed people do mileage tracking. We launched a year and a half ago in the US, and have also launched the product in the UK, with more countries to follow shortly.
• #5: Let me demo you our iOS app to show what the product does. Users categorize these transactions as business or personal. Users track their mileage, so, for example if they are working for Uber they can deduct the miles they drove their car for work. We give the users an overview of their finances, because many users do not have a great way of know whether they have a profit or not. And we integrate with TurboTax (which is also an Intuit product), so that users can easily file their taxes.
• #6: Now, I’m going to demo two workflows in QuickBooks Self Employed that demonstrate how we use web sockets and Redis Pub/Sub in tandem. I’m just going to demo the UI, and as go further into the presentation, I’ll show you the architecture and deeper layers of our product. The first workflow is the Bank Connection Flow. After launching the app, the user is led to the ‘connect a bank account’ screen. The user selects a bank and enters his credentials. We support 10,000 plus banks and credit unions in the US. Connecting to a bank takes anywhere from 30 seconds to 5 minutes. During this time we’re communicating with a backend service that actually logs in to the bank and downloads transactions. This is a critical process in the user flow, almost a make or break point. We need to WOW the user, and make sure the user understands the value of our product. At this moment, we want the UI to be updated with what stage the back end is at in the process of connecting a bank.
• #7: After the user has started categorizing transactions, the product learns from the categorization, and recommends rules to help the user save time. So if, for example, the user categorizes 3 Wal-mart transactions as business, we understand that he or she is going to categorize the rest as business as well. So the product pops up a recommendation to the user.
• #8: Recommending rules to user has made our users very happy. When they start using the product, they learn the value right away. We have an over 60% rate of acceptance with our rule recommendation engine, and it is going up all the time. Best of all, it is a unified experience across web and mobile. The web as well as mobile apps use web-sockets to get the recommendations.
• #9: These two workflows are enabled by….. Web Sockets for communicating between the front end (browser) and the back end. Redis Pub/Sub for communicating between back end processes.
• #10: Some of you may be familiar with what websockets are, but let me spend a couple of minutes explaining: WebSockets is a application protocol, that works over TCP. The idea behind the WebSocket protocol consists of reusing the established TCP connection between a Client and Server. After the HTTP handshake, the Client and Server start speaking WebSocket protocol by exchanging WebSocket envelopes. What this means for us is that it allows two way communication in the browser. This allows us to communicate with the browser in message bus type fashion. Before web socket support, we used to have long polling and other stop gap measures, but they are not required anymore.
• #11: Now, let’s move to Redis Pub/Sub. Most of us use Redis for caching, but Redis also has a very good pub/sub messaging framework built in. The terminology that Redis uses is a ‘channel’ instead of a ‘queue’ or a ‘topic’. A publisher publishes to a channel. A Subscriber subscribes to a channel or a pattern. These commands are super fast, and all of the pub/sub commands take linear time, O(n+m), where N is the number of clients subscribed directly to the channel and M is the number subscribed patterns.
• #12: The architecture of our site. We have web app processes that serve requests coming in through our web app. We have API processes that serve API requests coming in through our mobile apps. We run embedded Tomcat 8 in both servers. We have worker processes who do the heavy lifting, allowing us to scale without having to increase our app and api processes. Intra process communication is done using Redis pub/sub, i.e. Web app and API processes communicate with the worker process using Redis Pub/Sub
• #13: Spring is used extensively on the backend. We use a lot more Spring projects than shown above.
• #14: SockJS is a browser JavaScript library that provides a WebSocket like object. Under the hood SockJS tries to use native WebSockets first. If that fails it can use a variety of browser-specific transport protocols and presents them through WebSocket-like abstractions. STOMP is a simple text messaging protocol, and stompJS implements that protocol. Spring supports it, and so we use it.
• #16: The big question, why redis for Pub Sub. Our use case is a little different from regular use cases for RabbitMQ. RabbitMQ can handle thousands of messages a second, but it is not built to handle hundreds of queues being created dynamically in a minute. We went to production with RabbitMQ. We have multiple app servers, but a web socket is created to only one app server. That server needs to listen to a message sent by the worker process. So we need a unique queue per user. RabbitMQ performance dropped drastically after around 200 queues. CPU spiked and we had to drop the idea of using RabbitMQ. Instead of being able to create ~50 queues / sec, throughput dropped to ~2 queues / sec.
• #17: Now, we’ll go through the entire workflow, of a user launching the app and seeing a recommendation.
• #18: Creating a web socket connection is not enough. Web-sockets connection opens a pipe to the server. However you want to publish to a topic. That way different parts of the code can use different topics, just like on the back end.
• #20: EnableWebSocketMessageBroker is the Spring annotation to enable broker backed mesasaging over webSocket. The MessageBrokerRegistry configuration reserves certain endpoints for websocket broker Stomp endpoint is ws-init, for regular cases
• #21: We’re implementing an ApplicationListener where we are listening to the SessionSubscribeEvent. When a SessionSubscribeEvent occurs, we create a Redis Pub Sub channel for the user. Every user has a unique PUBSUB channel. And only the App process that listens to the websocket subscribes to the Redis channel. For example, if there are 5 app server taking requests, the web socket is connected to only 1 app server, and only that app server will subscribe to the user’s PUBSUB channel.
• #22: During tax peak we had over 4k simultaneous users on our site.
• #24: When the worker process detects a possible rule, it publishes a message to the user’s topic.
• #27: An unexpected benefit of creating Redis channels whenever a user launches our app is that we can count the exact number of users who are on our site. Our DevOps has written a script that runs every minute and captures the number of live channels in Redis. The picture above is of a Splunk search where we have charted the number of users using our app over a 7 day period. Mind you this is after tax season is over. During tax season we had record traffic. Things like these help us understand how many users are using our web app or mobile apps at any given time, and how long they are staying on the site as well.
• #28: Websockets are supported in both iOS and Android. There are quite a few open source libraries that work great. We are using https://github.com/daltoniam/Starscream in iOS and https://github.com/koush/android-websockets for android. There isn’t good support for Stomp. But that is not an issue. Spring has support for just Websockets without stomp. You’ll just have to make your own protocol, and for our use case it is not a problem. http://stackoverflow.com/questions/27158106/websocket-with-sockjs-spring-4-but-without-stomp
• #31: Redis channel limits - https://groups.google.com/forum/#!topic/redis-db/G-rFG0k7NiY. At least up to 10k. We have tested until 5k simultaneous channels, our peak traffic has reached 4k simultaneous connections. Websockets for iOS and android – WebsocketStompKit (iOS), gozirra (android) while there is support, it is not well maintained. XMPP vs websocket - http://stackoverflow.com/questions/31108780/stomp-or-xmpp-over-websocket . Stomp is easier to implement in Spring