Decomposing applications for scalability and deployability (devnexus 2013)
6 likes3,841 views
The document discusses the benefits of decomposing applications into microservices for improved deployability and scalability, using Cloud Foundry as a platform. It outlines obstacles associated with monolithic architectures, such as deployment difficulties and coordination challenges, while providing strategies for service communication and design. The advantages include independent service development and deployment, as well as fault isolation and technology flexibility.
![Example micro-service
require 'sinatra'
post '/' do
phone_number = params[:From]
registration_url = "#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}"
<<-eof
<Response>
<Sms>To complete registration please go to #{registration_url}</Sms>
</Response>
eof
end
For more on micro-services see
@fgeorge52
@crichardson](https://image.slidesharecdn.com/decomposingapplicationsdevnexus2013-130220162955-phpapp02/85/Decomposing-applications-for-scalability-and-deployability-devnexus-2013-28-320.jpg)
![Using Scala futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ... Two calls execute in parallel
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d And then invokes D
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D
@crichardson](https://image.slidesharecdn.com/decomposingapplicationsdevnexus2013-130220162955-phpapp02/85/Decomposing-applications-for-scalability-and-deployability-devnexus-2013-54-320.jpg)
![One command to create platform
services and deploy application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK vmc push:
Binding Service [si-mongo]: OK
•Reads the manifest file
•Creates the required platform services
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
•Deploys all the applications
Processing resources: OK
Packing application: OK
Uploading (12K): OK
Push Status: OK
Staging Application 'inventory': OK
Starting Application 'inventory': OK
Pushing application 'store'...
@crichardson](https://image.slidesharecdn.com/decomposingapplicationsdevnexus2013-130220162955-phpapp02/85/Decomposing-applications-for-scalability-and-deployability-devnexus-2013-80-320.jpg)
![Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading Application:
Checking for available resources: OK
Packing application: OK
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK
Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2
@crichardson](https://image.slidesharecdn.com/decomposingapplicationsdevnexus2013-130220162955-phpapp02/85/Decomposing-applications-for-scalability-and-deployability-devnexus-2013-84-320.jpg)
Decomposing applications for scalability and deployability (devnexus 2013)
- 1. Decomposing applications for deployability and scalability Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson [email protected] http://plainoldobjects.com @crichardson
- 2. Presentation goal How decomposing applications improves deployability and scalability and How Cloud Foundry helps @crichardson
- 3. About Chris @crichardson
- 4. (About Chris) @crichardson
- 5. About Chris() @crichardson
- 6. About Chris @crichardson
- 7. About Chris http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/ @crichardson
- 8. vmc push About-Chris Developer Advocate Signup at http://cloudfoundry.com @crichardson
- 9. Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps @crichardson
- 10. Let’s imagine you are building an e-commerce application @crichardson
- 11. Traditional web application architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache Database Inventory Service Shipping Service Simple to develop Tomcat test deploy scale @crichardson
- 12. But there are problems with a monolithic architecture @crichardson
- 13. Users expect a rich, dynamic and interactive experience h oug d en goo HTTP Request is n’t e ec tur Java Web hit Browser HTML/Javascript Application I arc ty le U s Old Real-time web ≅ NodeJS @crichardson
- 14. Intimidates developers @crichardson
- 15. Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure Fear of change Updates will happen less often e.g. Makes A/B testing UI really difficult @crichardson
- 16. Overloads your IDE and container Slows down development @crichardson
- 17. Obstacle to scaling development Accounting E-commerce Engineering application Shipping team @crichardson
- 18. Obstacle to scaling development WAR UI team StoreFrontUI Accounting Accounting Inventory team InventoryService Shipping team Shipping @crichardson
- 19. Obstacle to scaling development But I want the backend is not working to update the UI yet! Lots of coordination and communication required @crichardson
- 20. Requires long-term commitment to a technology stack @crichardson
- 21. Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps @crichardson
- 22. @crichardson
- 23. The scale cube Y axis - functional decomposition Scale by sim ing splitting r n ila litt rtitio different things a ng ing by ta p s da sp Sc is - thi ax ale Z X axis - horizontal duplication @crichardson
- 24. Y-axis scaling - application level WAR StoreFrontUI Accounting Service Inventory Service Shipping Service @crichardson
- 25. Y-axis scaling - application level accounting web application Accounting Service Store front web application inventory web application StoreFrontUI Inventory Service shipping web application Shipping Service Apply X axis cloning and/or Z axis partitioning to each service @crichardson
- 26. Partitioning strategies... Partition by verb, e.g. shipping service Partition by noun, e.g. inventory service Single Responsibility Principle Unix utilities - do one focussed thing well @crichardson
- 27. ...Partitioning strategies Too few Drawbacks of the monolithic architecture Too many - a.k.a. Nano-service anti-pattern Runtime overhead Potential risk of excessive network hops Potentially difficult to understand system Something of an art @crichardson
- 28. Example micro-service require 'sinatra' post '/' do phone_number = params[:From] registration_url = "#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}" <<-eof <Response> <Sms>To complete registration please go to #{registration_url}</Sms> </Response> eof end For more on micro-services see @fgeorge52 @crichardson
- 29. Service deployment options Isolation, manageability VM Linux Container/LXC JVM JAR/WAR/OSGI bundle/... Density/efficiency @crichardson
- 30. Real world examples http://techblog.netflix.com/ http://highscalability.com/amazon-architecture http://www.addsimplicity.com/downloads/ eBaySDForum2006-11-29.pdf http://queue.acm.org/detail.cfm?id=1394128 @crichardson
- 31. There are drawbacks @crichardson
- 32. Complexity See Steve Yegge’s Google Platforms Rant re Amazon.com @crichardson
- 33. Multiple databases & Transaction management @crichardson
- 34. Implementing features that span multiple services @crichardson
- 35. When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful @crichardson
- 36. But there are many benefits Scales development: develop, deploy and scale each service independently Update UI independently Improves fault isolation Eliminates long-term commitment to a single technology stack Modular, polyglot, multi- framework applications @crichardson
- 37. Two levels of architecture System-level Services Inter-service glue: interfaces and communication mechanisms Slow changing Service-level Internal architecture of each service Each service could use a different technology stack Pick the best tool for the job Rapidly evolving @crichardson
- 38. If services are small... Regularly rewrite using a better technology stack Adapt system to changing requirements and better technology without a total rewrite Pick the best developers rather than best <pick a language> developers polyglot culture @crichardson
- 39. The human body as a system @crichardson
- 40. 50 to 70 billion of your cells die each day @crichardson
- 41. Yet you (the system) remain you @crichardson
- 42. Can we build software systems with these characteristics? http://dreamsongs.com/Files/ DesignBeyondHumanAbilitiesSimp.pdf http://dreamsongs.com/Files/WhitherSoftware.pdf @crichardson
- 43. Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps @crichardson
- 44. Inter-service communication options Synchronous HTTP asynchronous AMQP Formats: JSON, XML, Protocol Buffers, Thrift, ... Even via the database Asynchronous is preferred JSON is fashionable but binary format is more efficient @crichardson
- 45. Asynchronous message-based communication wgrus-billing.war Accounting Service wgrus-store.war wgrus-inventory.war RabbitMQ StoreFrontUI (Message InventoryService MySQL Broker) wgrus-shipping.war ShippingService @crichardson
- 46. Benefits Decouples caller from server Caller unaware of server’s coordinates (URL) Message broker buffers message when server is down/ slow Supports a variety of communication patterns, e.g. point- to-point, pub-sub, ... @crichardson
- 47. Drawbacks Additional complexity of message broker Request/reply-style communication is more complex @crichardson
- 48. Writing code that calls services @crichardson
- 49. The need for parallelism Service B b = serviceB() Call in parallel c = serviceC() Service A Service C d = serviceD(b, c) Service D @crichardson
- 50. Java Futures are a great concurrency abstraction http://en.wikipedia.org/wiki/ Futures_and_promises @crichardson
- 51. Using Java Futures public class Client { private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy; Eventually contains result public void doSomething() throws ... { Future<Integer> result = executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } }); /// Do other things When needed wait for result int r = result.get(500, TimeUnit.MILLISECONDS); System.out.println(r); } } @crichardson
- 52. Better future implementations Guava ListenableFutures = better Scala’s composable Futures = really good Java 8 CompletableFuture = great @crichardson
- 53. Composable Futures val f1 = Future { ... ; 1 } val f2 = Future { ... ; 2 } Transforms Future val f4 = f2.map(_ * 2) assertEquals(4, Await.result(f4, 1 second)) Combines two futures val fzip = f1 zip f2 assertEquals((1, 2), Await.result(fzip, 1 second)) http://doc.akka.io/docs/akka/2.0.1/scala/futures.html @crichardson
- 54. Using Scala futures def callB() : Future[...] = ... def callC() : Future[...] = ... def callD() : Future[...] = ... Two calls execute in parallel val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d And then invokes D val result = Await.result(future, 1 second) http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D @crichardson
- 55. Spring Integration Provides the building blocks for a pipes and filters architecture Enables development of application components that are loosely coupled insulated from messaging infrastructure Messaging defined declaratively @crichardson
- 56. Handling partial failures Down? Slow? Service A Service B Down? Slow? @crichardson
- 57. About Netflix > 1B API calls/day 1 API call average 6 service calls Fault tolerance is essential http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html @crichardson
- 58. How to run out of threads HTTP Request X X X Thread 1 Thread 2 Thread 3 X Thread n Execute thread Service A X Service B pool Eventually all threads If service B is down then thread Tomcat will be blocked will be blocked @crichardson
- 59. Their approach Network timeouts and retries Invoke remote services via a bounded thread pool Use the Circuit Breaker pattern On failure: return default/cached data return error to caller https://github.com/Netflix/Hystrix See my talk tomorrow for more details @crichardson
- 60. Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps @crichardson
- 61. Presentation layer evolution.... WAR StoreFrontUI HTML / HTTP View Controller Browser Model @crichardson
- 62. ...Presentation layer evolution Browser Web application Static View Controller content JSON-REST RESTful Model Endpoints HTML 5 - Events Event JavaScript publisher No elaborate, server-side web framework required @crichardson
- 63. How to publish events to browser? @crichardson
- 64. NodeJS is the fashionable technology @crichardson
- 65. Why NodeJS? Familiar Javascript High-performance, scalable event-driven, non-blocking I/O model Compact runtime Over 17,000 modules developed by the community Simple event publishing using socket.io or SockJS @crichardson
- 66. Why not NodeJS? a.k.a. callback hell @crichardson
- 67. A modern web application Browser Node JS RESTful WS Service 1 HTML 5/ Server Java application Service 2 Script Events Socket.io Socket.io client server Service 3 @crichardson
- 68. NodeJS - using RESTful WS and AMQP REST Service REST Requests Node JS Events socket.io AMQP AMQP RabbitMQ Service @crichardson
- 69. Socket.io server-side var express = require('express') , http = require('http') , amqp = require(‘amqp’) ....; Handle server.listen(8081); socket.io ... var amqpCon = amqp.createConnection(...); connection io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(message.data.toString()); Republish socket.emit(‘tick’, m); as socket.io }; amqpCon.queue(“”, {}, event function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); }); Subscribe to }); AMQP queue https://github.com/cer/nodejs-clock @crichardson
- 70. Socket.io - client side <html> <body> The event is <span data-bind="text: ticker"></span> <script src="/socket.io/socket.io.js"></script> <script src="/knockout-2.0.0.js"></script> <script src="/clock.js"></script> Bind to model Connect to </body> </html> socket.io clock.js var socket = io.connect(location.hostname); function ClockModel() { self.ticker = ko.observable(1); Subscribe socket.on('tick', function (data) { self.ticker(data); to tick event }); }; Update ko.applyBindings(new ClockModel()); model @crichardson
- 71. Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps @crichardson
- 72. Original architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache Database InventoryService Shipping Tomcat @crichardson
- 73. Modern architecture p, lo Native Mobile HTML5 mobile Desktop Browser application application ve StoreUI StoreUI StoreUI st de NodeJS ? Asynchronous, Javascript is scalable communication StoreUI te e th w RabbitMQ oy do Spring/Scala pl Standalone web application Inventory Shipping “headless” ow Service Service de Spring/Java applications H d MySQL an Billing Service Customer Redis Inventory Mongo Order Database Database Database @crichardson
- 74. Traditional tools: monolithic applications @crichardson
- 75. Developing modular apps is more difficult Many more moving parts to manage Platform services: SQL, NoSQL, RabbitMQ Application services: your code Who is going to setup the environments: the developer sandbox? ... But Cloud Foundry helps... @crichardson
- 76. Easy polyglot application deployment and service provisioning OSS community Ap p lica 'o vFabric Postgres Private n S Clouds Data Services erv vFabric Public i ce Msg RabbitMQTM Clouds Services In Micro ter Other Services Clouds fac Additional partners e services … @crichardson
- 77. Creating a platform service instance $ vmc create-service mysql mysql1 Creating Service: OK $ vmc services ...... =========== Provisioned Services ============ +-------------+---------+ | Name | Service | +-------------+---------+ | mysql1 | mysql | +-------------+---------+ @crichardson
- 78. Multi-application manifest - part 1 --- applications: inventory/target: Path to application name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required platform services si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis @crichardson
- 79. Multi-application manifest - part 2 store/target: Path to application name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 Required platform services services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq @crichardson
- 80. One command to create platform services and deploy application $ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'... Creating Application: OK Creating Service [si-rabbit]: OK Binding Service [si-rabbit]: OK Creating Service [si-mongo]: OK vmc push: Binding Service [si-mongo]: OK •Reads the manifest file •Creates the required platform services Creating Service [si-redis]: OK Binding Service [si-redis]: OK Uploading Application: Checking for available resources: OK •Deploys all the applications Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OK Staging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'... @crichardson
- 81. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS @crichardson
- 82. Using Caldecott to tunnel into your services @crichardson
- 83. Caldecott = TCP over HTTP native protocol native HTTP Service Caldecott Caldecott protocol Service client Port gem application NNN Your computer Cloud Foundry @crichardson
- 84. Using Caldecott… $ vmc tunnel 1: mysql-135e0 2: mysql1 Which service to tunnel to?: 2 Password: ******** Stopping Application: OK Redeploying tunnel application 'caldecott'. Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OK Push Status: OK Binding Service [mysql1]: OK Staging Application: OK Starting Application: OK Getting tunnel connection info: OK Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50 Starting tunnel to mysql1 on port 10000. 1: none 2: mysql Which client would you like to start?: 2 @crichardson
- 85. …Using Caldecott Launching 'mysql --protocol=TCP --host=localhost --port=10000 -- user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50' Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 10944342 Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188 Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> @crichardson
- 86. Running JUnit test with Caldecott Configure your test code to use port + connection info @crichardson
- 87. Summary @crichardson
- 88. Monolithic applications are simple to develop and deploy BUT have significant drawbacks @crichardson
- 89. Apply the scale cube Modular, polyglot, and scalable applications Services developed, deployed and scaled independently @crichardson
- 90. Cloud Foundry helps .js Ap p e lica Private fac Data Services 'o ter Clouds n S r In erv ide Public i ov ce Msg Services In Pr Clouds ter ud Cl o fac Micro e Other Services Clouds @crichardson
- 91. @crichardson [email protected] http://plainoldobjects.com Questions? www.cloudfoundry.com @crichardson