Rest in Practice, Brazil 2010
Download as PPTX, PDF•2 likes•534 views
This document summarizes a presentation about rearchitecting the systems of a large telecommunications company to handle a massive increase in message volume from 1 million to 1 billion transactions per month. The initial architecture involved many bottlenecks and single points of failure. By applying lessons from web-scale architectures and building services and platforms focused on HTTP APIs and statelessness, the new architecture achieved massive scalability increases at a fraction of the initially estimated cost.
Rest in Practice, Brazil 2010
- 2. REST in Practice Jim Webber http://jim.webber.name Twitter: @jimwebber
- 3. Or: A Billion Transactions Per Month Using the Web for Enterprise Heavy Lifting Jim Webber http://jim.webber.name Twitter: @jimwebber
- 5. Today’s Enterprise Architecture Accounting Product Development Marketing Support
- 7. “SOA: Principles of Service Design underwent a thorough technical review involving over 60 reviewers from different vendors, organizations, and professions across North America, Europe, and Asia. The book has been formally endorsed by members of major SOA vendors, including IBM, Microsoft, Oracle, BEA, and Intel.” -- Thomas Erl
- 9. Nirvana Enterprise Architecture, or… BPM Service Business Service Business Service Business Service ESB Basic Service Basic Service Basic Service Basic Service Basic Service
- 10. ESB - Erroneous Spaghetti Box? Enterprise Service Bus
- 14. But we still do it.
- 15. Why? Because it’s “less risky”
- 16. Why? Because that’s what the market does
- 17. Why? Because we need the -ilities
- 18. So let’s talk about those -ilities
- 21. Web Scale!
- 23. End to End Secure Messaging End to End Secure Conversations
- 24. Security Tokens and claims Federating access with tokens
- 32. <message name="SayHelloRequest"> <part name="firstName" type="xsd:string"/> </message> <message name="SayHelloResponse"> <part name="greeting" type="xsd:string"/> </message> <portType name="Hello_PortType"> <operation name="sayHello"> <input message="tns:SayHelloRequest"/> <output message="tns:SayHelloResponse"/> </operation> </portType> <binding name="Hello_Binding" type="tns:Hello_PortType"> <soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="sayHello"> <soap:operationsoapAction="sayHello"/> <input> <soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:examples:helloservice" use="encoded"/> </input> <output> <soap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:examples:helloservice" use="encoded"/> </output> </operation> </binding> <service name="Hello_Service"> <documentation>WSDL File for HelloService</documentation> <port binding="tns:Hello_Binding" name="Hello_Port"> <soap:address location="http://localhost:8080/soap/servlet/rpcrouter"/> </port> </service> Hello World! <?xml version="1.0" encoding="UTF-8"?> <definitions name="HelloService" targetNamespace="http://www.ecerami.com/wsdl/HelloService.wsdl" xmlns="http://schemas.xmlsoap.org/wsdl/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:tns="http://www.ecerami.com/wsdl/HelloService.wsdl" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
- 33. Media Types The Web is a coordination platform
- 35. A Case Study Client • UK-based telecoms service provider • Global customers (no easy downtime) Problem • Scalability and system resilience issues • Huge growth curve to from a million to a billion messages per month • Costs
- 36. Platform Architecture Carrier Carrier Carrier Carrier Where’s my file gone? Bottleneck Nightmare! Gateway Integration Database Shared Filesystem Post Processing Voice to Text Algorithms Fix things up here Shoehorn it in that other program
- 38. The £10* Million Solution “Queue” Database Gateway Carrier Carrier Carrier Carrier Shared Filesystem Enterprise Service Bus Post Processing * Solution does not include actual implementation Voice to Text Algorithms
- 40. ThoughtWorks was given the output from a previous enterprise consultancy… …and told to “do that, but cheaper”
- 42. Think differently! (or even just stop and think)
- 43. Architecture is still important
- 44. Agile EA: Prioritise Service Delivery Start by fixing storage Carrier Carrier Carrier Carrier Gateway Integration Database Shared Filesystem Post Processing Voice to Text Algorithms
- 49. Storage Manager Project Delivery Small team Java solution 3 Week Inception Performance testing throughout 14 iterations – See Jones and Kua paper, Agile 2009 Continuously built and deployed for testing
- 50. Web-friendly Architecture (mostly wallet-friendly too!) Encapsulates storage detail! Fast! Free! Cacheable! Jboss/Tomcat Optimised disk access! Storage Manager GPFS SAN Spend £ on the right Infrastructure Optimised server config
- 53. What is going on with these peaks? here?
- 54. Hypothesis Directories are becoming full, slowing random access to files
- 55. Prediction A single directory will result in linearly degrading performance
- 57. Outcome Adapt the directory structure to take advantage of the underlying file system
- 58. Improving Enterprise Architecture Carrier Carrier Carrier Carrier Reduced Load Gateway Scalable! Integration Database Storage Manager Post Processing Voice to Text Algorithms “She’ll be right” Just shoehorn it in that other program
- 59. Storage Manager Benefits • Immediate reduced load on integration database • Enabled removal of code from systems • Delivered business value by keeping customers happy • “The best software we have ever deployed” – Operations director I’ll come back to this
- 61. What if… We took our inspiration from successful Web-scale companies?
- 62. What if… And took the time to understand our SLAs?
- 63. What if… And picked the right technology solution? Not just the one we’re being sold?
- 64. What if… We built a scalable compute platform?
- 66. Grid Project Delivery Larger team Java solution 3.5 Week Inception Performance testing throughout 18 iterations Continuously built and deployed for testing
- 67. Web-friendly Architecture (very wallet-friendly!) Fault Tolerant ASR ASR ASR ASR Fault Tolerant Grid Pipeline Just HTTP Embedded Jetty Embedded Jetty Embedded Jetty Embedded Jetty Embedded Jetty Grid Manager Grid Manager Grid Manager Grid Manager Grid Manager Just HTTP Sustainable container for business logic ASR £0 Software Infrastructure ASR ASR Grid Pipeline ASR
- 68. Improving Enterprise Architecture Scalable Processing Platform Massively Reduced Load Carrier Carrier Carrier Carrier Gateway Scalable! Integration Database Storage Manager Post Processing Grid “She’ll be right” Scalable, sustainable!
- 69. Grid Benefits • • • • • Scalable, resilient platform Aligned with business goals Further reduced load on integration database Set architectural patterns Deliver business value by processing more messages at lower cost than ever before • “The best software we have ever deployed” – Operations director Better every time!
- 70. R$27,300,000 Up-Front Cost of Bus Architecture
- 71. R$2,730,000 Actual Cost of Completed Project
- 73. The Web works And its economies of scale eclipse your vendors’ http://amzn.to/cqBBiq
Editor's Notes
- #5: We’re doing SOA, but it’s painful and and risky. Our technology platform eclipses our business imperatives.The technical/organisational benefits hold true, but we want a better way to deliver
- #11: Manchester:There are folks that have made this work. Neil Walker (in the room?) has applied Guerrilla SOA style architecture and governance to this model successfully. This is an isolated success story however.
- #12: Your very own Minas Tirith!
- #21: Just buy more big iron and cluster it.Unthinking!Harder to achieve than “just throwing hardware at it”
- #22: Single set of widely adopted protocols and formatsLoosely coupled by defaultApplication model encourages federated caching
- #25: WS-Trust for tokensWS-federation for multiparty authentication and authorisation
- #28: A more modern French bridge!Architect: Norman Foster, and Englishman
- #29: TCP over TCP/IP?Reliable store and forward: think about crash recovery – who’s authoritativeBrokers? Scalability problemsProprietary interconnects? Depends if you can justify the cost!
- #34: Status codes: cordinationVerbs: reliabiityMedia types: service contracts!Hypermedia: protocolSemantics and Microformats: a business vocabulary for contracts
- #36: Voice to text conversation, thousands of conversions per second globally
- #37: High latency in the SLASometimes as much as 20 minutesLow contention problem domain, system highly amenable to parallelisation and horizontal scale outBut nothing actually built to exploit this property
- #39: An expensive act of faithThe Bus architecture “just works”No evidenceThe Bus architecture will scaleNo evidence for, much evidence againstThe bus architecture is simpleThe implementation isn’t though!
- #43: What if we drew our inspiration from successful Web-scale companies?What are our goals for growth? Reliability?Why not use a different technology?Why deliver incrementally?Guerrilla SOA!Start with scalable storage
- #44: Rethink designThink Web not EnterpriseThink about the customers, not just about ITAnd do it quickly because no design is complete without rapid validation through testing and implementation – NO NAVAL GAZING
- #45: Carrier gateway can be scaled with enterprise tech like queuesIntegration database too risky to touch, needs replacing in medium term, but in short term less load is goodPost processing out of scope for nowFile system is too slow, too messy, no governance, flakyVoice to text algorithms, no governance, no way to properly scale, no monitoring, no ability to adapt to changing workload
- #46: In one iteration, we created a skeletal storage serviceBased on Web standards, just like Amazon’s S3We used open source tools, libraries and operating systems
- #47: We put high loads through it, and measured its behaviourWe met out SLA. Easily.
- #49: We invested a single iterationWe got data that suggested our architectural approach would workCost of infrastructure, approximately $0We planned and built a system based on this empirical data
- #50: Small team4 devs, 1 architect, 1 analyst, 1 QAInceptionArchitecture, story gathering, estimation14 iterationsReporting Weekly showcasesTeam kanbanJava-based solutionBDD’d throughoutDSLs used for functional testing, very cool!Performance testing throughout!Jmeter, integrated into buildContinuously built and deployed for testingNo production surprises
- #60: Immediate reduced load on integration databaseHigher system stability/fewer disasters!Enabled removal of code from systemsLess to maintain, fewer places for bugs to hideDelivered business value by keeping customers happyMore messages got through, incidents decreased, carriers much happier
- #66: This time around, the technical approach was already understoodOur challenge was to build a larger system, with many more moving parts
- #67: Small team9 devs, 1 architect, 1 analyst, 2 QAInceptionArchitecture, story gathering, estimation14 iterationsReporting Weekly showcasesTeam kanbanJava-based solutionBDD’d throughoutBrowser-based testingDSLs used for functional testing, very cool!Performance testing throughout!Jmeter, integrated into buildContinuously built and deployed for testingNo production surprises
- #70: Scalable, resilient platformAdd more hardware, scale horizontallyHardware crashes, system continuesAligned with business goalsGrowthExceptional customer experienceLowering costsFurther reduced load on integration databaseHigher system stability/fewer disasters!Set architectural precedent Voice to text algorithms have a stable place to live, no longer shoe-horningA place for everything, and everything in its place
- #71: Remember the original cost? Around $25 million, without any implementation
- #72: Total cost of this projectAround $2.5 million. Complete, delivered, functioning.Delivering value to the business