Design and implementation patterns for reviving relational monoliths

Editor's Notes

• #3: Aqueducts (ACK-wa-ducts) got their name from the Latin word for water, aqua, and the Latin word for channel, ductus. They built the first Roman aqueduct in 312 BC. by the time of the Empire, three hundred years later, most Roman towns had at least one aqueduct to bring in fresh water, and big cities like Rome had ten or more. 2000 years later, many of these are still amazing to see, but also still function —- pretty sure the questionable plumbing in my hotel this week isn’t going to last quite that long…
• #4: Wondering if we should just stay on the previous slide…and talk to these points We have a number of years experience working on large enterprise software development projects In these environments, there is often a combination of customer apps, and large packaged software offerings Often there is limited test coverage, and minimal automation As a quick aside – this isn’t a criticism of enterprise software or packaged software -- legacy software gets a bad name...one definition is “scary to make changes to” or ”lacking test coverage” --- but another very valid definition is “software that makes money”....for it to have lasted long to be legacy, must have done something right... In this type of environment – what is the best way to move forward We’ll walk you through a number of techniques that we’ve had success with (or that sounded cool...), and talk about how you’d get started with them
• #7: Hari
• #10: Responsive: The system responds in a timely manner if at all possible. - notice issues quickly, and resolve efficiently. Priority on consistent behavior Resilient: Continuing functioning, even when there are issues with dependencies replication, containment, isolation, delegation -> don't push your failures to your clients ensuring that parts of the system can fail and recover without compromising the system as a whole Elastic: responsive under changing demand. elasticity in a cost-effective way on commodity hardware and software platforms. Message Driven: Reactive Systems rely on asynchronous message-passing to establish a boundary between components that ensures loose coupling, isolation and location transparency. This boundary also provides the means to delegate failures as messages. Employing explicit message-passing enables load management, elasticity, and flow control by shaping and monitoring the message queues in the system and applying back-pressure when necessary. Location transparent messaging as a means of communication makes it possible for the management of failure to work with the same constructs and semantics across a cluster or within a single host. Non-blocking communication allows recipients to only consume resources while active, leading to less system overhead. Let's think about how to accomplish this: • We can become modular if our system is event-driven. We can divide the system into multiple micro-services/components/modules that are going to c ommunicate with each other using notifications. This way, we are going to react to the data flow of the system, represented by notifications. • To be scalable means to react to the ever-growing data, to react to load without falling apart. • Reacting to failures/errors will make the system more fault-tolerant. • To be responsive means reacting to user activity in a timely manner. Reactive programming—focusing on computation through ephemeral dataflow chains—tend to be event-driven, while reactive systems—focusing on resilience and elasticity through the communication, and coordination, of distributed systems—is message-driven Messages have a clear (single) destination, while events are facts for others to observe. In a reactive system, especially one which uses reactive programming, both events and messages will be present—as one is a great tool for communication (messages), and another is a great way of representing facts (events). Reactive programming is programming (same name, slightly different concept) with asynchronous data streams. asynchronous event stream, on which you can observe and do some side effects. anything can be a stream: variables, user inputs, properties, caches, data structures, etc. For example, imagine your Twitter feed would be a data stream in the same fashion that click events are. You can listen to that stream and react accordingly. On top of that, you are given an amazing toolbox of functions to combine, create and filter any of those streams. That's where the "functional" magic kicks in. A stream can be used as an input to another one. Even multiple streams can be used as inputs to another stream. You can merge two streams. You can filter a stream to get another one that has only those events you are interested in. You can map data values from one stream to another new one.
• #13: Command Query Responsibility Segregation forcing the same conceptual model for commands and queries, makes the model much more complex — and forces trade-offs CQRS allows you to separate the load from reads and writes allowing you to scale each independently. 
• #14: One area where we’ve seen great success with implementing a version of this is with product availability. By splitting out the commands that require updates to the inventory management system, from those that are reads….we’re able to dramatically improve response time of lookups, scale to meet the inventory system needs more specifics on this example….. very similar
• #19: Proxy Pattern Provide a surrogate or placeholder for another object to control access to it. Use an extra level of indirection to support distributed, controlled, or intelligent access. Add a wrapper and delegation to protect the real component from undue complexity.
• #20: This is becoming an expectation, rather than an aspiration — but it’s worth pointing out again Much of the pain associated with monoliths, isn’t the actual code, but instead the processes around it get rid of the snowflake servers — all configuration and setup needs to be version controlled tests need to provide confidence that you have a working system, and then need to run quickly if a portion of your path to production is painful — make it part of the daily routine….repetition will force the issue, and help to motivate resolution (don’t mean that quite as evilly as it sounds) consider phoenix environments — should be able to rebuild a new env quickly There is a huge amount of hidden costs associated with not doing this if you can’t run the monolith locally (or in an isolated per dev env) — you will have conflicts, and delays impacting multiple people fear of change creates tech debt — you don’t refactor because you don’t know if you’ve broken something new dev on boarding shouldn’t involve ritual sacrifice of farm animals — should be scripted (or containerized) and included in an easy to follow readme — make people excited to work on the code, not a punishment… keep implementations small, make rollback fast and predictable You have to start somewhere — make a test with the change you implement today, your future self will appreciate Automate test flows for high value flows —> customer can always checkout…etc
• #21: A number of options….blue/green or canary? In some cases, platforms can make this very easy for MS...but what about monoliths? You have options — need to think about tradeoffs ….feature toggles ….read/write separation