WinOps - Lessons learned from Enterprise DevOps with Microsoft technologies - MS Ignite The Tour Edition

Editor's Notes

• #7: Anyone know who this guy is? TSB – 550 branches across UK, 8500 staff, acquired in 2015 by Spanish Bank Banco Sabadell for £1.7Bn 20th TSB April 2018 started a “migration” from their former parent Lloyds Banking Group platform to the (new) Sabadell Proteo banking platform Suffice to say… it didn’t go well… Up to 1.9 customers were locked out of their accounts for days. People didn’t get paid. Mortgages weren’t paid. Direct debits and standing orders weren’t paid. People could see other people’s accounts People were defrauded of money… Complete meltdown. Things were was so bad TSB had to call in IBM to help. So bad the CEO lost his job… <click> So bad that when I gave this talk back at Ignite in September the problems still hadn’t been fixed <click>
• #8: The FCA (the UK equivalent of the FSCA / SARB in RSA) are contemplating mandating MTTR in the Financial Services regulations. Partly as a result of this issue. But there have been many other cases of high profile failures And these failures have a real cost <click>
• #9: GBP 176M = $3Tn ZAR So potentially half a billion pounds (xxx USD) cost… which is 5x the £100M saving they were hoping to make from the migration <change topic> But it’s NOT only the CEO’s and CIO’s that have to worry about losing their jobs in this brave new world… the 4th industrial revolution as some people are calling it… Who here would call themselves an IT Pro / Operations person / systems administrator? So back in 2013 2 academics Frey and Osborne published a paper where they analysed the likelihood of job losses due to automation based on a number of key characteristics of the work. What they came up with was… <click>
• #10: 78% likelihood for “IT Operations technician” So if any of you were wondering whether you should get into devops and learn some developer skills it’s worth pointing out that… <click>
• #11: Developers are in the “quite unlikely” bucket… Speaking of DevOps – let’s whip into a DevOps 101 to understand what DevOps is and isn’t <click>
• #13: Before we dive into WHAT Devops is, and isn’t, it’s worth asking the question “Why do we need a new operating model for IT”? There are many factors as play but I’d like to highlight 3 factors that have a major influence on the need for change The first is Cloud Computing – cloud computing has fundamentally changed the underlying economics of computing. New entrants – digital native startups – no longer need to build their own data centres, buy servers, storage and networking in order to get started. They can use per minute billed, global scale IaaS, SaaS and PaaS services and scale their usage to meet customer demand The second is the rise of the Smartphone – “we’ve got an app for that” is now a ubiquitous expectation across almost all consumer and B2B sectors. This has put enormous pressure on IT to ship new features, and new back-end integrations services - (API’s) - into their often aging legacy software estates Thirdly is the changing demographics of “digital natives” and “millennials” who have grown up in a internet-enabled, smartphone-centric world. They expect “on demand” services, on their phones and laptops, with a seamless customer experience between the two, and ideally that seamless experience extending into the physical world, for example for bricks&mortar retailers. All of this combines into a wave of Digital Disruption & Innovation that needs new, faster and better ways of deliverying IT services. The old ways just won’t work any more…
• #14: John P Kotter – Harvard Emeritus professor and leading management theorist for 30 years – but it best when he said [quote] In turn, this lead IT practitioners and researchers to look at what the leading tech (and often digital native) IT companies were doing – the Googles, Amazons and Netflixes of the world. And what they found was… [click]
• #15: These organisations where DIFFERENT – as Gene Kim, (co-Author of “the phoenix project which I urge you all to read…) said they have different “cultural norms, processes and practices” Which leads to his 2018 definition of DevOps… <click>
• #16: Click Click Click Click Click So when we looked patterns practices and cultural norms we found they broke down into 5 key areas – which leads to what’s called the CALMS model of DevOps <click>
• #17: What they found was that high performing IT organisations were different in 5 key areas… A FOUNDATION of Culture – they embraced change, were more autonomous and more empowered 4 Pillars of core practices to support DevOps Transformation and high-performance IT… Automation – they were leveraging the latest automation tools like Puppet, Chef and Ansible Lean IT – they were embracing Lean IT principles borrowed from manufacturing (particularly automotive and the Toyota Production System e.g. Kanban) but more importantly focussing on small batch sizes and the flow of work through the system Measurement – they were using advanced measurement techniques to provide constant feedback on their customer needs, and the impact of their IT initiatives. Less opinion, more data science! Sharing – a focus on breaking down silos between departments and sharing knowledge and best practice, often using ideas drawn from open source software development. Let’s quickly drill down into 2 of those in a bit more detail – breaking down the silos and the flow of work… [click]
• #18: most traditional IT organisation are organised by TECHNICAL ROLE into a series of silos – each with a different waiting queue, with a different SLA, each focussing on optimising it’s own part of the end to end value stream. Think of it like a car engine – if your carburettor or fuel injection system only focussed on “its role” and tried to “maximise its utilisation” it would pour the maximum amount of fuel and air it can up to the limit of its designed capacity into the cylinder, regardless of the current throttle input or engine revs. End result – the engine stalls and dies. The system breaks. DevOps is systems thinking – let’s look at the OUTCOME we’re trying to achieve, and design a model optimise for the fast flow of work, normally with smaller batch sizes, and organised around multi-disciplinary teams. SO what does a product-aligned, multi-disciplinary DevOps team look like? [click]
• #19: The idea of a DevOps product delivery team is that they own a defined product from initial design and development right through into production deployment AND operations – Dev & Ops together in a vertically integrated team. The product owner is empowered to prioritise the backlog of work, based on the organisational goals, customer needs and the fast feedback of data from customers and product (remember the M for measurement in CALMS). The goal here is to balance out the “user stories” (new features) with the “Operability stories” (things that make it easier to manage, reduce outages or cloud hosting costs etc). All of this underpinned by a focus on automating as much as possible, to free up time for higher value work.
• #21: Where you start across the DevOps pipeline is very dependent on the organisation Where is your constraint?
• #23: In an agile way, delivering cumulative outcomes that reduce the cost of delay and get value into the hands of our customers faster But all of this requires behavioural change from ALL the stakeholders… <click>
• #25: Cloud Enabled DevOps Optimised Integrated Systems API’s Agile & Lean Process Skilled Workforce
• #26: The performance difference between high and low performers across 4 key metrics that measure Speed & Stability is stark. And this flows through into higher organisational performance. So they did this survey in 2019… anyone want to guess the numbers? <click>
• #27: In 2018 they introduced the Elite performance category… How much do you think Paul Pester would have paid to have 7x the change of keeping his job… Or instead of 159 days the recovery time being 159/2604 = 1.5hrs… Ok, so onto part 3 – lets look at some of this in the real world based on case studies presented over the last 4 years of WinOps conferences in London <click>
• #28: Justs worth mentioning that these case studies all come with YouTube links to the case study presentations so you can all watch them at your leisure, and the presentation deck will be available after the session via the Ignite portal and I will tweet out a link as well. <click>
• #29: https://www.youtube.com/watch?v=oWe-u4HDDac&feature=youtu.be https://www.slideshare.net/WinOpsConf/ian-margetts-asos-journey-to-continuous-deployment
• #30: Due to acquisition worldwide they were supporting 14 different ticket platforms… https://www.youtube.com/watch?v=PL0IEhpxNWQ&feature=youtu.be https://www.slideshare.net/WinOpsConf/connon-macrae-evolution-of-ticketmasters-journey-to-devops
• #31: https://www.youtube.com/watch?v=A-3PGmM4gHE&feature=youtu.be
• #32: https://www.youtube.com/watch?v=SNh7nNdLbqQ
• #34: Any objection based on not having the technology on Windows is no longer valid – all the major tools vendors and cloud vendors support Windows workloads and technologies.
• #35: But if you thought technology was the problem… then YOU are probably the problem 
• #36: Transformations, change initiatives and Projects fail… more than they succeed. So how can you make your transformation scalable so that even if it’s a “fail” at organisation level you’ve still had a lasting impact at the individual or team level… This is why there is a lot of emphasis on “ways of working” so we start this transformation at the lowest, most granular level.
• #37: Remember the model of behaviour change? We have to take people along this journey Understanding that different people will change at different rates. And some might not change at all <click>
• #38: What we often see is what we call the Icecream sandwich model of organisational change C suite want it because they don’t want to end up like Paul Pester… Engineering want it because they want to learn all those shiny new skills (cos we like shiny!) Middle Management… are terrified of letting go of command & control because they don’t know how to LEAD so they are frozen in fear…
• #39: DevOps is all about systems thinking and focusing on the constraints that affect the ENTIRE system and then focusing your pilots and changes at the point of constraint. And things like value-stream mapping and event storming are great here. It’s amazing the power of getting a diverse group of people into a room with whiteboards, post-its and sharpies and actually get them to map out their dependencies….
• #40: It’s AMAZING the power of getting a diverse group of people into a room with whiteboards, post-its and sharpies and actually get them to map out their dependencies…. A lot of what we [DevOpsGroup] do often feels more like marriage guidance counselling for dysfunctional organisations where the relationship has broken down and it’s heading for divorce (or in IT terms “outsourcing”) unless we can bridge these silos.
• #41: Start small – all the case studies we discussed started with small pilots, hack days, 1 team and so forth… Hot Tips - Find the biggest pain point that people have been whinging and moaning about for ever, fix that, call what you did to fix it DevOps and then rinse & repeat…
• #42: We have a four stage model – Concept, Pilot, Adopt, Practice
• #43: And the final key lesson from these case studies is that even though they were IT led projects, and sometimes they were fixing “IT problems”, they were all solving the BUSINESS problem that was being impeded by the IT constraint – shipping features faster, improving stability and scalability etc. <Click> This is the secret to selling the transformation to the wider business – “we want to build these new capabilities that are going to help you win customers, build better products, get products to market faster” – not “I want to move all these servers to Azure using Azure Migrate and Azure Site Recovery” [Changing topic] So changing topics a bit now to focus on the WinOps community as a whole and as a call to action to what you can do back in your own communities to drive the DevOps on Windows message forward… <click>
• #50: Microsoft is mobilising great resources to help along with the great new Microsoft Learn platform too.