Four Keys to Efficient DevOps

Editor's Notes

• #6: Sean: other “steak”-holders unrelated to the end user, DoD -> tailoring to teams ⇐ no longer a fan of his own question :-)James: what’s the most challenging aspects of improving developmentRaj: what does “automation” mean to you? Gaps come from diff. mindsets
• #8: Why is this a thing? Perspective Developers focus on the code QA focuses on the testing Product Manager focuses on product Communication is Fundamental to Success History of software development Requirements System Requirement Specification Use Cases User Stories DevOps Complexity grows exponentially with team size Mutual Expectations Change is Expected Teams Stakeholders Vision Product Roadmap Politics Points Raj can contribute to this slide -Often Testers are often put in a situation to give go/no-go which is not right. Testers provide relevant information about the product and then the business needs to make that decision based on the information. Testers are “Information brokers” not “Gate Keepers”
• #9: Definition of Definition of Done A clear, consistent yardstick yielding a binary answer Simple checklist Between 3 and 10 items Lightweight and FocusedHow Context Specific Team Driven Tailor based on context Dimensions of Done SDLC Development Testing Unit Functional Enterprise Project Management Quality Gates Architectural Review System Integration Deployment Product Shippable Customer/Client Dependencies Raj’s points -Have a checklist for Definition of Done and make it visible to the team -The story is complete only when all the points in the checklist are completed
• #10: Sean’s example of DoD: At one company I was at, we had a very light product department relative to developers so getting full requirements was always a challenge. We ended up settling on Acceptance Criteria as the main definition of done. This was a blend of user stories and testing behavior. Here’s an example for adding a Forgot Password link to login: Go to the Log In page. There should be a “Forgot Password” link. Click on the “Forgot Password” link. You should go to a “Forgot Password” page, and it should match the attached mockup. Enter an invalid email. The form should show an error message (specified elsewhere in the requirements) Enter an email of a user that doesn’t exist in the system. The form should show an error message. And so on...
• #11: Challenges Agreement on the definition Who Decides Buy in
• #13: CI · Frequency - Want the automated tests to run each time a check in happens · Visibility - No one knows what is going on and when test fails/pass · Reporting · Identifying/eliminating flaky tests Device/Simulator/Emulator · Where do you want to run this on device or simulators? · On Simulators difficult to tests o Wi-Fi/4g/Airplane mode switching (Simulator and emulators cannot tap into Airplane mode) Maintenance · Who maintains the automated tests? · Who maintains the server machines running the automated tests? What if you run out of space or want to run multiple instances of your tests o People are going to cloud testing for this exact same reason Test coverage · Have 2 sets of tests o Smoke tests – Most Stable and only high level functionalities and runs quickly o Regression tests/suites – Run during regression testing phase or periodically based on requirement Performance · How fast does your tests run? Does it take 5 mins, 30 mins, 1 day? o People want quick feedback on this
• #14: Raj talks about Need quick feedback if something fails/broken. Find defects “Early” Need to keep everyone involved in automation effort as it is a “Whole Team” approach not just the automation team . Make it visible to everyone through e-mails, TV boards, stand ups etc Make people respect the automated tests. If it fails, whoever broke if fixes it. It is not only the automation team members who keep fixing it. Have stable tests and remove flaky tests. Increase your and the automated tests credibility Sean can comment on: As a developer, I can attest to trust for automated tests being key. If you have a test that is failing sporadically for a reason other than broken functionality, disable the test until it can be fixed. You don’t want get developers used to seeing tests fail because if they see too many failing and they are not responsible for the failures, developers get the habit of being skeptical towards test failures - or even flat-out ignoring them. James: Automation empowers the team to find more complicated or subtle defects and perform more robust edge case testing The early identification of defects significantly reduces cost to remediate
• #15: Raj talks about -How automation gaps are addressed during planning meetings -Who is there in the planning meetings - Whole Team includes PO/Solution architect/Tech Doc writer/Dev’s/QA/s -Examples on this slide --Through discussion how we identified not to automate “Guide Story” as there was another story which included complete redesign of the guide - Devs pointed it out and thus, it helps to plan automation as with the whole team. Saved a lot of time for automation testers --Automation needs considerable work/coding. Treat it a separate stories. Have “kick off” meetings on expectations/acceptance criteria, then have development, Code review, Testing, Demo and then Complete just like a normal development story --”Buffer Time” - Under-stimated different data setup needed for different types of products - Movies with different captions, Movies with different audio lang, Single Episode, mulit episode, rental products for one of the stories. Learnt a valuable lesson of under estimating.
• #19: We use a lot of industry standard tools and cloud services here at Perfecto. Everyone has their tools,
• #24: The goal of autonomy is to let people and processes fly with as little interference as possible Autonomy = end-to-end ownership of how to accomplish the goals that leadership sets Ever worked as the sole developer on a project or at a brand new company? Development goes very fast compared to larger companies. Part of that is because there are no existing systems, but part of it is that a lot of other stuff becomes involved outside of code. Two types of autonomy: Process autonomy (automation): construction of systems that minimize the time and error associated with repeated delivery; enables faster work Cultural autonomy: The business identifies goals, the team figures out how to accomplish them. Aligns team effort to delivering value through a great customer/end-user experience.
• #25: Construction of systems that minimize the error associated with repeatable work. Minimize error, enable faster work. There are many things that fall into this, and you can fall anywhere on the range: Automated Builds and Deployments - Maybe you don’t have anything. Maybe you have automated builds but they need to be manually deployed, etc. Environments: maybe there’s no instructions for developers on how to create a local development environment. Maybe you have a staging environment that simulates production. Documentation: if you were new, how much information can you find out without having to ask someone who already knows it? And is it easy enough to get to that information? There are other examples: Automated UI tests, Unit tests What you need is very custom to your team and project: If you have a big team, just a small change in some of these can save significant time If you’re a small team, you may not need these Prioritize based on your team and tackle one thing at a time. To give an example, I have a project right now that is a small application that only needs to be up for three hours for a live event. Everything is manual - I upload files to the server when I change them, I manually SSH into the server and restart it. Doing automated builds/deployments there would be a waste of my time.
• #26: Developers probably don’t use the product, or if they do, they don’t with the same mindset as an end user. It’s a culture thing, so must be slowly implemented over time Share end-user data with developers: email campaign results, A/B testing results, interesting GA data At one startup I was at in the entertainment industry, the analytics team sent out a bi-weekly report with the stats relevant to each dev team and an explanation of any changes. A/B testing results were available on the internal wiki. Let developers sit in on end user interaction: feedback sessions, focus groups, demos Product can share their notes from these sessions Share business objectives and priorities with developers: High-level business priorities (ex, quarterly goals) KPIs of features after rollout (ex, is conversion up after changing our contact form design?) Be aware of non-end user stakeholders: Investors External people w/purchasing power for your product At another startup in the energy sector, I’d sit in on demos to hear what people commented on. At one SaaS company, we had a feature that was never used but was in the product because people with purchasing power liked to see that it was there on demo.