Software Engineering Principles

"Service reliability math that every engineer should know" I think it's useful for engineers to understand what uptime and reliability mean in practice. These numbers paint a good picture of what's involved :) Now while service reliability is often reduced to a simple percentage, the reality is far more nuanced than those decimal points suggest. First, not all downtime is created equal. A single 8-hour outage has dramatically different business implications than 480 one-minute outages, even though both sum to the same annual downtime. This distinction is particularly relevant when considering service level agreements (SLAs) and how they’re measured. The impact of downtime also varies significantly based on when it occurs. Five minutes of downtime during peak business hours might cost more than an hour of downtime during off-hours. This temporal aspect of reliability is often overlooked in simple percentage calculations. Each additional nine of reliability typically requires an order of magnitude more engineering effort and operational complexity. Moving from 99.9% to 99.99% isn’t just a matter of being "10 times more reliable" – it often requires fundamental architectural changes: At 99.9% (8h 45m downtime/year), you might get away with single-region deployment and basic failover At 99.99% (52m 35s), you’re typically looking at multi-region deployment, sophisticated health checking, and automated failover At 99.999% (5m 15s), you need redundancy at every layer, real-time monitoring, and likely some form of active-active deployment At 99.9999% (31s), you’re dealing with advanced techniques like chaos engineering, automated canary deployments, and sophisticated traffic management While understanding the basic math of service reliability is crucial, the real engineering challenge lies in understanding the context, trade-offs, and business implications of reliability decisions. The next time you see a reliability requirement, don’t just think about the percentage – think about the entire socio-technical system required to achieve and maintain that level of service. The numbers are simple. The engineering reality behind them is anything but. #softwareengineering #programming

🪂 How To Make Your Design System AI-Ready (https://lnkd.in/dtnpy7CM), a practical guide on how to reduce drifts, minimize mistakes, maintain context and improve the quality of AI-generated prototypes — with structured spec files, automated auditing and token layers. Put together by Hardik Pandya from Atlassian. --- 🔹 1. Design Decisions Are Infrastructure AI-generated prototypes often don't deliver consistently decent results because of tiny inconsistencies scattered all across a design system. Often it's decisions made but not documented, hard-coded values never cleaned up, or relying too much on AI making sense of mock-ups or design flows on its own. Unsurprisingly, better AI prototypes come from better data — but also from better human guidance. We shouldn’t assume that AI knows how to choose the right component, and how to design with accessibility in mind. It needs priorities, a clear path on how we make decisions, design principles, examples, do's and don'ts. In fact, we should treat design decisions as infrastructure. That means that every time we make a decision — not just a design decision, but even decision on how actually prioritize our work and how we make decisions around here — it must find a path into the spec file that is then consumed by AI. --- 🔶 2. Three Layers: Spec Files + Token Layer + Audit To ensure quality, we establish design principles, guidelines, rules in a form of “spec files”). It's structured Markdown files that include spacing rules, color choices, component usage guidelines, priorities etc. AI is going to read and reuse that spec file every time it's going to generate a prototype. Because the spec files are text files, it's much more cost-effective, but also much more accurate just because we don't rely on AI recognizing or decoding patterns from mock-ups, but gets specific guidelines instead. In fact, extending code is often a more effective way than generating code from mock-ups. Token layer lists and keeps updated all tokens used throughout the design system. AI always chooses from a closed set of named variables instead of inventing plausible values ad-hoc. An audit script catches what AI gets wrong. It scans the prototype and flags every hard-coded value and flags it if necessary. It can be a regular software doing that, with AI waiting for its feedback to come back. Finally, when a design system ships updates, a sync routine flags which spec files need updating. The goal is to make sure that AI always reads up-to-date, current specs, not the ones written against an outdated version. --- 🔺 3. Examples of AI-Ready Design Systems ⌾ Atlassian: https://lnkd.in/dVsGc3Cp ⌾ Carbon: https://lnkd.in/d4zq4WWb ⌾ CMS Design System: https://lnkd.in/dHHzV3en ⌾ Nordhealth: https://lnkd.in/d8C4j2ZA Yet again, AI can’t magically resolve technical debt or design debt — it needs guidance, decisions, priorities and principles.

AI Product Management AI Product Management is evolving rapidly. The growth of generative AI and AI-based developer tools has created numerous opportunities to build AI applications. This is making it possible to build new kinds of things, which in turn is driving shifts in best practices in product management — the discipline of defining what to build to serve users — because what is possible to build has shifted. In this post, I’ll share some best practices I have noticed. Use concrete examples to specify AI products. Starting with a concrete idea helps teams gain speed. If a product manager (PM) proposes to build “a chatbot to answer banking inquiries that relate to user accounts,” this is a vague specification that leaves much to the imagination. For instance, should the chatbot answer questions only about account balances or also about interest rates, processes for initiating a wire transfer, and so on? But if the PM writes out a number (say, between 10 and 50) of concrete examples of conversations they’d like a chatbot to execute, the scope of their proposal becomes much clearer. Just as a machine learning algorithm needs training examples to learn from, an AI product development team needs concrete examples of what we want an AI system to do. In other words, the data is your PRD (product requirements document)! In a similar vein, if someone requests “a vision system to detect pedestrians outside our store,” it’s hard for a developer to understand the boundary conditions. Is the system expected to work at night? What is the range of permissible camera angles? Is it expected to detect pedestrians who appear in the image even though they’re 100m away? But if the PM collects a handful of pictures and annotates them with the desired output, the meaning of “detect pedestrians” becomes concrete. An engineer can assess if the specification is technically feasible and if so, build toward it. Initially, the data might be obtained via a one-off, scrappy process, such as the PM walking around taking pictures and annotating them. Eventually, the data mix will shift to real-word data collected by a system running in production. Using examples (such as inputs and desired outputs) to specify a product has been helpful for many years, but the explosion of possible AI applications is creating a need for more product managers to learn this practice. Assess technical feasibility of LLM-based applications by prompting. When a PM scopes out a potential AI application, whether the application can actually be built — that is, its technical feasibility — is a key criterion in deciding what to do next. For many ideas for LLM-based applications, it’s increasingly possible for a PM, who might not be a software engineer, to try prompting — or write just small amounts of code — to get an initial sense of feasibility. [Reached length limit. Full text: https://lnkd.in/gYY-hvHh ]

🚨 Zero Trust for AI Agents Anthropic just released "Zero Trust for AI Agents." As we're thinking about agentic permissions, applying a Zero Trust discipline is critical to secure adoption. AI agents interpret goals, call tools, chain actions, delegate to other agents, and maintain context across sessions. The trust surface is different. The paper introduces "least agency" — a concept that OWASP has been promoting — and the distinction from least privilege is worth sitting with. 👉 "Least privilege" asks what an identity can access. 👉 "Least agency" asks what an agent can do, under what conditions, with which tools, and with what level of oversight. Autonomous agents introduce action risk alongside access risk — and the boundaries around behavior need to be architecturally enforced, not assumed. The paper includes a design test worth writing down: 🔥 Does the control make the attack impossible, or merely tedious?🔥 The practical controls follow directly from Zero Trust fundamentals — cryptographic agent identity, short-lived credentials, tool allow-listing, sandboxed execution, and full traceability from prompt to action to outcome. None of this is new doctrine. It's existing architecture applied to a harder problem. Full disclosure: the paper cites NIST SP 800-207 on Zero Trust Architecture and the CISA Zero Trust Maturity Model, both of which I co-authored during my time supporting Federal Zero Trust efforts at CISA. Zero Trust is built for a world where we have to remove implicit trust. Agentic AI is the next version of that same problem — valid identities, valid credentials, legitimate-looking actions, and still no basis for assumed trust. Access is earned. Actions are constrained. Agency must be governed. 👉🏼 Link to Anthropic's paper in the comments.

Anthropic 𝗷𝘂𝘀𝘁 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝗱 𝗮 𝗱𝗲𝗻𝘀𝗲 𝗮𝗻𝗱 𝗵𝗶𝗴𝗵𝗹𝘆 𝗽𝗿𝗮𝗰𝘁𝗶𝗰𝗮𝗹 𝗿𝗲𝗽𝗼𝗿𝘁 𝗼𝗻 𝗵𝗼𝘄 𝘁𝗼 𝗯𝘂𝗶𝗹𝗱 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 — 𝗽𝗮𝗰𝗸𝗲𝗱 𝘄𝗶𝘁𝗵 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀 𝗳𝗿𝗼𝗺 𝗿𝗲𝗮𝗹-𝘄𝗼𝗿𝗹𝗱 𝗱𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁𝘀: ⬇️ Not just marketing, BUT a real, practical blueprint for developers and teams building AI agents that actually work. It explains how Claude Code (tool for agentic coding) can function as a software developer: writing, reviewing, testing, and even managing Git workflows autonomously. BUT in my view: The principles and patterns described in this document are not Claude-specific. You can apply them to any coding agent — from OpenAI’s Codex to Goose, Aider, or even tools like Cursor and GitHub Copilot Workspace. 𝗛𝗲𝗿𝗲 𝗮𝗿𝗲 7 𝗸𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀 𝗳𝗼𝗿 𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗯𝗲𝘁𝘁𝗲𝗿 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 — 𝘁𝗵𝗮𝘁 𝘄𝗼𝗿𝗸 𝗶𝗻 𝘁𝗵𝗲 𝗿𝗲𝗮𝗹 𝘄𝗼𝗿𝗹𝗱: ⬇️ 1. 𝗔𝗴𝗲𝗻𝘁 𝗱𝗲𝘀𝗶𝗴𝗻 ≠ 𝗷𝘂𝘀𝘁 𝗽𝗿𝗼𝗺𝗽𝘁𝗶𝗻𝗴 ➜ It’s not about clever prompts. It’s about building structured workflows — where the agent can reason, act, reflect, retry, and escalate. Think of agents like software components: stateless functions won’t cut it. 2. 𝗠𝗲𝗺𝗼𝗿𝘆 𝗶𝘀 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 ➜ The way you manage and pass context determines how useful your agent becomes. Using summaries, structured files, project overviews, and scoped retrieval beats dumping full files into the prompt window. 3. 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴 𝗶𝘀𝗻’𝘁 𝗼𝗽𝘁𝗶𝗼𝗻𝗮𝗹 ➜ You can’t expect an agent to solve multi-step problems without an explicit process. Patterns like plan > execute > review, tool use when stuck, or structured reflection are necessary. And they apply to all models, not just Claude. 4. 𝗥𝗲𝗮𝗹-𝘄𝗼𝗿𝗹𝗱 𝗮𝗴𝗲𝗻𝘁𝘀 𝗻𝗲𝗲𝗱 𝗿𝗲𝗮𝗹-𝘄𝗼𝗿𝗹𝗱 𝘁𝗼𝗼𝗹𝘀 ➜ Shell access. Git. APIs. Tool plugins. The agents that actually get things done use tools — not just language. Design your agents to execute, not just explain. 5. 𝗥𝗲𝗔𝗰𝘁 𝗮𝗻𝗱 𝗖𝗼𝗧 𝗮𝗿𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀, 𝗻𝗼𝘁 𝗺𝗮𝗴𝗶𝗰 𝘁𝗿𝗶𝗰𝗸𝘀 ➜ Don’t just ask the model to “think step by step.” Build systems that enforce that structure: reasoning before action, planning before code, feedback before commits. 6. 𝗗𝗼𝗻’𝘁 𝗰𝗼𝗻𝗳𝘂𝘀𝗲 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝘆 𝘄𝗶𝘁𝗵 𝗰𝗵𝗮𝗼𝘀 ➜ Autonomous agents can cause damage — fast. Define scopes, boundaries, fallback behaviors. Controlled autonomy > random retries. 7. 𝗧𝗵𝗲 𝗿𝗲𝗮𝗹 𝘃𝗮𝗹𝘂𝗲 𝗶𝘀 𝗶𝗻 𝗼𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻 ➜ A good agent isn’t just a wrapper around an LLM. It’s an orchestrator: of logic, memory, tools, and feedback. And if you’re scaling to multi-agent setups — orchestration is everything. Check the comments for the original material! Enjoy! Save 💾 ➞ React 👍 ➞ Share ♻️ & follow for everything related to AI Agents!

“𝗔𝗜 𝗶𝘀 𝗻𝗼𝘁 𝗮𝗯𝗼𝘂𝘁 𝗮𝗽𝗽𝘀… 𝗮𝗻𝗱 𝗶𝘁’𝘀 𝗱𝗲𝗳𝗶𝗻𝗶𝘁𝗲𝗹𝘆 𝗻𝗼𝘁 𝗮𝗯𝗼𝘂𝘁 𝗽𝗿𝗼𝗺𝗽𝘁𝘀.” This MIT lecture quietly does something most AI content never does. It forces you to stop thinking about tools for a minute and ask a much harder question: what is computation, really? It starts like a normal lecture. Then, before you know it, it is dismantling the way we talk about intelligence, learning, abstraction, and even what we think machines are doing when they “think.” 🎩 And just when you think MIT cannot get any more MIT… the professor puts on a wizard hat and turns eval and apply into something that feels half computer science, half spell-casting. Strange. Brilliant. Oddly unforgettable. 💡 𝗪𝗵𝘆 𝗱𝗼𝗲𝘀 𝘁𝗵𝗶𝘀 𝗺𝗮𝘁𝘁𝗲𝗿? Because too many people are building AI careers on surface-level fluency. They know the tools. They know the demos. They know the buzzwords. But the foundations? That is often where the silence begins. And that is risky. We keep using labels that sound far more advanced than they really are: → Artificial intelligence is not truly intelligent → AI agents do not really have agency → Machines do not “learn” the way people imagine they do That is why lectures like this matter so much. They take you beneath the hype and back to the layer that actually lasts: → abstraction → evaluation → computation To me, that is the real divide in AI now. Some people are learning how to use the latest tools. Others are learning how to understand what those tools are really doing. The second group will build the future. The first group will keep reposting it. What do you think matters more in AI right now: mastering the tools, or understanding the foundations underneath them? #AI #ArtificialIntelligence #ComputerScience #MIT #MachineLearning #Innovation #Technology #FutureOfWork #Learning

The era of "move fast and bill the VC" for AI development is officially over. And the bills are coming due. Oscar Le recently shared this screenshot (see image) after their team burned through 500 premium requests on a $7,000 yearly Cursor AI team subscription... in a single day. This isn't just about one Cursor's pricing change. It's a flashing red light for engineering leadership and developers building with AI today. For a long time, the true cost of AI has been masked by VC-subsidized API calls and "unlimited" plans. Those days are disappearing very quickly. P.S. Optimal AI has an unlimited plan because we use smart model routing techniques, our agents are multi “model”. This is how we can continue to provide quality outputs at a drastically lower cost. This is what the new reality is: The real cost of a request: What does an API call to the latest model actually cost? It's no longer a simple flat rate. The shift to token-based pricing means complex queries on powerful models can now cost orders of magnitude more than simple ones. Predictability is out the window. The end of subsidized experimentation: Many AI-powered tools that we've come to love were essentially burning venture capital to acquire users. As the market matures, these companies are being forced to pass the real costs onto their customers. The "rug pull" is the new business model. The talent tax is real: The most significant cost in any AI project isn't the API bill, it's the highly specialized (and expensive) talent required to build, fine-tune, and manage these systems effectively. As users, we need to shift our mindset from purely focusing on the magic of AI to also managing its economics. We must instill a culture of cost-awareness in our teams. This means: Deeper cost monitoring and attribution: Who on your team is running up the bill? What features are the most expensive to run? You can't manage what you can't measure. Smarter model routing: Is the latest and greatest model always necessary? Or can a smaller, more efficient model get the job done for a fraction of the cost? The "vibe" of building with AI was fun while it lasted. Now it's time to get serious about the P&L. The companies that thrive in this next chapter of AI will be the ones that master the art of building both innovative and economically viable products. What did your last AI bill look like and what are you doing to manage the rising costs of AI in your teams?

𝗟𝗟𝗠 𝘃𝘀. 𝗥𝗔𝗚 𝘃𝘀. 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 𝘃𝘀. 𝗔𝗴𝗲𝗻𝘁 𝘃𝘀. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 — 𝗪𝗵𝗲𝗻 𝘁𝗼 𝘂𝘀𝗲 𝘄𝗵𝗮𝘁 I keep getting one question from teams building with GenAI: Which approach should we choose? This one-pager visual breaks down the trade-offs. Below is the practical guide I use on real projects. 𝟭) 𝗟𝗟𝗠 What it is: Prompt → model → answer. Use when: General knowledge, ideation, drafting, small utilities. Watch out for: Hallucinations on domain-specific facts; limited to model’s pretraining. 𝟮) 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻) What it is: Query → retrieve context from a knowledge base → feed context + query to LLM → grounded answer. Model weights don’t change. Use when: You have proprietary docs, policies, catalogs, tickets, or logs that change frequently. Benefits: Lower cost than training, auditable sources, fast updates. Key tips: Good chunking, embeddings, metadata, and re-ranking determine quality more than the LLM choice. 𝟯) 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 What it is: Train the model on input→output pairs to change its weights (LLM → LLM′). Use when: You need consistent style, domain tone, or task-specific behavior (classification, templated replies, structured outputs). Benefits: Lower prompt complexity, stable behavior, smaller inference tokens. Caveats: Needs clean, labeled data; versioning and evaluation are critical. 𝟰) 𝗔𝗴𝗲𝗻𝘁 What it is: LLM + memory + tools/APIs with a think → act → observe loop. Use when: Tasks require multi-step reasoning, tool use (search, SQL, APIs), or state over time. Examples: Troubleshooting flows, data enrichment, workflow automation. Risks: Loops, tool misuse, latency. Use guardrails, timeouts, and action limits. 𝟱) 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 (𝗠𝘂𝗹𝘁𝗶-𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺𝘀) What it is: Coordinated roles (planner, executor, critic) that plan → act → observe → learn from feedback. Use when: Complex processes with decomposition, review, and collaboration across specialized agents. Examples: Customer ops copilots, multi-step ETL with validation, enterprise workflows spanning multiple systems. Challenges: Orchestration, determinism, monitoring, and cost control. Metrics that matter Grounding: citation hit-rate, answer verifiability (RAG) Quality: task accuracy, pass@k, error rate Efficiency: latency, tokens, cost per resolution Safety: hallucination rate, tool misuse, policy violations Reliability: determinism, replayability, test coverage Design Tips: Start with RAG before touching fine-tuning; data beats weights early on. Keep prompts short; push knowledge to the retriever or the dataset. Add evaluation harnesses from day one (gold sets, unit tests for prompts/tools). Log everything: context windows, actions, failures, and human overrides. Treat agents like software: versioning, guardrails, circuit breakers, and audits.

Rick Rubin went on stage in Helsinki the day after my talk. Someone asked how he resolves creative differences with artists. His answer was simple: change the conversation from "I disagree" to "let's build it." Then he shared a story: An artist played him a song. The transition didn't work. Rubin told him so. The artist said, "We'll just cut that part in half." Rubin thought to himself: What a dumb idea. But he didn't say that. He said, "Let's try it." The artist played it. It worked. Rubin is a legend. He's produced everyone from Johnny Cash to Jay-Z. Instead, he bit his tongue and let the artist prove him wrong. The principle: when you make an idea tangible, it stops being the person's idea. It becomes something you can both look at objectively and improve together. Once you build it, the truth is obvious. Here's what this looks like in practice: Your designer wants to change the entire homepage layout. You think it's too risky. Instead of three meetings debating it, you say: "Let's build a prototype and test it with 50 users this week." Your sales team wants to restructure the pricing page. Instead of blocking it because you're worried about conversions, you say: "Let's run it as an A/B test on 20% of traffic for two weeks." Your engineer wants to rebuild a core feature from scratch. You think it's overengineered. But instead of killing it in the planning phase, you say: "Spike it out in three days and show me if the performance gain is real." You're not saying yes to everything. You're saying, "Let's find out." Rubin also said something that stuck with me: "If there's disagreement, I always side with the artist's vision. Because to them, it's their career. To me, it's just one piece of my portfolio." Most leaders think backing down makes them look weak. Rubin knows that siding with the person who has the most at stake makes better work happen. Your job isn't to be right. It's to create the conditions where the best idea wins. Stop debating. Start building. P.S. This insight is from this week's newsletter where I break down why Yamaha dominates while Steinway got sold to private equity: https://lnkd.in/efSqP_9K P.P.S. Access additional research links, the podcast, and the full archive in the first comment 👇 Thank you to Nordic Business Forum!

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?