Best Practices for Working with AI Virtual Assistants

Working with AI Agents in production isn’t trivial if you’re regulated. Over the past year, we’ve developed five best practices: 1. Secure integration. Not “agent over the top” integration - While its obvious to most you’d never send sensitive bank or customer information directly to a model like ChatGPT often “AI Agents” are SaaS wrappers over LLMs - This opens them to new security vulnerabilities like prompt injection attacks - Instead AI Agents should be tightly contained within an existing, audited, 3rd party approved vendor platform and only have access to data within that 2. Standard Operating Procedures (SOPs) are the best training material - They provide a baseline for backtesting and evals - If an Agent is trained on and follows that procedure you can then baseline performance against human agents and the AI Agents over time 3. Using AI Agents to power first and second lines of defense - In the first line, Agents accelerate compliance officer’s reviews, reducing manual work - In the second line, they provide a consistent review of decisions and maintain a higher consistency than human reviewers (!) 4. Putting AI Agents in a glass box makes them observable - One worry financial institutions have is explainability, under SR 11-7 models have to be explainable - The solution is to ensure every data element accessed, every click, every thinking token is made available for audit, and rationale is always presented 5. Starting in co-pilot before moving to autopilot - In co-pilot mode an Agent does foundational data gathering and creates recommendations while humans are accountable for every individual decision  - Once an institution has confidence in that agents performance they can move to auto decisioning the lower-risk alerts.

Over the last year, I’ve seen many people fall into the same trap: They launch an AI-powered agent (chatbot, assistant, support tool, etc.)… But only track surface-level KPIs — like response time or number of users. That’s not enough. To create AI systems that actually deliver value, we need 𝗵𝗼𝗹𝗶𝘀𝘁𝗶𝗰, 𝗵𝘂𝗺𝗮𝗻-𝗰𝗲𝗻𝘁𝗿𝗶𝗰 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 that reflect: • User trust • Task success • Business impact • Experience quality    This infographic highlights 15 𝘦𝘴𝘴𝘦𝘯𝘵𝘪𝘢𝘭 dimensions to consider: ↳ 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 — Are your AI answers actually useful and correct? ↳ 𝗧𝗮𝘀𝗸 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻 𝗥𝗮𝘁𝗲 — Can the agent complete full workflows, not just answer trivia? ↳ 𝗟𝗮𝘁𝗲𝗻𝗰𝘆 — Response speed still matters, especially in production. ↳ 𝗨𝘀𝗲𝗿 𝗘𝗻𝗴𝗮𝗴𝗲𝗺𝗲𝗻𝘁 — How often are users returning or interacting meaningfully? ↳ 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 𝗥𝗮𝘁𝗲 — Did the user achieve their goal? This is your north star. ↳ 𝗘𝗿𝗿𝗼𝗿 𝗥𝗮𝘁𝗲 — Irrelevant or wrong responses? That’s friction. ↳ 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗗𝘂𝗿𝗮𝘁𝗶𝗼𝗻 — Longer isn’t always better — it depends on the goal. ↳ 𝗨𝘀𝗲𝗿 𝗥𝗲𝘁𝗲𝗻𝘁𝗶𝗼𝗻 — Are users coming back 𝘢𝘧𝘵𝘦𝘳 the first experience? ↳ 𝗖𝗼𝘀𝘁 𝗽𝗲𝗿 𝗜𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 — Especially critical at scale. Budget-wise agents win. ↳ 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻 𝗗𝗲𝗽𝘁𝗵 — Can the agent handle follow-ups and multi-turn dialogue? ↳ 𝗨𝘀𝗲𝗿 𝗦𝗮𝘁𝗶𝘀𝗳𝗮𝗰𝘁𝗶𝗼𝗻 𝗦𝗰𝗼𝗿𝗲 — Feedback from actual users is gold. ↳ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 — Can your AI 𝘳𝘦𝘮𝘦𝘮𝘣𝘦𝘳 𝘢𝘯𝘥 𝘳𝘦𝘧𝘦𝘳 to earlier inputs? ↳ 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 — Can it handle volume 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 degrading performance? ↳ 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 — This is key for RAG-based agents. ↳ 𝗔𝗱𝗮𝗽𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗦𝗰𝗼𝗿𝗲 — Is your AI learning and improving over time? If you're building or managing AI agents — bookmark this. Whether it's a support bot, GenAI assistant, or a multi-agent system — these are the metrics that will shape real-world success. 𝗗𝗶𝗱 𝗜 𝗺𝗶𝘀𝘀 𝗮𝗻𝘆 𝗰𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗼𝗻𝗲𝘀 𝘆𝗼𝘂 𝘂𝘀𝗲 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀? Let’s make this list even stronger — drop your thoughts 👇

We’re entering an era where AI isn’t just answering questions — it’s starting to take action. From booking meetings to writing reports to managing systems, AI agents are slowly becoming the digital coworkers of tomorrow!!!! But building an AI agent that’s actually helpful — and scalable — is a whole different challenge. That’s why I created this 10-step roadmap for building scalable AI agents (2025 Edition) — to break it down clearly and practically. Here’s what it covers and why it matters: - Start with the right model Don’t just pick the most powerful LLM. Choose one that fits your use case — stable responses, good reasoning, and support for tools and APIs. - Teach the agent how to think Should it act quickly or pause and plan? Should it break tasks into steps? These choices define how reliable your agent will be. - Write clear instructions Just like onboarding a new hire, agents need structured guidance. Define the format, tone, when to use tools, and what to do if something fails. - Give it memory AI models forget — fast. Add memory so your agent remembers what happened in past conversations, knows user preferences, and keeps improving. - Connect it to real tools Want your agent to actually do something? Plug it into tools like CRMs, databases, or email. Otherwise, it’s just chat. - Assign one clear job Vague tasks like “be helpful” lead to messy results. Clear tasks like “summarize user feedback and suggest improvements” lead to real impact. - Use agent teams Sometimes, one agent isn’t enough. Use multiple agents with different roles — one gathers info, another interprets it, another delivers output. - Monitor and improve Watch how your agent performs, gather feedback, and tweak as needed. This is how you go from a working demo to something production-ready. - Test and version everything Just like software, agents evolve. Track what works, test different versions, and always have a backup plan. - Deploy and scale smartly From APIs to autoscaling — once your agent works, make sure it can scale without breaking. Why this matters: The AI agent space is moving fast. Companies are using them to improve support, sales, internal workflows, and much more. If you work in tech, data, product, or operations — learning how to build and use agents is quickly becoming a must-have skill. This roadmap is a great place to start or to benchmark your current approach. What step are you on right now?

After building 58 AI Agents, here are 12 essential tips (steal these and get ahead): 1. Delegate. - Stop thinking, “What manual processes can I automate?” - Instead, ask, “If I had a marketing agency, what would I want them to handle?” - Think bigger—AI isn’t just a time-saver, it’s a workforce multiplier. 2. Automation vs. AI Automation vs. AI Agents. - Automation: A series of steps executed automatically. - AI Automation: The same, but with an AI step. - AI Agents: Decide how to act, what to do, and what data to use. 3. AI Agents go beyond chat. 3 ways to trigger an Agent: - On demand (chat or button click). - On a webpage (via Chrome extension). - Via an event (just like an automation). 4. Use ChatGPT (or similar) to build. - Writing clear instructions (“prompts”) is harder than it looks. - Determining an Agent’s decision-making process is even harder. - ChatGPT is an essential tool for thinking through both. 5. There’s a fine line between useful and over-engineered. - Simple Agents get used. Complex ones get abandoned. - Start small—iterate later. - Traditional automation is no different. 6. Stronger use cases I’ve found: - Prioritizing feature requests based on product strategy - Pulling insights from a Zapier Table of consolidated data (cost savings, top-performing areas, etc.). - Researching a company, person, or product—then structuring the data and determining when to notify someone. 7. Use decision-making frameworks. - AI Agents, like humans, need structured decision-making. - MoSCoW, Eisenhower Matrix, SWOT—pick one and embed it. - You’ll understand why your Agent made a decision, not just what it did. 8. Data sources are the most powerful component. - Agents process large data sets instantly—that’s their edge. - The better your data, the better your Agent. - Build robust databases, and your Agents will thrive. 9. Agents need systems (just like you). - The future isn’t just Agents—it’s Agents + Tables + Workflows + Interfaces. - You’re not just automating—you’re designing an AI-powered organization. - Systems > Standalone Agents. 10. Two essential skills for building. - Delegating future work (that you've already done before). - Pushing the Agent to tackle tasks that haven’t been done before. - Both require serious brainpower and take time to master. 11. Set guardrails while also allowing for mistakes. - Restrict access in integrated apps to avoid risk. - Be okay with the Agent making some mistakes. - Master the balancing act to become an expert Agent builder. 12. The biggest bottleneck is you. - Are you clear on priorities? Goals? Expectations? - An Agent can only be as clear as you are. - Get your own systems right, and your AI will follow. One of the best skills you can learn in 2025 is Agent building. Models are getting better every. single. day. They'll do more and be smarter. Best way to learn: start building. Let's all learn together 💪 Consider subscribing to my newsletter: https://lnkd.in/gtxpSwap