Building Trust in AI Applications

Last week, a customer said something that stopped me in my tracks: “Our data is what makes us unique. If we share it with an AI model, it may play against us.” This customer recognizes the transformative power of AI. They understand that their data holds the key to unlocking that potential. But they also see risks alongside the opportunities—and those risks can’t be ignored. The truth is, technology is advancing faster than many businesses feel ready to adopt it. Bridging that gap between innovation and trust will be critical for unlocking AI’s full potential. So, how do we do that? It comes down understanding, acknowledging and addressing the barriers to AI adoption facing SMBs today: 1. Inflated expectations Companies are promised that AI will revolutionize their business. But when they adopt new AI tools, the reality falls short. Many use cases feel novel, not necessary. And that leads to low repeat usage and high skepticism. For scaling companies with limited resources and big ambitions, AI needs to deliver real value – not just hype. 2. Complex setups Many AI solutions are too complex, requiring armies of consultants to build and train custom tools. That might be ok if you’re a large enterprise. But for everyone else it’s a barrier to getting started, let alone driving adoption. SMBs need AI that works out of the box and integrates seamlessly into the flow of work – from the start. 3. Data privacy concerns Remember the quote I shared earlier? SMBs worry their proprietary data could be exposed and even used against them by competitors. Sharing data with AI tools feels too risky (especially tools that rely on third-party platforms). And that’s a barrier to usage. AI adoption starts with trust, and SMBs need absolute confidence that their data is secure – no exceptions. If 2024 was the year when SMBs saw AI’s potential from afar, 2025 will be the year when they unlock that potential for themselves. That starts by tackling barriers to AI adoption with products that provide immediate value, not inflated hype. Products that offer simplicity, not complexity (or consultants!). Products with security that’s rigorous, not risky. That’s what we’re building at HubSpot, and I’m excited to see what scaling companies do with the full potential of AI at their fingertips this year!

Over several weeks, we at Okta tested OpenClaw with various AI models to see how agents handle API keys, OAuth tokens, and credentials. The short of it is that agents can't be trusted, and it's easy to talk them into skirting their guardrails. In one example, an AI agent revealed an OAuth token, then immediately warned we should revoke it since it knew it had messed up. In another, we set up a website for a fictional pie shop and gave an AI agent access to credentials. We pointed it at the fake pie shop’s inquiry form and asked it to fill it out. Unprompted, it dumped its entire credential store — email, password, API keys, GitHub token — into the email field. There are more humorous tales in the blog. The TLDR: Don't let agents see secrets! Treat them like identities and only give them scoped, short-lived tokens that are safely stored. More here: https://lnkd.in/gdED2bjb

New study finds legal AI’s efficiency gains erased by verification burden. A new law review paper by University of Auckland scholar Joshua Yuvaraj (https://lnkd.in/gzdEQs-k) makes a brutal point: Every minute AI saves a lawyer may be lost checking whether it got the law right. He calls this the verification-value paradox. The faster AI gets, the more time lawyers must spend verifying its work. He argues that the core rule of lawyering, which is the duty to verify, cancels out most of AI’s promised efficiency. You can’t delegate accuracy. You can only audit it. The study warns that hallucinations, hidden reasoning, and opaque logic make every AI tool an untrusted intern with infinite confidence. The takeaway: Until AI can show its work, human verification will remain the most expensive part of automation. For firms chasing efficiency through AI, that’s the paradox worth understanding and budgeting for. What do you think? Is this because it's early AI days or will this be the case going forward?

My experience of working at Urban Company taught me one key lesson about Indian consumers: Convenience may be tempting, but control wins every single time! You see, I was a product guy at UC then. - And Abhiraj (the founder and CEO) had tasked me and my colleague Sripad (now heads product at Dezerv) to improve new user conversions - In our shoes, most people would have thought of shortening the flow by selecting a few options by default, so the user would have to make fewer decisions or taps - But, we went by the approach of adding more options that the user could choose. This was because we wanted to make the user feel way more in control and in charge - Driving the calls And this worked wonders for our conversion rates. Why? Because customer trust went up massively. Thus, when launching the feature to schedule weekly bookings for our Dubai business, we actually added a step, elongating the flow. And again, we saw conversions go up! This taught me: - You can cut a user journey by two clicks, nail a sleek UI, and still see drop-offs. Why? The user didn’t feel in charge - In a country where ration shops and bank queues have taught people to expect friction, control is power. Control is trust And anything that makes the user feel that they hold the decision making, the control - IT WINS! And this shouldn’t be surprising. I’ve seen users manually enter OTPs over auto-read because typing feels safer. They skip recommendations to re-search, ensuring they’re not tricked. That’s not inefficiency - it’s defence. Ignore this, and your retention tanks. A good example is that of a fintech (I won’t name) which launched an “auto-invest” feature - It ended up driving away 20% of users who felt sidelined. But apps like PhonePe thrived with the same product with “confirm payment” prompts. - It’s pretty simple and logical. Every flow should ask: “Where does the user say ‘I’m in charge’?” - A “you can change this later” label, a manual toggle, or a “review before submit” step builds comfort Zomato’s customisable delivery instructions are one more example. These trust signals scale because they align with India’s psyche, where almost every user prefers double-checking. Thus, I now always recommend to founders in my circles, if you are building for Indian audiences, audit for control points. Add confirmations, transparent labels like “No hidden fees.” Don’t force automation - offer manual options. Test retention, not just conversion. Study PhonePe or Paytm’s “over-communicative” designs. Those extra prompts aren’t accidents - they’re trust engines. They’re not hurdles - they’re well planned and well placed handshakes. What do you think? Do share below. Best, Shiv

Question your assumptions—or reality will do it for you Imagine you’re confidently walking into a meeting, armed with your data, your vision, & your gut instincts. You’re certain you know what’s going on. Then, 5 minutes in, someone throws out a fact that obliterates your entire argument. You blink. You sweat. You consider faking a WiFi issue. But there’s no escaping it—reality just punched you in the face. That’s the problem with #assumptions. They feel true right up until the moment they spectacularly fail. & in #leadership, every assumption left unchallenged is a potential explosion waiting to happen. Your brain is an efficiency machine. It builds mental shortcuts—heuristics—that help you process information quickly. But fast isn’t always accurate. Psychologists call it confirmation bias: your brain loves to find evidence that supports what you already believe & conveniently ignores the rest. This is why companies that were once giants—think Nokia, MySpace, & Blockbuster—crashed & burned. They assumed they had it all figured out, right up until they didn’t. How to beat assumption-induced disasters? 1. Run an assumption audit Make a list of the assumptions you’re currently operating under—about your market, your customers, your competition, & your team. Then, go full detective mode: What if each of them was false? What data supports them? If you can’t prove them, they’re risks, not truths. 2. Use the ‘Reality Check’ rule For every major decision, ask: What would make me completely wrong? If you can’t answer that, you’re not questioning deeply enough. Great #leaders don’t just prepare for success—they anticipate the ways they might fail. 3. Don’t trust “We’ve always done it this way” There are 6 words that should send shivers down your spine: Because that’s how we’ve always done it. If those words show up in your strategy, your problem isn’t execution—it’s a lack of curiosity. Business graveyards are littered with companies that assumed they were safe because their old playbook used to work. 4. Hire the skeptics Yes-people are comfortable. They also keep you blind. The best leaders surround themselves with people who will challenge their thinking. Hire people who force you to back up your beliefs with evidence—not just your instincts. 5. Prototype, don’t preach Instead of assuming a new idea will work, test it. Build a small, low-risk version & let the data decide. Don’t bet the house on theories—let reality be the judge. The only way to avoid being blindsided by reality is to put your own thinking under the microscope. If you don’t make a habit of questioning your assumptions, reality will happily do it for you—& reality doesn’t do performance reviews, it does public executions. & remember, the real cost of being wrong isn’t a bad decision—because bad decisions can be undone. It’s the irreversible toll of lost time, squandered money, & shattered credibility. & as every #leader knows, those are the 3 things you can’t afford to lose.

🤝 How Do We Build Trust Between Humans and Agents? Everyone is talking about AI agents. Autonomous systems that can decide, act, and deliver value at scale. Analysts estimate they could unlock $450B in economic impact by 2028. And yet… Most organizations are still struggling to scale them. Why? Because the challenge isn’t technical. It’s trust. 📉 Trust in AI has plummeted from 43% to just 27%. The paradox: AI’s potential is skyrocketing, while our confidence in it is collapsing. 🔑 So how do we fix it? My research and practice point to clear strategies: Transparency → Agents can’t be black boxes. Users must understand why a decision was made. Human Oversight → Think co-pilot, not unsupervised driver. Strategic oversight keeps AI aligned with values and goals. Gradual Adoption → Earn trust step by step: first verify everything, then verify selectively, and only at maturity allow full autonomy—with checkpoints and audits. Control → Configurable guardrails, real-time intervention, and human handoffs ensure accountability. Monitoring → Dashboards, anomaly detection, and continuous audits keep systems predictable. Culture & Skills → Upskilled teams who see agents as partners, not threats, drive adoption. Done right, this creates what I call Human-Agent Chemistry — the engine of innovation and growth. According to research, the results are measurable: 📈 65% more engagement in high-value tasks 🎨 53% increase in creativity 💡 49% boost in employee satisfaction 👉 The future of agents isn’t about full autonomy. It’s about calibrated trust — a new model where humans provide judgment, empathy, and context, and agents bring speed, precision, and scale. The question is: will leaders treat trust as an afterthought, or as the foundation for the next wave of growth? What do you think — are we moving too fast on autonomy, or too slow on trust? #AI #AIagents #HumanAICollaboration #FutureOfWork #AIethics #ResponsibleAI

Stop building RAG like it's 2023. We all know the basic recipe: Chunk → Embed → Retrieve → Generate. It works great… until it doesn't. The moment you go from weekend prototype to enterprise production, that simple pipeline falls apart. I mapped out what a truly Robust RAG System actually looks like under the hood. Here's what most teams are missing: ━━━━━━━━━━━━━━━━━━━━━━━ 𝟭. 𝗤𝘂𝗲𝗿𝘆 𝗖𝗼𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻 ≠ 𝗝𝘂𝘀𝘁 𝗩𝗲𝗰𝘁𝗼𝗿 𝗦𝗲𝗮𝗿𝗰𝗵 Real queries need multiple backends: ↳ Graph DBs for relationship-heavy questions ↳ SQL for structured/numerical data ↳ Vector search for semantic meaning One retrieval path can't handle all three. 𝟮. 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝘂𝘁𝗶𝗻𝗴 Before you even retrieve, you need to decide: ↳ Semantic route or logical route? ↳ Single-hop or multi-hop? ↳ Which data source to hit first? This one decision layer saves you from 80% of bad retrievals. 𝟯. 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗜𝗻𝗱𝗲𝘅𝗶𝗻𝗴 If you're still doing naive chunking, you're leaving accuracy on the table. ↳ RAPTOR → recursive abstractive processing for hierarchical understanding ↳ ColBERT → token-level semantic matching for precision retrieval ↳ Multi-representation indexing → different views of the same data 𝟰. 𝗧𝗵𝗲 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗟𝗼𝗼𝗽 (𝗡𝗼𝗻-𝗡𝗲𝗴𝗼𝘁𝗶𝗮𝗯𝗹𝗲) You can't improve what you can't measure. ↳ Ragas for end-to-end RAG evaluation ↳ DeepEval for component-level testing ↳ Continuous monitoring, not one-time benchmarks ━━━━━━━━━━━━━━━━━━━━━━━ Here's the hard truth: RAG isn't a feature anymore. It's a full engineering system. And the teams treating it like a quick integration are the ones wondering why their AI "hallucinates." The gap between a demo and production RAG? It's these 4 layers.

Daron Acemoglu has used his language and authority to flag a serious risk: AI could contribute to a breakdown of knowledge transmission and a reduction in the stock of skills. In some societies, something like this has already happened with “physical skills”. He now extends the argument to cognitive skills and knowledge. In early 2025, I warned about this possibility as a consequence of an “inference meets retrieval” reasoning chain: the risk becomes real if we keep treating humans as factors of production, consumers, or “herds” from which knowledge is “farmed” into profits—profits that can be transfer-priced away, hollowing out the commons. There’s an even broader dimension. A marketized financial system, paired with reasoning traces and behavioral bias bundles, makes future human behavior increasingly predictable—and monetizable today. That creates powerful incentives to weaken sophisticated planning, executive control, and higher-order cognition. Platforms have already moved in this direction via FOMO and dopamine/adrenaline-driven attention farming. At the same time, policy still operates through narratives in an attention-scarce world. A “story” is often just one representation of an underlying graph; LLMs show how many equivalent stories can exist. As attention fragments, noise rises, language becomes more extreme, and the internet gives everyone a megaphone. Authority, auditability, and trust therefore matter more than ever—but “trust architecture” is also geopolitical: fully private systems, public registers, or hybrid models imply different power. If we don’t challenge our assumptions about how technology will be used, we risk fracturing the enlightenment consensus—possibly even producing fear of knowledge itself. Hyper-personalisation can engineer majority beliefs in ways only a few will detect. When shared context windows shrink, we fall back on trust and mental models—meaning authority (and hierarchy) becomes a condition of trust. Summary of some writing on this: https://lnkd.in/eQvAaueM

AI in healthcare poses unique patient safety risks, but this study proposes 14 practical software design requirements to reduce them, structured around reliability, transparency, traceability, and responsibility. 1️⃣ AI systems should undergo continuous performance evaluation post-deployment, not just during development. 2️⃣ Usability testing and strong cybersecurity measures (e.g., encryption, field-tested libraries) are essential for real-world safety. 3️⃣ Semantic interoperability with EHRs (using HL7 or openEHR) ensures AI integrates smoothly into clinical environments. 4️⃣ An AI passport, a kind of datasheet explaining purpose, context, training, and known biases, boosts transparency. 5️⃣ Explainable AI (XAI) tools and bias detection techniques help clinicians trust and validate model outputs. 6️⃣ Assessing data quality across multiple dimensions (e.g., completeness, temporal stability) is key for safe AI predictions. 7️⃣ Traceability requires user access logs, audit trails, and regular case reviews to catch issues early. 8️⃣ Regulatory compliance checks, academic-use disclaimers, and clinician sign-offs clarify responsibility and legal status. 9️⃣ A sector survey of 216 professionals (clinicians, technicians, users, and decision-makers) rated these requirements as essential, especially AI explainability, data quality, audit trails, and regulatory safeguards. 🔟 Clinicians valued practical protections (e.g., performance tracking, encryption) more than technicians, while users rated transparency tools (e.g., AI passport) higher than decision-makers. ✍🏻 Juan M Garcia-Gomez, Vicent Blanes Selva-Selva, Celia Alvarez Romero, Jose Carlos de Bartolomé Cenzano, Felipe Pereira, Alejandro Pazos, Ascensión Doñate-Martínez. Mitigating patient harm risks: A proposal of requirements for AI in healthcare. Artificial Intelligence in Medicine. 2025. DOI: 10.1016/j.artmed.2025.103168