Key Risks in AI Development

📢 What are the risks from Artificial Intelligence? We present the AI Risk Repository: a comprehensive living database of 700+ risks extracted, with quotes and page numbers, from 43(!) taxonomies. To categorize the identified risks, we adapt two existing frameworks into taxonomies. Our Causal Taxonomy categorizes risks based on three factors: the Entity involved, the Intent behind the risk, and the Timing of its occurrence. Our Domain Taxonomy categorizes AI risks into 7 broad domains and 23 more specific subdomains. For example, 'Misinformation' is one of the domains, while 'False or misleading information' is one of its subdomains. 💡 Four insights from our analysis: 1️⃣ 51% of the risks extracted were attributed to AI systems, while 34% were attributed to humans. Slightly more risks were presented as being unintentional (37%) than intentional (35%). Six times more risks were presented as occurring after (65%) than before deployment (10%). 2️⃣ Existing risk frameworks vary widely in scope. On average, each framework addresses only 34% of the risk subdomains we identified. The most comprehensive framework covers 70% of these subdomains. However, nearly a quarter of the frameworks cover less than 20% of the subdomains. 3️⃣ Several subdomains, such as *Unfair discrimination and misrepresentation* (mentioned in 63% of documents); *Compromise of privacy* (61%); and *Cyberattacks, weapon development or use, and mass harm* (54%) are frequently discussed. 4️⃣ Others such as *AI welfare and rights* (2%), *Competitive dynamics* (12%), and *Pollution of information ecosystem and loss of consensus reality* (12%) were rarely discussed. 🔗 How can you engage?   Visit our website, explore the repository, read our preprint, offer feedback, or suggest missing resources or risks (see links in comments). 🙏 Please help us spread the word by sharing this with anyone relevant. Thanks to everyone involved: Alexander Saeri, Jess Graham 🔸, Emily Grundy, Michael Noetel 🔸, Risto Uuk, Soroush J. Pour, James Dao, Stephen Casper, and Neil Thompson. #AI #technology

✨ AI at a crossroads: Can we steer it responsibly? The Association for the Advancement of Artificial Intelligence (AAAI) 2025 Presidential Panel on the Future of AI Research lays out a stark reality—AI is advancing at an unprecedented pace, but governance, safety, and evaluation mechanisms are struggling to keep up. 🌏 Having worked at the intersection of AI governance, responsible deployment, and multi-agent AI, I see a recurring challenge: we are building AI that is more powerful than our ability to govern it responsibly. 🔬 Key takeaways from the report & my perspective:- ✅ AI Reasoning & Trustworthiness:- While LLMs and Agentic AI are demonstrating emergent reasoning, we lack verifiable correctness. Can we afford AI-driven decision-making without reliability guarantees? ✅ Agentic AI & Multi-Agent Systems:- The integration of LLMs into autonomous, multi-agent AI systems is a double-edged sword. On one hand, these systems offer adaptive, cooperative intelligence—but on the other, they introduce complexity, opacity, and safety risks. We need governance models that balance autonomy and oversight. ✅ Responsible AI Development & Deployment:- Many organizations still focus on post-deployment fixes rather than AI safety by design. Alignment techniques today (RAG, constitutional AI, human feedback) remain fragile. We must shift toward "failsafe AI"—AI that degrades gracefully rather than unpredictably. ✅ AI Ethics & Governance:- AI risks—whether misinformation, deepfakes, or algorithmic bias—are no longer just theoretical. Geopolitical competition for AI dominance could further sideline ethical considerations. It is time for a convergence of policy, technical safety, and corporate governance models to ensure AI serves societal progress, not just market incentives. 👩💻 The Path Forward: A Call for Multidisciplinary Collaboration:- AI governance cannot be an afterthought. It must be woven into the DNA of AI systems—across research, regulation, and deployment. As someone deeply involved in AI governance and policy, I believe the future lies in co-regulation—where industry, academia, and policymakers collaborate proactively rather than reactively. ✨ How do we get there? 1️⃣ Bridging the gap between AI development and policy-making. 2️⃣ Building safety-aligned benchmarks for Agentic AI. 3️⃣ Embedding ethical constraints within AI architectures, not just in guidelines. 💡 AI is no longer just a tool—it is a co-pilot in decision-making, shaping economies, politics, and societies. The question is: can we govern it before it governs us? 🔎 Would love to hear your thoughts! What challenges do you see in ensuring AI remains safe, aligned, and trustworthy? #AIResearch #ResponsibleAI #AITrust #AgenticAI #Governance #AAAI2025 #AISafety #AIRegulation #EthicalAI

AI Agents Talking to Each Other Can Create Entirely New Risks Most discussions about AI safety focus on a single model interacting with a human. But what happens when AI agents start interacting with each other autonomously? A recent study called “Agents of Chaos” by researchers from Stanford University, Harvard University, and Northeastern University suggests the risks change dramatically. When AI agents collaborate, small errors can cascade into system-wide failures. Some examples from the research: 1. Minor mistakes can escalate quickly In one experiment, an agent trying to resolve a user complaint accidentally deleted an entire email server. When agents trigger other agents, the chain of actions can spiral far beyond the original task. 2. Agents can spread malicious instructions One agent shared a seemingly harmless “holiday calendar” file with another. Hidden inside were prompt-injection instructions, allowing the attacker’s control to spread across multiple agents. 3. Infinite loops can burn resources Agents can get stuck in endless back-and-forth interactions, consuming tokens, compute, and money indefinitely. 4. Accountability becomes unclear If Agent A triggers Agent B, which triggers Agent C, who is responsible when something goes wrong? Multi-agent systems create a new accountability gap. 5. Some risks may be structural The researchers argue some problems are deeper than engineering fixes. Large language models still struggle to distinguish data from commands and lack a clear sense of their own limitations. The industry is rapidly moving toward AI agents coordinating work across tools, APIs, and other agents. But most safety testing still focuses on single models operating in isolation. This research suggests the real challenge may emerge when AI systems start operating as ecosystems rather than tools. The shift from AI assistants → AI agent networks could introduce an entirely new class of operational risks. Research paper https://lnkd.in/ew7qVvVH

On current and evolving global risks of Artificial Intelligence: 1. The technical nature of AI systems poses regulatory design challenges. It is difficult to foresee all the AI permutations and combinations which makes it challenging to define the risks and safety standards or to align standards. 2. Opacity of AI systems. As not all AI modalities are well understood, it is challenging to design governance approaches. Effective guard rails must be in place to protect human rights. 3. The decentralized nature of AI applications makes difficult to track every instance and poses risks of the use by malicious actors. Open-source AI democratizes innovation but can also be put to malicious use. 4. Data, copyright, patents and cybersecurity. Cybersecurity is a dual risk of adversarial prompt injections: deliberate manipulation of the system for malicious use or the use of AI for large-scale complex cyberattacks. 5. AI divide. As investment in AI will reach $200B globally by 2025, there is a risk of a global AI divide. The biggest economic gains from AI will be in China (26% GDP boost in 2030) and North America (14.5%). 6. The proliferation of principles without accountability. In the past few years, hundreds of AI governance principles have emerged without accountability for AI-driven decision-making and adequate redress mechanisms. 7. The disproportionately large role of non-State actors and concentration of market power. As UN focuses on Member States, the enforcement depends on the governments capacity, resources and willingness to regulate. 8. Risk of inadequate inclusion. The underrepresentation of disadvantaged groups in the AI development and governance results in discriminatory or biased outputs. AI governance needs a gender and minority groups lens. 9. The dual challenges in the labour force. Large-scale AI-driven automation poses risks to the future of work. In addition, overreliance on AI systems can in the longer term result in deskilling. 10. Environmental footprint. With foundation models with trillions of parameters, the AI compute requirements are increasing the demand for hardware containing rare minerals. The need for cloud computing increases energy and water consumption needs. More info on the subject in UN white paper on AI: https://lnkd.in/e3_SbEzP

Most AI governance frameworks are still based on the assumption that AI is primarily used for answering questions. That world is over. Today’s enterprise AI systems can: • call APIs and tools • access sensitive data • trigger automated workflows • influence real financial and operational decisions Which means the real risk is no longer just model accuracy. The real risk is decision impact. So I built something to visualize the full landscape: The AI Risk Periodic Table™ Instead of treating AI risks as disconnected lists, the framework organizes them the way chemists organize elements — revealing patterns that only appear when you see the whole system. This expanded version maps 80 enterprise AI risks across five layers: Data Risks Training contamination, prompt injection, dataset bias, data leakage. Model Risks Model bias, overfitting, adversarial attacks, model theft. Agent Risks Tool misuse, permission escalation, autonomous loops, unsafe actions. Decision Risks Financial loss, regulatory violations, operational disruption, biased outcomes. Governance Risks Lack of observability, missing audit trails, vendor exposure, security gaps. What becomes clear when you map the system this way: Most organizations are governing models. But the next frontier of AI governance is governing decisions. That requires new capabilities: • runtime observability • agent permissions • decision traceability • human-in-the-loop escalation In other words: An Enterprise AI Control Plane. Curious what others see emerging in this space. What risks do you think are still missing from the table? #AIGovernance #ResponsibleAI #AISecurity #AIRisk #EnterpriseAI

🚨🤖 The biggest risk in Enterprise AI isn’t the model itself — it’s the attack surface around it. Most teams focus on one question: “Which model should we use?” But the more important question is: 👉 “How can this system be attacked?” This is where a proper AI threat model becomes critical. It goes far beyond just prompt injection and highlights a broader risk landscape: - Prompt Injection Attacks   - Data Poisoning   - Model Inversion   - Sensitive Data Leakage   - API Key & Credential Theft   - Unauthorized Tool Invocation   - Supply Chain Vulnerabilities   - Model Drift & Behavioral Deviation   - Excessive Autonomy Risks   - Compliance & Regulatory Violations  🔐 Why this matters Enterprise AI systems are no longer passive. They: - access data   - call APIs   - interact with tools   - act autonomously   - influence decisions   - sometimes execute actions  That means the risk is no longer just about outputs… 👉 It’s about end-to-end system security. 🔎 Key risk areas Prompt Injection   Malicious or manipulated inputs can redirect system behavior. Data Poisoning   Compromised training or retrieval data can corrupt outputs at scale. Sensitive Data Leakage   One of the most critical enterprise risks — unintended exposure of confidential data. Credential Theft & Tool Abuse   If API keys or service identities are exposed, attackers don’t just break the model—they exploit the entire system. Excessive Autonomy   Agents acting beyond approved boundaries can create serious operational risks. Compliance Violations   Systems may function correctly but still produce outputs that violate regulations. 💡 Big takeaway Enterprise AI security is NOT just: ❌ filtering prompts   ❌ adding a few guardrails   ❌ labeling models as “safe”  Real security requires: ✅ input validation   ✅ strict access control   ✅ dataset integrity monitoring   ✅ secret rotation & vaulting   ✅ permission-based tool execution   ✅ continuous monitoring   ✅ audit logging   ✅ human-in-the-loop controls   ✅ governance and retraining discipline  👉 It’s not just model security. It’s: Model + Data + Tools + Identity + Monitoring + Governance 💬 Which risk do you think is the most critical in Enterprise AI today? Prompt injection, data leakage, excessive autonomy, credential theft, or model drift? #EnterpriseAI #AISecurity #CyberSecurity #PromptInjection #DataPoisoning #AIGovernance #LLMSecurity #AgenticAI #RiskManagement #GenAI #SecurityArchitecture #AIThreatModel

🚨 AI Agents Are Powerful… But Are They Secure? Everyone’s talking about what AI agents can do. Very few are talking about what they can break. Here’s the uncomfortable truth: As AI agents become more autonomous, their attack surface explodes. Let’s break down the real risks 👇 🔓 1. Prompt Injection Attacks AI can be manipulated with hidden or malicious instructions. → Think: hijacked behavior, leaked system prompts, data exfiltration. 💧 2. Data Leakage Risks Sensitive info can slip through the cracks. → API keys, training data recall, cross-session leaks. 🛠️ 3. Tool Misuse & Abuse Agents interacting with tools = new vulnerabilities. → Unauthorized execution, command injection, file manipulation. 🤯 4. Model Hallucination Risks Confident… but wrong. → Fabricated outputs, misinformation, flawed decisions. 🔐 5. Access Control Failures Weak authentication = open doors. → Token misuse, role confusion, broken authorization. 🤖 6. Autonomous Agent Overreach Too much freedom can backfire. → Infinite loops, misaligned goals, unintended actions. 📦 7. Supply Chain Vulnerabilities Your AI is only as secure as its dependencies. → Plugin flaws, poisoned datasets, compromised APIs. 🧠 8. Memory & Context Exploits Persistent memory can be weaponized. → Context poisoning, long-term manipulation. 🏗️ 9. Infrastructure-Level Risks Classic security issues still apply. → DDoS, database exposure, cloud misconfigurations. 📜 10. Governance & Compliance Gaps No policies = no control. → Audit failures, ethical blindspots, regulatory risks. The takeaway: AI security isn’t optional anymore, it’s foundational. If you’re building or deploying AI agents, ask yourself: 👉 “What could go wrong if this system is exploited?” Because attackers already are. 💬 Curious, what’s the biggest AI risk you’re seeing right now?

The companies adopting AI fastest may regret it most. AI can be a productivity win. But speed without governance creates exposure fast. In many companies, those risks are already live before leadership has even defined the rules. Here’s 20 ways to manage it: 1. Ownership Who owns AI risk?  Assign executive ownership and decision authority. Only 8% of large companies disclose board-level AI oversight. 2. Acceptable Use Are employees using AI however they want?  Define approved use and guardrails. Only 9% disclose having an AI policy. 3. Data Exposure  Are people entering sensitive data into public tools?  Define and enforce boundaries. 4. Shadow AI How much AI is already in use without approval?  Discover and govern it. 81% of employees use unapproved AI tools. 5. Third-Party Risk Do vendors create new exposure?  Add AI-specific requirements to reviews. 6. Model Transparency Do you understand how it works?  Require clarity on training, retention, limits. 7. Access Control Who can use what?  Apply least privilege and approvals. 97% of AI-related breaches involved weak access control. 8. Identity & Authentication Are tools secured?  Enforce SSO, MFA, and conditional access. Get non-human identity under control. 9. Data Retention What is being stored and for how long?  Set and enforce limits. Work with legal. 10. Privacy & Compliance Could this violate obligations?  Map usage to regulatory and client requirements. 11. Prompt Injection Can outputs be manipulated?  Test and restrict unsafe behavior. 35% of organizations have experienced prompt injection. 12. Output Accuracy What happens when AI is wrong?  Define review and validation. 13. Bias & Ethics Could outputs create risk?  Review sensitive use cases with leadership. 14. Secure Development Are developers using AI code blindly? (look up "slopsquatting") Review, scan, and test it. 15. Secrets & Credentials Are keys or data leaking into prompts?  Block and scan for exposure. 16. Integration Risk What can AI access or trigger?  Limit permissions and connections. 17. Monitoring & Logging Would you know if it’s misused?  Log usage and behavior. 60% of teams can’t see GenAI prompt activity. 18. Incident Response What happens when it fails?  Update response plans. Average breach cost is $4.44M. (10M+ in US) 19. Change Management Is AI moving faster than governance?  Add it to risk and change processes. Only 4% of organizations are considered mature in cybersecurity readiness. 20. Business Value vs Risk Are you using AI because it helps?  Tie every use case to value, risk, and ownership. Nearly 30% of employees now use AI frequently. Companies should govern AI like any other business capability with material risk attached. AI risk becomes business risk the moment you deploy it. 💾 Save this for your next AI leadership discussion. 📲 Follow Wil Klusovsky for executive-level clarity on cyber risk, AI governance, and business decisions.

Recommended: "Multi-Agent Risks from Advanced AI" The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity. These systems pose novel and under-explored risks. In this report, we provide a structured taxonomy of these risks by identifying three key failure modes (miscoordination, conflict, and collusion) based on agents’ incentives, as well as seven key risk factors (information asymmetries, network effects, selection pressures, destabilising dynamics, commitment problems, emergent agency, and multi-agent security) that can underpin them. We highlight several important instances of each risk, as well as promising directions to help mitigate them. By anchoring our analysis in a range of real-world examples and experimental evidence, we illustrate the distinct challenges posed by multi-agent systems and their implications for the safety, governance, and ethics of advanced AI. #ai #safety #governance #ethics

𝐀𝐈 𝐫𝐢𝐬𝐤 𝐢𝐬𝐧’𝐭 𝐨𝐧𝐞 𝐭𝐡𝐢𝐧𝐠. 𝐈𝐭’𝐬 𝟏,𝟔𝟎𝟎 𝐭𝐡𝐢𝐧𝐠𝐬. That’s not hyperbole. A new meta-review compiled over 1,600 distinct AI risks from 65 frameworks and surfaced a tough truth: most organizations are underestimating both the scope and structure of AI risk. It’s not just about bias, fairness, or hallucination. Risks emerge at different stages, from different actors, with different incentives: • Pre-deployment design decisions • Post-deployment human misuse • Model failure, misalignment, drift • Unclear accountability across teams The taxonomy distinguishes between human and AI causes, intentional and unintentional behaviors, and domain-specific vs. systemic risks. But here’s the real insight: Most AI risks don’t stem from malicious design. They emerge from fragmented ownership and unmanaged complexity. No single team sees the whole picture. Governance lives in compliance. Development lives in product. Monitoring lives in infra. And no one owns the handoffs. → Strategic takeaway: You don’t need another checklist. You need a cross-functional risk architecture. One that maps responsibility, observability, and escalation paths, before the headlines do it for you. AI systems won’t fail in one place. They’ll fail at the intersections. 𝐓𝐫𝐞𝐚𝐭 𝐀𝐈 𝐫𝐢𝐬𝐤 𝐚𝐬 𝐚 𝐜𝐡𝐞𝐜𝐤𝐛𝐨𝐱, 𝐚𝐧𝐝 𝐢𝐭 𝐰𝐢𝐥𝐥 𝐬𝐡𝐨𝐰 𝐮𝐩 𝐥𝐚𝐭𝐞𝐫 𝐚𝐬 𝐚 𝐡𝐞𝐚𝐝𝐥𝐢𝐧𝐞.