Navigating AI Risks

Three AI recruiters look at the same 109 CVs. They agree only 14% of the time. That’s not the start of a joke. And that's not efficiency. That’s what I call 'Rank Roulette'. When I tested ChatGPT, Gemini and Grok against the same job spec and anonymised CV set, here’s what happened: • 14% overlap in shortlists → Four times out of five, the models disagreed. • ±2.5 places volatility → Yesterday’s #2 became today’s #5. • 55% of CVs never surfaced → Candidates vanished with no audit trail. • 96% recycled rationales → Fluent, but shallow logic. We’re told by vendors and in-house 'tinkerers' that LLMs can “shortlist in seconds”. The truth: they behave more like over-confident interns - smooth on the surface, but shockingly inconsistent. And the worst part? It’s not even random. In a follow-up piece, I explored why this happens: a technical quirk called batch non-determinism. In plain English: your candidate’s fate changes depending on what else the server was processing at that moment. Until volatility is tamed, hands-off AI screening with LLMs is more than risky. It’s completely unexplainable, indefensible and a governance nightmare. Go to the comments for 👉 Full research 👉 Follow-up on why AI recruiters play favourites

Your AI recruiting agent or use case might be brilliant. It might also be illegal. If your AI screens, ranks, or evaluates candidates - you're operating in an increasingly actively regulated environment. And not just in the US. NYC requires annual bias audits. Illinois requires notice. California requires 4-year data retention. Colorado requires impact assessments with $20,000 per violation penalties. The EU classifies all recruiting AI as high-risk. South Korea's AI Basic Act explicitly lists hiring as high-impact. Brazil and Chile have GDPR-style rights against automated employment decisions. Singapore's Workplace Fairness Act covers AI-driven hiring decisions. This isn't a US-and-EU issue. It's global. Something else you need to look out for - your compliance is only as strong as the gap between your published AI notice and what your people actually do. A recruiter pastes a resume into ChatGPT on a busy Tuesday. Or simply uses their company-approved solution in a way that wasn't approved. That tool/use case hasn't been audited. There's no notice. No audit trail. The employer is still liable. I wrote a full breakdown of the regulatory landscape - US, EU, and the global wave most people don't see coming - and what TA teams need to do about it. Check it out 👇

📢 What are the risks from Artificial Intelligence? We present the AI Risk Repository: a comprehensive living database of 700+ risks extracted, with quotes and page numbers, from 43(!) taxonomies. To categorize the identified risks, we adapt two existing frameworks into taxonomies. Our Causal Taxonomy categorizes risks based on three factors: the Entity involved, the Intent behind the risk, and the Timing of its occurrence. Our Domain Taxonomy categorizes AI risks into 7 broad domains and 23 more specific subdomains. For example, 'Misinformation' is one of the domains, while 'False or misleading information' is one of its subdomains. 💡 Four insights from our analysis: 1️⃣ 51% of the risks extracted were attributed to AI systems, while 34% were attributed to humans. Slightly more risks were presented as being unintentional (37%) than intentional (35%). Six times more risks were presented as occurring after (65%) than before deployment (10%). 2️⃣ Existing risk frameworks vary widely in scope. On average, each framework addresses only 34% of the risk subdomains we identified. The most comprehensive framework covers 70% of these subdomains. However, nearly a quarter of the frameworks cover less than 20% of the subdomains. 3️⃣ Several subdomains, such as *Unfair discrimination and misrepresentation* (mentioned in 63% of documents); *Compromise of privacy* (61%); and *Cyberattacks, weapon development or use, and mass harm* (54%) are frequently discussed. 4️⃣ Others such as *AI welfare and rights* (2%), *Competitive dynamics* (12%), and *Pollution of information ecosystem and loss of consensus reality* (12%) were rarely discussed. 🔗 How can you engage?   Visit our website, explore the repository, read our preprint, offer feedback, or suggest missing resources or risks (see links in comments). 🙏 Please help us spread the word by sharing this with anyone relevant. Thanks to everyone involved: Alexander Saeri, Jess Graham 🔸, Emily Grundy, Michael Noetel 🔸, Risto Uuk, Soroush J. Pour, James Dao, Stephen Casper, and Neil Thompson. #AI #technology

AI agents are not yet safe for unsupervised use in enterprise environments The German Federal Office for Information Security (BSI) and France’s ANSSI have just released updated guidance on the secure integration of Large Language Models (LLMs). Their key message? Fully autonomous AI systems without human oversight are a security risk and should be avoided. As LLMs evolve into agentic systems capable of autonomous decision-making, the risks grow exponentially. From Prompt Injection attacks to unauthorized data access, the threats are real and increasingly sophisticated. The updated framework introduces Zero Trust principles tailored for LLMs: 1) No implicit trust: every interaction must be verified. 2) Strict authentication & least privilege access – even internal components must earn their permissions. 3) Continuous monitoring – not just outputs, but inputs must be validated and sanitized. 4) Sandboxing & session isolation – to prevent cross-session data leaks and persistent attacks. 5) Human-in-the-loop, i.e., critical decisions must remain under human control. Whether you're deploying chatbots, AI agents, or multimodal LLMs, this guidance is a must-read. It’s not just about compliance but about building trustworthy AI that respects privacy, integrity, and security. Bottom line: AI agents are not yet safe for unsupervised use in enterprise environments. If you're working with LLMs, it's time to rethink your architecture.

“Mapping Cybersecurity Threats to Defenses: A Strategic Approach to Risk Mitigation” Most of the time we talk about reducing risk by implementing controls, but we don’t talk about if the implemented controls will reduce the Probability or Impact of the Risk. The below matrix helps organizations build a robust, prioritized, and strategic cybersecurity posture while ensuring risks are managed comprehensively by implementing controls that reduces the probability while minimising the impact. Key Takeaways from the Matrix 1. Multi-layered Security: Many controls address multiple attack types, emphasizing the importance of defense in depth. 2. Balance Between Probability and Impact: Controls like patch management and EDR reduce both the likelihood of attacks (probability) and the harm they can cause (impact). 3. Tailored Controls: Some attacks (e.g., DDoS) require specific solutions like DDoS protection, while broader threats (e.g., phishing) are countered by multiple layers like email security, IAM, and training. 4. Holistic Approach: Combining technical measures (e.g., WAF) with process controls (e.g., training, third-party risk management) creates a comprehensive security posture. This matrix can be a powerful tool for understanding how individual security controls align with specific threats, helping organizations prioritize investments and optimize their cybersecurity strategy. Cyber Security News ®The Cyber Security Hub™

Are you a legal or compliance team member responsible for reviewing new AI vendor tools but struggling with bandwidth to do so? Then this post is for you. Internal requests to legal and compliance teams to approve new AI vendors are overwhelming - every tech vendor now has some form of AI functionality, and if the internal mandate is "all new AI must be reviewed and approved", then legal and compliance teams simply won't have enough hours in the day. Necessity therefore breeds an uncomfortable compromise - legal and compliance teams are forced to create simple gating rules that ensure the most risky AI tools still get escalated to them for their review, while lower risk AI tools get waived through without formal review. If you find yourself in that difficult position, and can ask only a handful of key gating questions to the business or vendor, what should they be? For most commercial organisations, I'd suggest the five questions set out in the diagram below. The first two are self-explanatory - though, of course, if you're asking this of business colleagues rather than the AI vendor directly, you'll need to give them a simple list of what is prohibited or high risk for them to check against. The third question aims to ensure that the AI vendor processes your data only as a processor - and that it doesn't use confidential, commercially sensitive, proprietary or personal data to train its models, with the risk that the AI's outputs will regurgitate your data to others. The fourth question is oriented towards detecting agentic AI use cases - specifically those that will make autonomous decisions in the sense of the GDPR - i.e. those that have legal or significant effects on people, necessarily requiring closer review and scrutiny. The final question tries to address the biggest B2B risk of using genAI tools - namely that they (or their outputs) gets used for external-facing purposes, such as integration into customer products, interacting with customers through chatbots, or writing content for websites. Purely internal use of genAI, while not without its risks, raises lower concerns in most commercial cases. Yes, of course there are holes you can pick in this risk-managed approach - but the simple reality is that most legal and compliance teams don't have the luxury of infinite time, budget or headcounts to review every AI tool the business wants to use, and need to do the best they can with limited resources. If you find yourself in this position, then these questions will hopefully throw you a lifeline!

On August 1, 2024, the European Union's AI Act came into force, bringing in new regulations that will impact how AI technologies are developed and used within the E.U., with far-reaching implications for U.S. businesses. The AI Act represents a significant shift in how artificial intelligence is regulated within the European Union, setting standards to ensure that AI systems are ethical, transparent, and aligned with fundamental rights. This new regulatory landscape demands careful attention for U.S. companies that operate in the E.U. or work with E.U. partners. Compliance is not just about avoiding penalties; it's an opportunity to strengthen your business by building trust and demonstrating a commitment to ethical AI practices. This guide provides a detailed look at the key steps to navigate the AI Act and how your business can turn compliance into a competitive advantage. 🔍 Comprehensive AI Audit: Begin with thoroughly auditing your AI systems to identify those under the AI Act’s jurisdiction. This involves documenting how each AI application functions and its data flow and ensuring you understand the regulatory requirements that apply. 🛡️ Understanding Risk Levels: The AI Act categorizes AI systems into four risk levels: minimal, limited, high, and unacceptable. Your business needs to accurately classify each AI application to determine the necessary compliance measures, particularly those deemed high-risk, requiring more stringent controls. 📋 Implementing Robust Compliance Measures: For high-risk AI applications, detailed compliance protocols are crucial. These include regular testing for fairness and accuracy, ensuring transparency in AI-driven decisions, and providing clear information to users about how their data is used. 👥 Establishing a Dedicated Compliance Team: Create a specialized team to manage AI compliance efforts. This team should regularly review AI systems, update protocols in line with evolving regulations, and ensure that all staff are trained on the AI Act's requirements. 🌍 Leveraging Compliance as a Competitive Advantage: Compliance with the AI Act can enhance your business's reputation by building trust with customers and partners. By prioritizing transparency, security, and ethical AI practices, your company can stand out as a leader in responsible AI use, fostering stronger relationships and driving long-term success. #AI #AIACT #Compliance #EthicalAI #EURegulations #AIRegulation #TechCompliance #ArtificialIntelligence #BusinessStrategy #Innovation 

When AI Meets Security: The Blind Spot We Can't Afford Working in this field has revealed a troubling reality: our security practices aren't evolving as fast as our AI capabilities. Many organizations still treat AI security as an extension of traditional cybersecurity—it's not. AI security must protect dynamic, evolving systems that continuously learn and make decisions. This fundamental difference changes everything about our approach. What's particularly concerning is how vulnerable the model development pipeline remains. A single compromised credential can lead to subtle manipulations in training data that produce models which appear functional but contain hidden weaknesses or backdoors. The most effective security strategies I've seen share these characteristics: • They treat model architecture and training pipelines as critical infrastructure deserving specialized protection • They implement adversarial testing regimes that actively try to manipulate model outputs • They maintain comprehensive monitoring of both inputs and inference patterns to detect anomalies The uncomfortable reality is that securing AI systems requires expertise that bridges two traditionally separate domains. Few professionals truly understand both the intricacies of modern machine learning architectures and advanced cybersecurity principles. This security gap represents perhaps the greatest unaddressed risk in enterprise AI deployment today. Has anyone found effective ways to bridge this knowledge gap in their organizations? What training or collaborative approaches have worked?

The real risk with Artificial Intelligence today is not that it’s being used as a tool—but that it’s increasingly treated as a silver bullet. This shift, often described by experts as 𝗔𝗜 𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝗶𝘀𝗺, reflects a growing belief that AI can resolve complex social, ethical, and organizational problems simply by applying more data and better models. AI has undeniably earned its place as a utility. It automates routine work, improves efficiency, and enables data-driven decisions across domains such as healthcare, finance, and agriculture. But problems emerge when this utility mindset mutates into blind faith. When AI is framed as a universal solution, four failure modes consistently surface: 𝗢𝘃𝗲𝗿-𝗿𝗲𝗹𝗶𝗮𝗻𝗰𝗲 𝗮𝗻𝗱 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻 𝗯𝗶𝗮𝘀 People defer to AI recommendations over their own judgment—especially when outputs sound confident. The result is diminished critical thinking, weaker challenge, and reduced creativity. 𝗙𝗮𝗯𝗿𝗶𝗰𝗮𝘁𝗲𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻 (𝗵𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗶𝗼𝗻𝘀) AI systems can produce plausible but false outputs. High-profile legal cases, where generative AI tools fabricated court citations, illustrate how credibility can collapse when verification is skipped. 𝗕𝗶𝗮𝘀 𝗮𝗺𝗽𝗹𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 Trained on historical data, AI systems often reproduce—and sometimes intensify—existing racial, gender, and socioeconomic biases, particularly in hiring, credit scoring, and criminal justice applications. 𝗘𝗿𝗼𝘀𝗶𝗼𝗻 𝗼𝗳 𝗵𝘂𝗺𝗮𝗻 𝗮𝗴𝗲𝗻𝗰𝘆 Decision-making responsibility quietly shifts from humans to machines, creating a “responsibility gap” where ethical, political, and accountability judgments are effectively outsourced. 𝗧𝗵𝗲 𝗪𝗮𝘆 𝗙𝗼𝗿𝘄𝗮𝗿𝗱: 𝗛𝘂𝗺𝗮𝗻-𝗶𝗻-𝘁𝗵𝗲-𝗟𝗼𝗼𝗽 Researchers such as 𝗦𝘁𝘂𝗮𝗿𝘁 𝗥𝘂𝘀𝘀𝗲𝗹𝗹 argue that the antidote to AI solutionism is not less AI—but better integration of human judgment. 𝗛𝘆𝗯𝗿𝗶𝗱 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝘄𝗼𝗿𝗸𝘀 𝗯𝗲𝘀𝘁 AI excels at pattern recognition and scale; humans excel at context, values, and judgment. The highest-quality outcomes emerge when the two are deliberately combined. 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝗯𝗹𝗲 𝗔𝗜 𝗯𝘆 𝗱𝗲𝘀𝗶𝗴𝗻 Organizations are moving toward principles of transparency, accountability, and human verification—ensuring that consequential decisions are reviewed, challenged, and owned by people. The trend is clear: the future is not about replacing human intelligence, but about building “𝘀𝗮𝗳𝗲-𝗯𝘆-𝗱𝗲𝘀𝗶𝗴𝗻” 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀—assistants that augment human capability rather than substitute for it. AI is powerful. But without humans firmly in the loop, it is not wise.

The AI gave a clear diagnosis. The doctor trusted it. The only problem? The AI was wrong. A year ago, I was called in to consult for a global healthcare company. They had implemented an AI diagnostic system to help doctors analyze thousands of patient records rapidly. The promise? Faster disease detection, better healthcare. Then came the wake-up call. The AI flagged a case with a high probability of a rare autoimmune disorder. The doctor, trusting the system, recommended an aggressive treatment plan. But something felt off. When I was brought in to review, we discovered the AI had misinterpreted an MRI anomaly. The patient had an entirely different condition—one that didn’t require aggressive treatment. A near-miss that could have had serious consequences. As AI becomes more integrated into decision-making, here are three critical principles for responsible implementation: - Set Clear Boundaries Define where AI assistance ends and human decision-making begins. Establish accountability protocols to avoid blind trust. - Build Trust Gradually Start with low-risk implementations. Validate critical AI outputs with human intervention. Track and learn from every near-miss. - Keep Human Oversight AI should support experts, not replace them. Regular audits and feedback loops strengthen both efficiency and safety. At the end of the day, it’s not about choosing AI 𝘰𝘳 human expertise. It’s about building systems where both work together—responsibly. 💬 What’s your take on AI accountability? How are you building trust in it?