Frameworks for AI Security Governance

Insightful Sunday read regarding AI governance and risk. This framework brings some much-needed structure to AI governance in national security, especially in sensitive areas like privacy, rights, and high-stakes decision-making. The sections on restricted uses of AI make it clear that AI should not replace human judgment, particularly in scenarios impacting civil liberties or public trust. This is particularly relevant for national security contexts where public trust is essential, yet easily eroded by perceived overreach or misuse. The emphasis on impact assessments and human oversight is both pragmatic and proactive. AI is powerful, but without proper guardrails, it’s easy for its application to stray into gray areas, particularly in national security. The framework’s call for thorough risk assessments, documented benefits, and mitigated risks is forward-thinking, aiming to balance AI’s utility with caution. Another strong point is the training requirement. AI can be a black box for many users, so the framework rightly mandates that users understand both the tools’ potential and limitations. This also aligns well with the rising concerns around “automation bias,” where users might overtrust AI simply because it’s “smart.” The creation of an oversight structure through CAIOs and Governance Boards shows a commitment to transparency and accountability. It might even serve as a model for non-security government agencies as they adopt AI, reinforcing responsible and ethical AI usage across the board. Key Points: AI Use Restrictions: Strict limits on certain AI applications, particularly those that could infringe on civil rights, civil liberties, or privacy. Specific prohibitions include tracking individuals based on protected rights, inferring sensitive personal attributes (e.g., religion, gender identity) from biometrics, and making high-stakes decisions like immigration status solely based on AI. High-Impact AI and Risk Management: AI that influences major decisions, particularly in national security and defense, must undergo rigorous testing, oversight, and impact assessment. Cataloguing and Monitoring: A yearly inventory of high-impact AI applications, including data on their purpose, benefits, and risks, is required. This step is about creating a transparent and accountable record of AI use, aimed at keeping all deployed systems in check and manageable. Training and Accountability: Agencies are tasked with ensuring personnel are trained to understand the AI tools they use, especially those in roles with significant decision-making power. Training focuses on preventing overreliance on AI, addressing biases, and understanding AI’s limitations. Oversight Structure: A Chief AI Officer (CAIO) is essential within each agency to oversee AI governance and promote responsible AI use. An AI Governance Board is also mandated to oversee all high-impact AI activities within each agency, keeping them aligned with the framework’s principles.

✴ AI Governance Blueprint via ISO Standards – The 4-Legged Stool✴ ➡ ISO42001: The Foundation for Responsible AI #ISO42001 is dedicated to AI governance, guiding organizations in managing AI-specific risks like bias, transparency, and accountability. Focus areas include: ✅Risk Management: Defines processes for identifying and mitigating AI risks, ensuring systems are fair, robust, and ethically aligned. ✅Ethics and Transparency: Promotes policies that encourage transparency in AI operations, data usage, and decision-making. ✅Continuous Monitoring: Emphasizes ongoing improvement, adapting AI practices to address new risks and regulatory updates. ➡#ISO27001: Securing the Data Backbone AI relies heavily on data, making ISO27001’s information security framework essential. It protects data integrity through: ✅Data Confidentiality and Integrity: Ensures data protection, crucial for trustworthy AI operations. ✅Security Risk Management: Provides a systematic approach to managing security risks and preparing for potential breaches. ✅Business Continuity: Offers guidelines for incident response, ensuring AI systems remain reliable. ➡ISO27701: Privacy Assurance in AI #ISO27701 builds on ISO27001, adding a layer of privacy controls to protect personally identifiable information (PII) that AI systems may process. Key areas include: ✅Privacy Governance: Ensures AI systems handle PII responsibly, in compliance with privacy laws like GDPR. ✅Data Minimization and Protection: Establishes guidelines for minimizing PII exposure and enhancing privacy through data protection measures. ✅Transparency in Data Processing: Promotes clear communication about data collection, use, and consent, building trust in AI-driven services. ➡ISO37301: Building a Culture of Compliance #ISO37301 cultivates a compliance-focused culture, supporting AI’s ethical and legal responsibilities. Contributions include: ✅Compliance Obligations: Helps organizations meet current and future regulatory standards for AI. ✅Transparency and Accountability: Reinforces transparent reporting and adherence to ethical standards, building stakeholder trust. ✅Compliance Risk Assessment: Identifies legal or reputational risks AI systems might pose, enabling proactive mitigation. ➡Why This Quartet? Combining these standards establishes a comprehensive compliance framework: 🥇1. Unified Risk and Privacy Management: Integrates AI-specific risk (ISO42001), data security (ISO27001), and privacy (ISO27701) with compliance (ISO37301), creating a holistic approach to risk mitigation. 🥈 2. Cross-Functional Alignment: Encourages collaboration across AI, IT, and compliance teams, fostering a unified response to AI risks and privacy concerns. 🥉 3. Continuous Improvement: ISO42001’s ongoing improvement cycle, supported by ISO27001’s security measures, ISO27701’s privacy protocols, and ISO37301’s compliance adaptability, ensures the framework remains resilient and adaptable to emerging challenges.

The Cyber Security Agency of Singapore (CSA) has published “Guidelines on Securing AI Systems,” to help system owners manage security risks in the use of AI throughout the five stages of the AI lifecycle. 1. Planning and Design: - Raise awareness and competency on security by providing training and guidance on the security risks of #AI to all personnel, including developers, system owners and senior leaders. - Conduct a #riskassessment and supplement it by continuous monitoring and a strong feedback loop. 2. Development: - Secure the #supplychain (training data, models, APIs, software libraries) - Ensure that suppliers appropriately manage risks by adhering to #security policies or internationally recognized standards. - Consider security benefits and trade-offs such as complexity, explainability, interpretability, and sensitivity of training data when selecting the appropriate model to use (#machinelearning, deep learning, #GenAI). - Identify, track and protect AI-related assets, including models, #data, prompts, logs and assessments. - Secure the #artificialintelligence development environment by applying standard infrastructure security principles like #accesscontrols and logging/monitoring, segregation of environments, and secure-by-default configurations. 3. Deployment: - Establish #incidentresponse, escalation and remediation plans. - Release #AIsystems only after subjecting them to appropriate and effective security checks and evaluation. 4. Operations and Maintenance: - Monitor and log inputs (queries, prompts and requests) and outputs to ensure they are performing as intended. - Adopt a secure-by-design approach to updates and continuous learning. - Establish a vulnerability disclosure process for users to share potential #vulnerabilities to the system. 5. End of Life: - Ensure proper data and model disposal according to relevant industry standards or #regulations.

This new white paper "Introduction to AI assurance" by the UK Department for Science, Innovation, and Technology from Feb 12, 2024, provides an EXCELLENT overview of assurance methods and international technical standards that can be utilized to create and implement ethical AI systems. The new guidance is based on the UK AI governance framework, laid out in the 2023 white paper "A pro-innovation approach to AI regulation". This white paper defined 5 universal principles applicable across various sectors to guide and shape the responsible development and utilization of AI technologies throughout the economy: - Safety, Security, and Robustness - Appropriate Transparency and Explainability - Fairness - Accountability and Governance - Contestability and Redress The 2023 white paper also introduced a suite of tools designed to aid organizations in understanding "how" these outcomes can be achieved in practice, emphasizing tools for trustworthy AI, including assurance mechanisms and global technical standards. See: https://lnkd.in/gydvi9Tt The new publication, "Introduction to AI assurance," is a deep dive into these assurance mechanisms and standards. AI assurance encompasses a spectrum of techniques for evaluating AI systems throughout their lifecycle. These range from qualitative assessments for evaluating potential risks and societal impacts to quantitative assessments for measuring performance and legal compliance. Key techniques include: - Risk Assessment: Identifies potential risks like bias, privacy, misuse of technology, and reputational damage. - Impact Assessment: Anticipates broader effects on the environment, human rights, and data protection. - Bias Audit: Examines data and outcomes for unfair biases. - Compliance Audit: Reviews adherence to policies, regulations, and legal requirements. - Conformity Assessment: Verifies if a system meets required standards, often through performance testing. - Formal Verification: Uses mathematical methods to confirm if a system satisfies specific criteria. The white paper also explains how organizations in the UK can ensure their AI systems are responsibly governed, risk-assessed, and compliant with regulations: 1.) For demonstrating good internal governance processes around AI, a conformity assessment against standards like ISO/IEC 42001 (AI Management System) is recommended. 2.) To understand the potential risks of AI systems being acquired, an algorithmic impact assessment by a accredited conformity assessment body is advised. This involves (self) assessment against a proprietary framework or responsible AI toolkit. 3.) Ensuring AI systems adhere to existing data protection regulations involves a compliance audit by a third-party assurance provider. This white paper also has exceptional infographics! Pls, check it out, and TY Victoria Beckman for posting and providing us with great updates as always!

🙋🏼♀️ I am back with yet another post on AI in cyber! Previously, I shared about the dual nature of AI in cyber, the emerging threats and the rise of MLSecOps. Today, spotlight is on the emerging frameworks acting as essential guideposts for navigating this complex intersection. They provide a structured approach to secure AI systems and help define roadmaps for securing AI throughout its lifecycle. 🔭 ENISA's Framework for AI Cybersecurity Practices (#FAICP) FAICP offers a three-layered holistic approach to AI security, from basic cybersecurity principles to AI-specific and sector-specific considerations. Layer I focuses on cybersecurity foundations to secure the ICT infrastructure hosting AI systems. Layer II addresses unique challenges posed by AI components throughout their lifecycle. Layer III provides tailored practices for high-risk AI systems in specific industries. 🔭 #Google's Secure AI Framework (#SAIF) SAIF is designed to enhance security across AI operations to adapt to emerging threats in AI systems. It focuses on expanding strong security foundations to the AI ecosystem, extending detection and response to bring AI into an organization's threat universe and automating defenses to keep pace with existing and new threats. SAIF aims to harmonize platform-level controls for consistent security and faster feedback loops. 🔭 #MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) #ATLAS provides a knowledge base of adversary tactics and techniques specific to AI systems, tools for identifying potential vulnerabilities in AI systems and strategies for understanding and mitigating AI-specific threats. 🔭 Databricks' Data and AI Security Framework (#DASF) #DASF focuses on securing data and AI systems in cloud environments, implementing best practices for data protection and privacy, and ensuring compliance with relevant regulations in AI development and deployment. 🔭 NIST AI Risk Management Framework (#RMF) NIST's AI RMF offers a structured approach to managing risks in AI systems. It focuses on governance, context mapping, risk assessment, and risk treatment providing guidelines for the design, development, and evaluation of AI systems. This framework aims to enhance the trustworthiness and responsible development of AI. 🔭 EU AI Act While not a security framework per se, the #EUAI Act is a proposed regulation that aims to categorize AI systems based on risk levels (unacceptable, high, limited, minimal). It imposes strict requirements on high-risk AI systems ensuring transparency, accountability, and human oversight in AI development and deployment. These frameworks offer a comprehensive toolkit for cybersecurity practitioners, enabling them to secure AI systems effectively while navigating the intricate challenges at the intersection of AI and cybersecurity. Interested to read more on each❓Check the comments for links!😄 #AIinCyber #CybersecurityFrameworks #atlasmitre #NISTRMF #SAIF #DASF #EUAI #FAICP

Secure AI Lifecycle (SAIL) Framework With the rapid adoption and evolution of AI teams are often looking. for a framework to help turn principles and guidance into action. Excited to have collaborated with the Pillar Security team on this publication, which I think is a helpful tool for security and software practitioners building with and on AI systems. From executable data in the form of prompts, agency via Agentic AI and nuanced angles to consider such as model poisoning and more. The paper: 🔷 Lays out the AI development lifecycle and AI security landscape 🔷 Maps more than 70 risks across various AI development and deployment phases 🔷 Provides mitigations, mapped to leading frameworks such as ISO and NIST AI RMF 🔷 Captures a comprehensive definition of AI system components Recommend folks give this a read and bookmark 👇 #ai #cybersecurity #ciso #appsec

Over the past few months, I’ve been working behind the scenes on an initiative that’s shaping how we approach AI security at scale, the CSA AI Controls Matrix. If I’ve been quieter than usual, it’s because I’ve been focused on defining practical security controls that help organizations secure AI-driven technologies, third-party AI integrations, and enterprise AI adoption. AI is fundamentally shifting how businesses operate, but with that comes new security challenges: 🔹 How do we evaluate AI supply chain risks as third-party AI services become more embedded in SaaS and enterprise environments? 🔹 What baseline security controls should exist for AI models, training data, and operational workflows? 🔹 How do we balance risk management with the speed of AI innovation? The CSA AI Controls Matrix provides a structured, risk-based framework to help security teams navigate these challenges. It’s designed to be practical and adaptable, giving organizations clear guidance on how to integrate security, governance, and risk management into their AI strategies. 📝 The Peer Review is Still Open This is a collaborative effort, and industry input is critical. If you work in AI security, governance, compliance, or risk, I encourage you to review the matrix and provide feedback. The more perspectives we gather, the stronger the framework will be. https://lnkd.in/gCgNhxAi I’d love to hear your thoughts: What security gaps do you see in AI adoption today? #AI #Security #ThirdPartyRisk #CloudSecurity #AICompliance #SecurityArchitecture #Cybersecurity #SaaS

Let’s break down the NIST AI Risk Management Framework. If you work in cybersecurity, GRC, or support federal systems, this is something you need to start paying attention to — right now. The NIST AI RMF isn’t about how to build AI. It’s about how to use AI responsibly — in a way that’s secure, fair, and aligned with your organization’s mission. Here’s the quick breakdown: 1. Govern – Who’s responsible for the AI? What are the policies? What’s the oversight? 2. Map – What does the AI system do? Where are the risks and what data is it using? 3. Measure – How are we evaluating the outcomes, bias, and impact? 4. Manage – What controls are we putting in place to reduce harm and stay aligned with our goals? This is the kind of framework that’s going to shape how AI gets authorized for use in the government and beyond — especially with new contracts going to companies like OpenAI and Anduril. If you’re trying to pivot into this space — or stay ahead in it — start learning how risk, compliance, and AI intersect. It’s not just about knowing the tech. It’s about knowing how to govern it. This is where the future of GRC is headed. #NIST #AICompliance #RMF

OWASP GenAI Security Project published "The State of Agentic AI Security and Governance". This report provides a comprehensive view of today’s landscape for securing and governing autonomous AI systems. It explores the frameworks, governance models, and global regulatory standards shaping responsible Agentic AI adoption. Designed for developers, security professionals, and decision-makers, the report serves as a practical guide for navigating the complexities of building, managing, and deploying agentic applications safely and effectively. #genai #security #riskmanagement #agenticai #threats #mitigations #risklandscape #threats #risks #agents #frameworks