Responsible Machine Learning
0 likesâ˘88 views
Responsible AI refers to developing and deploying AI systems in a safe, ethical, and trustworthy manner, centered around principles like fairness, reliability, privacy, inclusiveness, transparency, and accountability. Microsoft has established a Responsible AI standard to guide these efforts, employing tools such as the Responsible AI dashboard in Azure Machine Learning to assess fairness, interpretability, and privacy while ensuring robust accountability through MLOps practices. As AI technology becomes ingrained in daily life, prioritizing these principles is essential for fostering trust and achieving equitable outcomes.
Responsible Machine Learning
- 2. Eng Teong Cheah Microsoft MVP
- 3. What is Responsible AI? Responsible Artificial Intelligence (Responsible AI) is an approach to developing, assessing, and deploying AI systems in a safe, trustworthy, and ethical way. AI systems are the product of many decisions made by those who develop and deploy them. From system purpose to how people interact with AI systems, Responsible AI can help proactively guide these decisions toward more beneficial and equitable outcomes. That means keeping people and their goals at the center of system design decisions and respecting enduring values like fairness, reliability, and transparency.
- 4. What is Responsible AI? Microsoft has developed a Responsible AI Standard. It's a framework for building AI systems according to six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. For Microsoft, these principles are the cornerstone of a responsible and trustworthy approach to AI, especially as intelligent technology becomes more prevalent in products and services that people use every day.
- 5. Fairness and inclusiveness AI systems should treat everyone fairly and avoid affecting similarly situated groups of people in different ways. For example, when AI systems provide guidance on medical treatment, loan applications, or employment, they should make the same recommendations to everyone who has similar symptoms, financial circumstances, or professional qualifications. Fairness and inclusiveness in Azure Machine Learning: The fairness assessment component of the Responsible AI dashboard enables data scientists and developers to assess model fairness across sensitive groups defined in terms of gender, ethnicity, age, and other characteristics.
- 6. Reliability and safety To build trust, it's critical that AI systems operate reliably, safely, and consistently. These systems should be able to operate as they were originally designed, respond safely to unanticipated conditions, and resist harmful manipulation. How they behave and the variety of conditions they can handle reflect the range of situations and circumstances that developers anticipated during design and testing
- 7. Reliability and safety Reliability and safety in Azure Machine Learning: The error analysis component of the Responsible AI dashboard enables data scientists and developers to: ⢠Get a deep understanding of how failure is distributed for a model. ⢠Identify cohorts (subsets) of data with a higher error rate than the overall benchmark. These discrepancies might occur when the system or model underperforms for specific demographic groups or for infrequently observed input conditions in the training data.
- 8. Transparency When AI systems help inform decisions that have tremendous impacts on people's lives, it's critical that people understand how those decisions were made. For example, a bank might use an AI system to decide whether a person is creditworthy. A company might use an AI system to determine the most qualified candidates to hire. A crucial part of transparency is interpretability: the useful explanation of the behavior of AI systems and their components. Improving interpretability requires stakeholders to comprehend how and why AI systems function the way they do. The stakeholders can then identify potential performance issues, fairness issues, exclusionary practices, or unintended outcomes.
- 9. Transparency Transparency in Azure Machine Learning: The model interpretability and counterfactual what-if components of the Responsible AI dashboard enable data scientists and developers to generate human-understandable descriptions of the predictions of a model.
- 10. Transparency The model interpretability component provides multiple views into a model's behavior: ⢠Global explanations. For example, what features affect the overall behavior of a loan allocation model? ⢠Local explanations. For example, why was a customer's loan application approved or rejected? ⢠Model explanations for a selected cohort of data points. For example, what features affect the overall behavior of a loan allocation model for low-income applicants?
- 11. Transparency The counterfactual what-if component enables understanding and debugging a machine learning model in terms of how it reacts to feature changes and perturbations. Azure Machine Learning also supports a Responsible AI scorecard. The scorecard is a customizable PDF report that developers can easily configure, generate, download, and share with their technical and non-technical stakeholders to educate them about their datasets and models health, achieve compliance, and build trust. This scorecard can also be used in audit reviews to uncover the characteristics of machine learning models.
- 12. Privacy and security As AI becomes more prevalent, protecting privacy and securing personal and business information are becoming more important and complex. With AI, privacy and data security require close attention because access to data is essential for AI systems to make accurate and informed predictions and decisions about people. AI systems must comply with privacy laws that: ⢠Require transparency about the collection, use, and storage of data. ⢠Mandate that consumers have appropriate controls to choose how their data is used.
- 13. Privacy and security Privacy and security in Azure Machine Learning: Azure Machine Learning enables administrators and developers to create a secure configuration that complies with their companies' policies. With Azure Machine Learning and the Azure platform, users can: ⢠Restrict access to resources and operations by user account or group. ⢠Restrict incoming and outgoing network communications. ⢠Encrypt data in transit and at rest. ⢠Scan for vulnerabilities. ⢠Apply and audit configuration policies.
- 14. Accountability The people who design and deploy AI systems must be accountable for how their systems operate. Organizations should draw upon industry standards to develop accountability norms. These norms can ensure that AI systems aren't the final authority on any decision that affects people's lives. They can also ensure that humans maintain meaningful control over otherwise highly autonomous AI systems.
- 15. Accountability Accountability in Azure Machine Learning: Machine learning operations (MLOps) is based on DevOps principles and practices that increase the efficiency of AI workflows. Azure Machine Learning provides the following MLOps capabilities for better accountability of your AI systems: ⢠Register, package, and deploy models from anywhere. You can also track the associated metadata that's required to use the model. ⢠Capture the governance data for the end-to-end machine learning lifecycle. The logged lineage information can include who is publishing models, why changes were made, and when models were deployed or used in production. ⢠Notify and alert on events in the machine learning lifecycle. Examples include experiment completion, model registration, model deployment, and data drift detection. ⢠Monitor applications for operational issues and issues related to machine learning. Compare model inputs between training and inference, explore model-specific metrics, and provide monitoring and alerts on your machine learning infrastructure.