Extending Apache Ranger Authorization Beyond Hadoop: Review of Apache Ranger Extensibility Framework & Case Study for integration with Apache HAWQ
Download as PPTX, PDF3 likes1,465 views
The document provides an overview of Apache Ranger, focusing on its role in securing various Hadoop components through authentication, authorization, and auditing. It details the administration and integration of Ranger with components like HDFS, Hive, and HBase, emphasizing policy management and audit log generation. Additionally, it outlines how to enable services with Ranger, including service definitions and plugin interfaces.
Extending Apache Ranger Authorization Beyond Hadoop: Review of Apache Ranger Extensibility Framework & Case Study for integration with Apache HAWQ
- 1. 1 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache Ranger – Extending Authorization beyond Hadoop Ramesh Mani Apache Ranger committer, PMC member Abhay Kulkarni Apache Ranger committer
- 2. 2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Overview • Securing Hadoop components • HDFS, Hive, HBase, Knox, Strom, YARN, Kafka, Solr, Atlas, Nifi • Authentication, Authorization and Audit • Authentication • Identifying the user • Authorization • Authoring • Defining access policies • Implementing Access Policies • Audit • Access audit logs • Operation logs
- 3. 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Agenda • Ranger • What is Ranger? • Why Ranger? • Enabling a component to use Ranger for access control • How to define authorization needs of a component for Ranger integration? • How to enforce authorization policies for component? • Custom extension • Demo on Sample Application • A simple application with Ranger Authorization
- 4. 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Authorization and Auditing w/ Ranger HDFS Ranger Administration Portal HBase Hive Server2 Ranger Policy Store Ranger Audit Store Ranger Plugin HadoopComponents Enterprise Users Ranger Plugin Ranger Plugin Legacy Tools & Data Governance Integration API Knox Storm Ranger Plugin Ranger Plugin EnterpriseServices:Security KafkaRanger Plugin Solr Ranger Plugin YarnRanger Plugin Log4j HDFS Solr
- 5. 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Ranger Admin Service Manager • Delivers a ‘single pane of glass’ for the security administrator • Centralizes administration of security policy • Ensures consistent coverage across the entire Hadoop stack
- 6. 6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Ranger Policy - HBase Allow group “analyst” to have READ access to Order Table and OrderDetails Column Family
- 7. 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Ranger Auditing • Ranger plugins generate detailed audit logs for accesses to protected resources. Audit logs include details like: user, resource, type of access, time of access, client IP address, access-result, ID of the policy that allowed/denied the access • Audit logs to one or more destinations – Solr, HDFS, Log4j, ... • Interactive view of audit logs using Ranger Admin
- 8. 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Tag based policy Classification based security on cross-component data assets
- 9. 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved What does it take to enable my component with ranger? • Service Definition • Defined in a JSON and registered in Ranger Admin • Backbone of the Ranger Stack Model • Ranger Plugin • Provides the necessary interfaces for Authorization and Auditing • Component implements glue code
- 10. 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Ranger Software Architecture • Stack Model • Data driven model • Interpreter of the access-control profile (Service Definition) of the component • Result • Component specific GUI for policy authoring • Policy validation • Policy execution engine • Policy driven Enforcement of access control and Audit logs
- 11. 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Ranger Process Architecture and Modules : Refresher Policy Authoring Policy Validation Policy persistence Policy Synchronization Policy Evaluation Engine Audit Infrastructure Glue code Ranger Admin Component
- 12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Ranger-Enabling a component Component Service Definition Component with Glue code Ranger Enabled Component
- 13. 13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Service Definition • Component’s authorization requirements • What are the resources ? • What are the access operations ? • Advanced Features • Context Enrichment (tag-servicedef) • Dynamic policy conditions (hive-servicedef) • GUI Presentation and rendering • Defined in a JSON for each service. • Stored in Ranger DB • Creating Service Definition • curl call to add the service definition in the Ranger admin curl -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X POST http://node-1.example.com:6080/service/public/v2/api/servicedef -d @sampleapp.json
- 14. 14 © Hortonworks Inc. 2011 – 2017. All Rights Reserved • Description of component • Resources • File/directory • Access Types • Read/Write/Execute • Context Enrichers • Dynamic Policy conditions Demo Component
- 15. 15 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Service Definition JSON
- 16. 16 © Hortonworks Inc. 2011 – 2017. All Rights Reserved • RangerAccessRequest / RangerAccessRequestImpl • RangerAccessResource / RangerAccessResourceImpl • RangerAccessResultProcessor / RangerDefaultAuditHandler • RangerBasePlugin Glue Code Interfaces Component Plugin Request / Resource Result Processor Policy Engine Authorization Interface Policy refresher
- 17. 17 © Hortonworks Inc. 2011 – 2017. All Rights Reserved SampleApp: Authorizer interface
- 18. 18 © Hortonworks Inc. 2011 – 2017. All Rights Reserved SampleApp: Ranger Authorizer implementation
- 19. 19 © Hortonworks Inc. 2011 – 2017. All Rights Reserved Apache Ranger: how to contribute? • Ranger Home : http://ranger.apache.org • Ranger Wiki : https://cwiki.apache.org/confluence/display/RANGER • Ranger JIRAs : https://issues.apache.org/jira/browse/RANGER • Project Mailing Lists • Users : [email protected] • Developers : [email protected] • Commits : [email protected]
Editor's Notes
- #3: We have a lot to cover, want to apologize in advance
- #4: We have a lot to cover, want to apologize in advance
- #5: We have a lot to cover, want to apologize in advance
- #6: We have a lot to cover, want to apologize in advance
- #7: We have a lot to cover, want to apologize in advance
- #8: We have a lot to cover, want to apologize in advance
- #9: We have a lot to cover, want to apologize in advance
- #10: We have a lot to cover, want to apologize in advance
- #14: We have a lot to cover, want to apologize in advance
- #16: We have a lot to cover, want to apologize in advance
- #20: We have a lot to cover, want to apologize in advance