Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
6 likes1,738 views
The document provides an overview of security considerations and practices within the Hadoop ecosystem, highlighting the importance of securing multi-tenant environments and compliance with data access policies. It covers key security components such as authentication, authorization, confidentiality, and auditing, and discusses tools and integration with systems like Kerberos and LDAP. Recommendations for practical security deployment and administration in Hadoop are also included to ensure secure operations and management.
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
- 1. Securing the Hadoop Ecosystem Patrick Angeles Big Data Warehouse Meetup Feb 10, 2014
- 2. Why is Security Important?
- 3. About Me Hadooping for 5+ years • Responsible for several secure Hadoop deployments • Did e-commerce and consumer analytics (PCI, PII, etc.) • Crypto and PKI in a previous life. •
- 4. Why Secure Hadoop? • Multi-tenancy • • You want your cluster to store data and run workloads from multiple users and groups Compliance • You have policies on which personnel can view what data
- 5. Agenda Hadoop Ecosystem Interactions • Security Concepts • Security in Practice • • • IT Infrastructure Integration Deployment Recommendations
- 6. Hadoop on its Own WebHdfs client HDFS client Hadoop NN SNN DN TT Map Task DN TT Map Task DN TT Reduce Task HttpFS MR client hdfs, httpfs & mapred users JT end users protocols: RPC/data transfer/HTTP
- 7. Hadoop and Friends service users end users clients protocols: RPCs/data/HTTP/Thrift/Avro-RPC services clients Hbase Zookeeper RPC Hbase RPC Zookeeper Oozie HTTP Oozie WebHdfs Pig HTTP Hue Crunch HTTP browser HTTP Cascading MapRed RPC Hadoop RPC Flume Sqoop Impala Hive Hive Metastore Thrift Avro RPC Thrift Flume Impala
- 8. Security Concepts Authentication • Authorization • Confidentiality • • • Encryption Auditing • Traceability
- 9. Authentication • End Users to Services, as a user • • • • Services to Services, as a service • • • CLI & libraries: Kerberos (kinit or keytab) Web UIs: Kerberos SPNEGO & pluggable HTTP auth MR tasks use delegation tokens Credentials: Kerberos (keytab) Client SSL certificates (for shuffle encryption) Services to Services, on behalf of a user • Proxy-user (after Kerberos for service)
- 10. Authorization • HDFS Data • • HBase Data • • Fine-grained authorization through Apache Sentry (Incubating) Jobs (Hadoop, Oozie) • • Read/Write Access Control Lists (ACLs) at table level Hive Server 2 and Impala • • File System permissions (Unix like user/group permissions) Job ACLs for Hadoop Scheduler Queues, manage & view jobs Zookeeper • ACLs at znodes, authenticated & read/write
- 11. Confidentiality • Data in transit RPC: using SASL • HDFS data: using SASL • HTTP: using SSL (web UIs, shuffle). Requires SSL certs • • Data at rest Nothing out of the box • Doable by: custom ‘compression’ codec or local file system encryption •
- 12. Auditing • Who accessed (read/write) FS data • • • Who submitted, managed, or viewed a Job or a Query • • NN audit log contains all file opens, creates NN audit log contains all metadata ops, e.g. rename, listdir JT, RM, and Job History Server logs contain history of all jobs run on a cluster Who submitted, managed, or viewed a workflow • Oozie audit logs contain history of all user requests
- 13. Auditing Gaps • Not all projects have explicit audit logs • • • It is difficult to correlate jobs & data access • • • Audit-like information can be extracted by processing logs Eg: Impala query logs are distributed across all nodes Eg: Map-Reduce jobs launched by Pig job Eg: HDFS data accessed by a Map-Reduce job Tools written on top of Hadoop can do this well
- 15. Integration: Kerberos Users don’t want Yet Another Credential • Corp IT doesn’t want to provision thousands of service principals • Solution: local KDC + one-way trust • Run a KDC (usually MIT Kerberos) in the cluster • • • Put all service principals here Set up one-way trust of central corporate realm by local KDC • Normal user credentials can be used to access Hadoop
- 16. Integration: Groups • Much of Hadoop authorization uses “groups” • • Users’ groups are not stored in Hadoop anywhere • • • User ‘patrick’ might belong to groups ‘analysts’, ‘eng’, etc. Refers to external system to determine group membership NN/JT/Oozie/Hive servers all must perform group mapping Default plugins for user/group mapping: • • • ShellBasedUnixGroupsMapping – forks/runs `/bin/id’ JniBasedUnixGroupsMapping – makes a system call LdapGroupsMapping – talks directly to an LDAP server
- 17. Integration: Kerberos + LDAP Central Active Directory LDAP group mapping [email protected] … Hadoop Cluster NN JT Local KDC Cross-realm trust hdfs/[email protected] yarn/[email protected] …
- 18. Integration: Web Interfaces • Most web interfaces authenticate using SPNEGO • • • • Standard HTTP authentication protocol Used internally by services which communicate over HTTP Most browsers support Kerberos SPNEGO authentication Hadoop components which use servlets for web interfaces can plug in custom filter • Integrate with intranet SSO HTTP solution
- 19. Recommendations • Security configuration is a PITA • • Do only what you really need Enable cluster security (Kerberos) only if un-trusted groups of users are sharing the cluster • Otherwise use edge-security to keep outsiders out Only enable wire encryption if required • Only enable web interface authentication if required •
- 20. Security Enablement • Secure Hadoop enablement order 1. 2. 3. 4. 5. 6. 7. HDFS RPC (including SNN check-pointing) JobTracker RPC TaskTrackers RPC & LinuxTaskControler Hadoop web UI Configure monitoring to work with security Other services (HBase, Oozie, Hive Metastore, etc) Continue with authorization and network encryption if needed
- 21. Administration • Use an admin/management tool • • • Several inter-related configuration knobs To manage principals/keytabs creation and distribution Automatically configures monitoring for security
- 22. Q&A
Editor's Notes
- #10: Proxy-user setup:Relying party is configured to recognized super-users who are allowed to impersonate