Securing Data in Hybrid on-premise and Cloud Environments using Apache Ranger

© Cloudera, Inc. All rights reserved.
SECURING DATA IN HYBRID ENVIRONMENTS
USING APACHE RANGER
Don Bosco Durai, Privacera
Apache Ranger PMC
Madhan Neethiraj, Cloudera
Apache Ranger PMC, Apache Atlas PMC

© Cloudera, Inc. All rights reserved. 2© Cloudera, Inc. All rights reserved.
DISCLAIMER
• This document may contain product features and technology directions that are under
development, may be under development in the future or may ultimately not be developed.
• Project capabilities are based on information that is publicly available within the Apache
Software Foundation project websites ("Apache"). Progress of the project capabilities can be
tracked from inception to release through Apache, however, technical feasibility, market
demand, user feedback and the overarching Apache Software Foundation community
development process can all effect timing and final delivery.
• This document’s description of these features and technology directions does not represent a
contractual commitment, promise or obligation from Cloudera and Privacera to deliver these
features in any generally available product.
• Product features and technology directions are subject to change, and must not be included in
contracts, purchase orders, or sales agreements of any kind.
• Since this document contains an outline of general product development plans, customers
should not rely upon it when making purchasing decisions.

ABOUT PRIVACERA
Privacera Confidential
CLOUDACCESS MANAGER CLOUD
DISCOVERY
Storage
SQL
No SQL
Streaming,
Serverless,
ML
CLOUD
ANONYMIZATION

AGENDA
Apache Ranger overview
Security Challenges Hybrid Deployment
Implementing Hybrid Security using Ranger
New Features: Security Zones, Role Based Access Control, Conditions at Policy Scope
Demo
Questions

APACHE RANGER: OVERVIEW - HISTORY
Jul 2014
Enters Incubation
Nov 2014
Ranger 0.4.0
Jun 2015
Ranger 0.5.0
x
Jul 2016
Ranger 0.6.0
Nov 2016
Ranger 0.6.2x
Jan 2017
Ranger TLP
graduation!
Jun 2017
Ranger 0.7.1
Mar 2018
Ranger 1.0.0
• Committers: 29
• Contributors
from:
eBay, MSFT,
Huawei, Pandora,
Accenture, ING,
Talend, ZTE
Ranger 1.1.0Ranger 0.7.x
• Tag based Masking
• Export/import of Policies
• $User and macros
• User Sync Nested LDAP
Support
• Plugin status tab
• “Show columns” and
“describe extended support”
• Incremental LDAP Sync
• Time based policies
• Metadata security
• Audit only (compliance) role
• Hive UDF usage authorization
• Show Hive query in audits
• Policy labels
• Audit enhancements
Feb 2017
Ranger 0.7.0
Jul 2018
Ranger 1.1.0
May 2014
XASecure
Acquisition
Ranger 2.0.0
~May 2019
Ranger 2.0.0
Oct 2018
Ranger 1.2.0
Jan 2016
Ranger 0.5.1
Aug 2016
Ranger 0.6.1
• Hadoop3 version updates
• Security zones
• Policy level custom
conditions
• Role based authorization
• DB Schema optimization
for faster policy CRUD
• Hadoop Trusted-proxy
authentication

APACHE RANGER: OVERVIEW – FEATURES
• Centralized policy administration
• Centralized auditing
• Dynamic row filtering
• Dynamic data masking
• Tag based authorization and data-masking policies
• Rich & extendable policy enforcement engine
• Key Management System (KMS)
• New Feature: Security Zones
• New Feature: Support for Roles Based Access Control
• New Feature: Conditions at policy scope

APACHE RANGER: OVERVIEW – CENTRALIZED AUTHORIZATION

SECURITY IN HYBRID ENVIRONMENT

HYBRID DEPLOYMENT: OVERVIEW
On Premise
HDFS Hive Kafka Spark
Hive
Ranger
HDInsight
HiveSpark
EMR
Ranger
Ranger DB
Presto
Security
Admins
Data
Stewards

HYBRID DEPLOYMENT: SECURITY CHALLENGES
• Every environment has different security model
• Access policies needs to be set in each environment
• Policies needs to be consistent
• The granularity of access control are not the same
• Policies can go out of sync very soon
• Regulation and compliance requirements on what data
can be copied to cloud and whether it should be
encrypted or deidentified

HDInsight
Option #1
Restrict Data from On-premise
Option #2
Centralized Ranger

HYBRID DEPLOYMENT: OPTION #1
• Filter & Redact data copied to cloud
• Use Hive to export data to S3
• Apply Ranger Row Level Filtering and Column Masking on ETL user (e.g.
s3etl)
• Setup cloud native access policies for copied data

APACHE RANGER: ROW-FILTER, COLUMN-MASKING POLICIES
ID CONSENT TAX_ID NAME EMAIL
1 Y 123456789 John john@acme.com
2 Y 987654321 Jane jane@acme.com
3 N 789654123 Mary mary@acme.com
4 Y 321789654 David david@acme.com
5 N 456321789 Max max@acme.com
ID CONSENT TAX_ID NAME EMAIL
1 Y xxxxxxxxxx John dkrx@acme.com
2 Y xxxxxxxxxx Jane yafe@acme.com
4 Y xxxxxxxxxx David aumd2@acme.com

APACHE RANGER: ROW-FILTER, COLUMN-MASKING POLICIES

HYBRID DEPLOYMENT: OPTION #1 – PROS AND CONS
• Advantages
• Simple to implement
• Fine grained policies enforced on premise using Filtering, Redaction and Transformation
• Use cloud security policy for coarse grain policies
• Make data accessible to non-Ranger supported services like AWS Redshift, AWS Athena,
SageMaker, etc.
• Limitation
• Not real-time
• If policies changes, then data need to be recopied to cloud
• Need to manage policies on both the sides

HYBRID DEPLOYMENT: OPTION #2 - CENTRALIZED SECURITY
On Premise
HDFS Hive Kafka Spark
Hive
Ranger
HDInsight
HiveSpark
EMR
Ranger
Ranger DB
Presto
Security
Admins
Data
Stewards

HYBRID DEPLOYMENT: OPTION #2
• Common Ranger Admin or Ranger Database for all environments
• Single Ranger to manage the policies for all environments
• If you are using the same name for resources, e.g. Database, Table and
Column name, then a same policy would be used by all the environments
• Tag-based policies can be used to authorize access to cloud-specific data as
well
• Use new Ranger features under development to support central policy
management
• Security Zone
• Scoped Policy
• Roles in Ranger

HYBRID DEPLOYMENT: OPTION #2 – PROS AND CONS
• Advantages
• Centrally Manage security policies for all environments
• Policy changes applied in real-time in all environments
• Leverage Tag Based policies for consistent behavior
• Increasing support for Ranger by 3rd party vendors. Privacera, StarBurst, Dremio, Microsoft,
EMC Isilon, etc.
• Limitation
• Need reliable and secure network connectivity between premise and cloud (site to site VPN)
• All cloud components might be not supported by Open Source Ranger.
• Ranger integration for cloud environment is not supported by the community and will require
additional setup in the cloud services/deployments

PRIVACERA EXTENSION TO APACHE RANGER

DEMO

SECURITY ZONES

APACHE RANGER: SECURITY ZONES - INTRODUCTION
• Partition resources for easier administration of security policies
• Policies in a zone are applied only for resources included in the
zone. For example:
• a landing zone policy for db=* applies only for the resources of landing
zone. It will not impact other resources, like db=marketing
• Policy administration for each zone can be delegated to specific
users/groups
Zone HDFS Hive HBase Kafka
landing /landing/ db=*landing
staging /staging/ db=*staging table=*staging
marketing /marketing db=marketing table=marketing topic=mktg_campaign

APACHE RANGER: SECURITY ZONES - INTRODUCTION
• Audit log includes zone name, allows to quickly filter accesses to
resources of a zone
• REST API for Security Zone administration
• Example use cases:
• ‘on-prem’ zone for resources that should only be accessible from on-prem
clusters
• ‘test-data’ zone for resources that can be used for test purposes by wider
set of users/groups, without impacting production data

APACHE RANGER: SECURITY ZONES - ADMINISTRATION

APACHE RANGER: SECURITY ZONES - POLICY ADMINISTRATION
• Users see only zones in
which they have admin
privileges
• Zone support extends to
access, data-masking,
row-filter and tag-based
policies

APACHE RANGER: SECURITY ZONES – AUDIT LOGS
• Shows zone of the
accessed resource
• Audits can be filtered by
zone
• Only policies in zone of
the accessed resource
are used to authorize

ROLE BASED ACCESS CONTROL

APACHE RANGER: ROLE BASED ACCESS CONTROL - INTRODUCTION
• Ranger policy model extended to support roles
• RBAC is widely used in enterprise applications & cloud environments
• Roles can be used in
• resource-based authorization policies
• tag-based authorization policies
• data-masking policies
• row-filtering policies
• Role management REST API

APACHE RANGER: ROLE BASED ACCESS CONTROL – ROLE ADMIN

APACHE RANGER: ROLE BASED ACCESS CONTROL - POLICY

CONDITIONS AT POLICY SCOPE

APACHE RANGER: CONDITIONS AT POLICY SCOPE - INTRODUCTION
• Conditions can now be set at policy scope, in addition to policy-item scope
• Simplifies use of conditions in policies
• Example use cases:
• Policies specific to access cluster i.e. on-prem, cloud
• Multiple policies for a given tag, for different tag-attribute values
i.e. PII type=email, PII: type=ccn

APACHE RANGER: CONDITIONS AT POLICY SCOPE - SAMPLE
Access cluster type: cloud

APACHE RANGER: CONDITIONS AT POLICY SCOPE - SAMPLE
tagAttr.type == ‘ccn’
tagAttr.type == ‘email’

THANK YOU

Securing Data in Hybrid on-premise and Cloud Environments using Apache Ranger

More Related Content

What's hot (20)

Similar to Securing Data in Hybrid on-premise and Cloud Environments using Apache Ranger (20)

More from DataWorks Summit (20)

Recently uploaded (20)

Securing Data in Hybrid on-premise and Cloud Environments using Apache Ranger