Scalable and Privacy-preserving Data Integration - Part 2

www.scads.de
SCALABLE AND PRIVACY-PRESERVING
DATA INTEGRATION
- PART 2 -
ERHARD RAHM
ZIAD SEHILI
UNIVERSITY OF LEIPZIG

 ScaDS Dresden/Leipzig
 Big Data Integration
 Scalable entity resolution / link discovery
 Large-scale schema/ontology matching
 Holistic data integration
 Privacy-preserving record linkage (PPRL)
 Privacy for Big Data
 PPRL basics
 Scalable PPRL
 Graph-based data integration and analytics
 Introduction
 Graph-based data integration / business intelligence (BIIIG)
 Hadoop-based graph analytics (GRADOOP)
AGENDA
2

Privacy
- right of individuals to determine by themselves when, how and to what
extent information about them is communicated to others (Agrawal
2002)
Privacy threats
• extensive collection of personal/private information / surveillance
• Information dissemination: disclosure of sensitive/confidential
information
• Invasions of privacy: intrusion attacks to obtain access to private
information
• Information aggregation: combining data, e.g. to enhance personal
profiles or identify persons (de-anonymization)
PRIVACY
3

 Protection especially for personally identifiable information (PID)
 name, birthdate, address, email address etc
 healthcare and genetic records, financial records
 criminal justice investigations and proceedings …
 Challenge: preserve privacy despite need to use person-related data
for improved analysis / business success (advertisement,
recommendations), website optimizations, clinical/health studies,
identification of criminals …
 tracking and profiling of web / smartphone / social network users
(different kinds of cookies, canvas fingerprinting …)
 often user agreement needed
INFORMATION / DATA PRIVACY
4

 Need for comprehensive privacy support (“privacy by design”)
 Privacy-preserving publishing of datasets
 Anonymization of datasets
 Privacy-preserving data mining
 analysis of anonymized data without re-identification
 Privacy-preserving record linkage
 object matching with encoded data to preserve privacy
 prerequisite for privacy-preserving data mining
PRIVACY FOR BIG DATA
8

 Anonymization
 removing, generalizing or changing personally identifying attributes
so that people whom the data describe remain anonymous
 no way to identify different records for same person (e.g., different
points in time) or to match/combine records from different sources
 Pseudonymization
 replace most identifying fields within a data record are replaced by
one or more artificial identifiers, or pseudonyms
 one-way pseudonymization (e.g. one-way hash functions) vs. re-
identifiable pseudonymization
 records with same pseudonym can be matched
 improved potential for data analysis
ANONYMIZATION VS PSEUDONYMIZATION
9

RE-IDENTIFICATION OF
„ANONYMOUS DATA“ (SWEENEY 2001)
10
 US voter registration data
 69% unique on postal code (ZIP) and birth date
 87% US-wide with sex, postal code and birth data
 Solution approach: K-Anonymity
 any combination of values appears
at least k times
 generalize values, e.g., on ZIP or birth date

K-ANONYMITY EXAMPLE
11
from: Nielsen et al: Proc BTW 2015

 Physically integrated data (e.g. data warehouse) about persons entails greatest
privacy risks
 Data mining over distributed data can better protect personal data by limiting data
exchange, e.g. using SMC methods
PRIVACY-PRESERVING DATA MINING
12
Local
Data
Local
Data
Local
Data
Warehouse
Data
Mining
Local
Data
Mining
Local
Data
Local
Data
Local
Data
Data
Mining
Combiner
Local
Data
Mining
Local
Data
Mining

HORIZONTAL VS VERTICAL DATA DISTRIBUTION
13
Source: J. Vaidya, C. Clifton: Privacy-Preserving data mining: Why, How, and When. IEEE Security&Privacy 2004

 object matching with encoded data to preserve privacy
 data exchange / integration of person-related data
 many uses cases: medicine, sociology (“population informatics”),
business, …
 privacy aspects
 need to support secure 1-way encoding (pseudonymization)
 protection against attacks to identify persons
 conflicting requirements:
 high privacy
 match effectiveness (need to support fuzzy matches)
 scalability to large datasets and many parties
PRIVACY-PRESERVING RECORD LINKAGE (PPRL)
14

PRIVACY-PRESERVING RECORD LINKAGE
15
data encryption blocking
similarity functions
Vatasalan, Christen, Verykios: A taxonomy of privacy-preserving record linkage techniques. Information Systems 2013

 Two-party protocols
 only two data owners communicate who wish to link their data
 Three-party protocols
 Use of a trusted third party (linkage unit, LU)
 LU will never see unencoded data, but collusion is possible
 Multi-party protocols (> 2 data owners)
 with or without linkage unit
BASIC PPRL CONFIGURATIONS
16

 Simple record linkage protocol
(PP)RL PROTOCOL
www.scads.de 17
Owner 1
Dataset
Linkage Unit
Linkage
algorithm
matches
Owner 2
Dataset
Pairs of matching recordsPairs of matching records
Quality
Scalability

 Thee party protocol (Abstract)
PPRL PROTOCOL
www.scads.de 18
Owner 1
Trusted
Linkage Unit
matches
Owner 2parameters
Dataset Dataset
encryption encryption
Linkage
algorithm
Pairs of IdsPairs of Ids
Privacy
Quality
Scalability

PPRL EXAMPLE IN HEALTH DOMAIN
19
C.W. Kelman, A.J. Bass. C.D. Holman: Research use of linked health data--a best practice protocol. Aust NZ J Public Health. 2002
© Peter Christen, ANU

 Compute a function across several parties, such as no party learns the
information from the other parties, but all receive the final results
 Example 1: millionaire problem
 two millionaires, Alice and Bob, are interested in knowing which of
them is richer but without revealing their actual wealth.
 Example 2: secure summation
SECURE MULTI-PARTY COMPUTATION (SMC)
20

 Adversary models (from crytography)
 Honest-But-Curious: parties follow agreed-on protocols
 Malicious
 Privacy attacks
 Crack data encoding based on background knowledge:
 Frequency attack
 Dictionary attack
 Collusion between parties
PRIVACY ASPECTS
21

 effective and simple encoding uses cryptographic bloom filters
(Schnell et al., 2009)
 tokenize all match-relevant attribute values, e.g. using bigrams or
trigrams
 typical attributes: first name, last name (at birth), sex, date of birth, country of
birth, place of birth
 map each token with a family of one-way hash functions to fixed-size
bit vector (fingerprint)
 original data cannot be reconstructed
 match of bit vectors (Jaccard similarity) is good approximation of
true match result
PPRL WITH BLOOM FILTERS
22

SIMILARITY COMPUTATION - EXAMPLE
23
mas
tho hom oma mantho hom oma mas
0 1 0 0 1 1 0 1 0 0 0 0 1 1 0 1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
h1(mas)= 3 h2(mas)= 7 h3(mas)= 11
tho
hom
oma
man
0 1 0 1 1 1 0 1 0 0 0 1 1 1 0 1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 1 1 0 1 1 0 1 0 0 0 0 1 1 0 1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
SimJaccard (r1, r2) = (r1 ᴧ r2) / (r1 ᴠ r2)
SimJaccard (r1, r2) = 7/11
0 1 0 0 1 1 0 1 0 0 0 0 1 1 0 1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
h1(man)= 2 h2(man)= 0 h3(man)= 13
tho
hom
oma
thomanthomas

 same optimization techniques than for regular
object matching apply
 (private) blocking approaches
 filtering for specific similarity metrics / thresholds
to reduce number of comparisons
 privacy-preserving PPJoin (P4Join)
 metric space: utilize triangular inequality
 parallel linkage (Hadoop, GPUs, …)
SCALABLE PPRL
24

 Many possibilities: standard blocking, sorted-neighborhood, Locality-
sensitive Hashing (LSH) …
 Simple approach for standard blocking
 Each database performs agreed-upon standard blocking, e.g. using
soundex or another function on selected attributes (e.g., names)
 Encoded records are transferred blockwise to LU, possibly with
added noise records for improved security
 LU only matches records per block
 different blocks could be processed in parallel
PRIVATE BLOCKING
25

 one of the most efficient similarity join algorithms
 determine all pairs of records with simJaccard(x,y) ≥ t
 use of filter techniques to reduce search space
 length, prefix, and position filter
 relatively easy to run in parallel
 good candidate to improve scalability for PPRL
 evaluate set bit positions instead of (string) tokens
PP-JOIN: POSITION PREFIX JOIN (XIAO ET AL, 2008)
26

 matching records pairs must have similar lengths
 length / cardinality: number of set bits in bit vector
 Example for minimal similarity t = 0,8:
 record B of length 4 cannot match with C and all records with
greater length (number of set positions), e.g., A
LENGTH FILTER
SimJaccard(x, y) ≥ t ⇒ |x| ≥ | y| ∗ t
Bit vector
0 1 0 1 1 1 1 1 1 1 0 0 0
ID
1 0 1 0 0 0 0 0 1 1 0 0 0
0 0 0 1 1 1 1 1 1 1 0 0 0
B
C
A
card.
8
4
7
length filter
7 * 0.8 = 5.6 > 4

 Similar records must have a minimal overlap α in their sets of tokens (or set bit
positions)
 Prefix filter approximates this test
 reorder bit positions for all fingerprints according to their overall frequency from
infrequent to frequent
 exclude pairs of records without any overlap in their prefixes with
 Example (t = 0.8)
PREFIX FILTER
28
prefix_length(x) =  ( (1-t)∗|x|) + 1 
SimJaccard(x, y) ≥ t ⇔ Overlap(x, y) ≥ α =  ∗ | | | |) ) 
0 0 0 1 1 1prefix fingerprint
1 0 1
0 0 0 1 1 1
0 1 0 1 1
reordered fingerprint
0 1 0 1 1 1 1 1 1 1 0 0 0 0
ID
1 0 1 0 0 0 0 0 1 1 0 0 0 0
0 0 0 1 1 1 1 1 1 1 0 0 0 0
B
C
A
card.
8
4
7
AND operation on prefixes shows non-zero result for C and A so that these records still need
to be considered for matching

 improvement of prefix filter to avoid matches even for overlapping
prefixes
 estimate maximally possible overlap and checking whether it is below the minimal
overlap α to meet threshold t
 original position filter considers the position of the last common prefix token
 revised position filter
 record x, prefix length 9
 record y, prefix length 8
 highest prefix position (here fourth pos. in x) limits possible overlap with
other record: the third position in y prefix cannot have an overlap with x
 maximal possible overlap = #shared prefix tokens (2) + min (9-3, 8-3)= 7
< minimal overlap α = 8
P4JOIN: POSITION FILTER
29
1 1 0 1
1 1 1

 comparison between NestedLoop, P4Join, MultiBitTree
 MultiBitTree: best filter approach in previous work by Schnell
 applies length filter and organizes fingerprints within a binary tree so
that fingerprints with the same set bits are grouped within sub-trees
 can be used to filter out many fingerprints from comparison
 two input datasets R, S
 determined with FEBRL data generator
N=[100.000, 200.000, …, 500.000]. |R|=1/5⋅N, |S|=4/5⋅N
 bit vector length: 1000
 similarity threshold 0.8
EVALUATION
30

 runtime in minutes on standard PC
 similar results for P4Join and Multibit Tree
 relatively small improvements compared to NestedLoop
EVALUATION RESULTS
31
Approach
Dataset size N
100 000 200 000 300 000 400 000 500 000
NestedLoop 6.1 27.7 66.1 122.0 194.8
MultiBitTree 4.7 19.0 40.6 78.2 119.7
P4Join 2.3 15.5 40.1 77.8 125.5

 Operations on bit vectors easy to compute on GPUs
 Length and prefix filters
 Jaccard similarity
 Frameworks CUDA und OpenCL support data-parallel
execution of general computations on GPUs
 program („kernel“) written in C dialect
 limited to base data types (float, long, int, short, arrays)
 no dynamic memory allocation (programmer controls memory
management)
 important to minimize data transfer between main memory and
GPU memory
GPU-BASED PPRL
32

 partition inputs R and S (fingerprints sorted by length) into equally-
sized partitions that fit into GPU memory
 generate match tasks per pair of partition
 only transfer to GPU if length intervals per partition meet length
filter
 optional use of CPU thread to additionally match on CPU
EXECUTION SCHEME
S0 S1 S2 S3 S4
R0
R1
R2
R3
GPU thread
CPU thread(s)
Match task
matches
MR0,S3
GPU memory
Replace
S3 with S4
Read
MR0,S4
Kernel0
Kernel|R0|‐1
…
GPU
0‐S4
|R0|‐1‐S4
bitsR0 S4
bits
R0 S3
change S3 with S4
33
bitscardprefix
bitscardprefix
bits
bitscardprefix
bitscardprefix
Main memory (host)

100 000 200 000 300 000 400 000 500 000
GForce GT 610 0.33 1.32 2.95 5.23 8.15
GeForce GT 540M 0.28 1.08 2.41 4.28 6.67
GPU-BASED EVALUATION RESULTS
34
GeForce GT 610
• 48 Cuda Cores@810MHz
• 1GB
• 35€
GeForce GT 540M
• 96 Cuda Cores@672MHz
• 1GB
 improvements by up to a factor of 20, despite low-profile graphic cards
 still non-linear increase in execution time with growing data volume

 Characteristics of distance functions d for metric spaces:
 d(x, x) = 0 reflexivity
 d(x, y) 0 positiveness
 d(x, y) = d(y, x) symmetry
 d(x, y) + d(y, z) d(x, z) triangular inequality
 Sample metric-space distance functions: Euclidean distance, edit
distance, Hamming distance, Jaccard coefficient
 distance = 1 – similarity
METRIC SPACE-BASED DISTANCE FUNCTIONS
35www.scads.de
z
y
x
d(x, z)

 utilize triangle equality to avoid similarity computations
 consider distances for query object q to pivot object p
 check whether we need to determine distance d(o,q) to object o for
given similarity threshold (maximal distance max-dist=rad(q))
 From triangle inequality we know:
 d(p,o) + d(o,q) ≥ d(p,q) ⇒ d(p,q) – d(p,o) <= d(o,q)
 we can safely exclude object o if d(p,q) – d(p,o) > max-dist = rad(q)
REDUCTION OF SEARCH SPACE
36www.scads.de
q
o
p
rad(q)
d(p, q)

 Preprocessing:
 Given: a set of objects (fingerprints) S1 to be matched
 choose a subset P={p1,…, pj} of objects to be the pivots
 different possibilities, e.g., random subset
 assign each object oi to the nearest pivot pj and save the distance d(pj,oi),
determine radius(p) as maximal distance from p to its assigned objects
 Matching:
 Given: set of query objects (fingerprints) S2, threshold max-dist
 for each q of S2 and each pivot p do:
 determine distance d(p,q)
 If d(p,q) <= rad(p)+max-dist:
for each object assigned to p
 check whether it can be skipped due to triangular inequality (d(p,q)-d(p,o)> max-dist)
 otherwise match o with p (o matches q if d(o,q) <= max-dist)
LINKAGE APPROACH
37www.scads.de

 Comparison with previous approaches using the same datasets
 Runtime in minutes (using faster PC than in previous evaluation)
 Pivot-based approach shows the best results and is up to 40X faster
than other algorithms
EVALUATION
38www.scads.de
Algorithms
Datasets
100 000 200 000 300 000 400 000 500 000
NestedLoop 3.8 20.8 52.1 96.8 152.6
MultiBitTree 2.6 11.3 26.5 50.0 75.9
P4Join 1.4 7.4 24.1 52.3 87.9
Pivots (metric space) 0.2 0.4 0.9 1.3 1.7

 Privacy for Big Data
 privacy-preserving publishing / record linkage / data mining
 tradeoff between protection of personal/sensitive data and data
utility for analysis
 complete anonymization prevents record linkage -> 1-way
pseudonymization of sensitive attributes good compromise
 Scalable Privacy-Preserving Record Linkage
 bloom filters allow simple, effective and relatively efficient match
approach
 performance improvements by blocking / filtering / parallel PPRL
 effective filtering by P4Join and utilizing metric-space distance
functions
 GPU usage achieves significant speedup
SUMMARY
39

 High PPRL match quality for real, dirty data
 Quantitative evaluation of privacy characteristics
 Efficient SCM approaches for multiple sources without linkage unit
 Combined study of PPRL + data mining
OUTLOOK / CHALLENGES
40

 E.A. Durham, M. Kantarcioglu, Y. Xue, C. Tóth, M.Kuzu, B.Malin: Composite Bloom Filters for Secure Record Linkage. IEEE Trans.
Knowl. Data Eng. 26(12): 2956-2968 (2014)
 B. Fung, K. Wang, R.Chen, P.S Yu: Privacy-preserving data publishing: A survey of recent developments. ACM CSUR 2010
 R. Hall, S. E. Fienberg: Privacy-Preserving Record Linkage. Privacy in Statistical Databases 2010: 269-283
 D.Karapiperis, V. S. Verykios: A distributed near-optimal LSH-based framework for privacy-preserving record linkage. Comput.
Sci. Inf. Syst. 11(2): 745-763 (2014)
 C.W. Kelman, A.J. Bass. C.D. Holman: Research use of linked health data--a best practice protocol. Aust NZ J Public Health.
2002
 R.Schnell, T. Bachteler, J. Reiher: Privacy-preserving record linkage using Bloom filters. BMC Med. Inf. & Decision Making 9: 41
(2009)
 Schnell, R.: Privacy-preserving Record Linkage and Privacy-preserving Blocking for Large Files with Cryptographic Keys Using
Multibit Trees. Proc. Joint Statistical Meetings, American Statistical Association, 2013
 Z. Sehili, L. Kolb, C. Borgs, R. Schnell, E. Rahm: Privacy Preserving Record Linkage with PPJoin. Proc. BTW Conf. 2015
 Z. Sehili, E. Rahm: Speeding Up Privacy Preserving Record Linkage for Metric Space Similarity Measures. TR, Univ. Leipzig, 2016
 J. Vaidya, C. Clifton: Privacy-Preserving data mining: Why, How, and When. IEEE Security&Privacy, 2004
 D. Vatasalan, P- Christen, V.S. Verykios: An evaluation framework for privacy-preserving record linkage. Journal of Privacy and
Confidentialty 2014
 D. Vatasalan, P- Christen, C.M. O’Keefe, V.S. Verykios: A taxonomy of privacy-preserving record linkage techniques. Information
Systems 2013
 C. Xiao, W. Wang, X. Lin, J.X. Yu: Efficient Similarity Joins for Near Duplicate Detection. Proc. WWW 2008
REFERENCES
41

Scalable and Privacy-preserving Data Integration - Part 2

More Related Content

Similar to Scalable and Privacy-preserving Data Integration - Part 2 (20)

Recently uploaded (20)

Scalable and Privacy-preserving Data Integration - Part 2