CHhhjjuiiiiiiiiiiS18003 Unit 3 Class ppt.pptx

Mining Data Streams
(Unit III)
Mining of Massive Datasets
Jure Leskovec, Anand Rajaraman, Jeff Ullman
Stanford University
http://www.mmds.org

Topics to Learn
◼ Introduction to Streams Concepts – Stream data model and architecture -
Stream Computing, Sampling data in a stream – Filtering streams –
Counting distinct elements in a stream – Estimating moments – Counting
oneness in a window – Decaying window - Realtime Analytics Platform
(RTAP) applications - Case studies - real time sentiment analysis, stock
market predictions.
Mining of Massive Datasets, http://www.mmds.org 2
*

New Topic: Infinite Data
*

Data Streams
◼Data arrives in a stream or streams, and if it is not
processed immediately or stored, then it is lost
forever
◼In many data mining situations, we do not know the
entire data set in advance
◼Stream Management is important when the input rate
is controlled externally:
▪ Google queries
▪ Twitter or Facebook status updates
◼We can think of the data as infinite and
non-stationary (the distribution changes
over time)
*

11/10/2022 Mining Data Streams 5
assumption: data arrives in a stream or streams,
and if it is not processed immediately or stored, then it is
lost forever. Moreover,
we shall assume that the data arrives so rapidly that it is
not feasible to store
it all in active storage (i.e., in a conventional database),
and then interact with
it at the time of our choosing.

6
The Stream Model
◼Input elements enter at a rapid rate,
at one or more input ports (i.e., streams)
▪We call elements of the stream tuples
◼The system cannot store the entire stream
accessibly
◼Q: How do you make critical calculations
about the stream using a limited amount of
(secondary) memory?
Mining of Massive Datasets, http://www.mmds.org
*

Side note: SGD is a Streaming Alg.
◼Stochastic Gradient Descent (SGD) is an
example of a stream algorithm
◼In Machine Learning we call this: Online Learning
▪Allows for modeling problems where we have
a continuous stream of data
▪We want an algorithm to learn from it and
slowly adapt to the changes in data
◼Idea: Do slow updates to the model
▪SGD (SVM, Perceptron) makes small updates
▪So: First train the classifier on training data.
▪Then: For every example from the stream, we slightly
update the model (using small learning rate)
*

Applications
◼ Mining query streams
▪ Google wants to know what queries are more frequent today than yesterday
◼ Mining click streams
▪ Yahoo wants to know which of its pages are getting an unusual number of hits in
the past hour
◼ Mining social network news feeds
E.g., look for trending topics on Twitter, Facebook
◼ Sensor Networks
▪ Many sensors feeding into a central controller
◼ Telephone call records
▪ Data feeds into customer bills as well as settlements between telephone companies
◼ IP packets monitored at a switch
▪ Gather information for optimal routing
▪ Detect denial-of-service attacks
*

Examples of Stream Sources
◼ Sensor Data
▪ Data produced is a stream of real numbers. It is not a very interesting
stream, since the data rate is so low. Many sensors feeding into a
central controller.
◼ Image Data
▪ Satellites often send down to earth streams consisting of many
terabytes of images per day.
◼ Internet and Web Traﬃc
▪ Google receives several hundred million search queries per day.
Yahoo! accepts billions of “clicks” per day on its various sites. Many
interesting things can be learned from these streams.
* Mining of Massive Datasets, http://www.mmds.org 9

Sensor Data Example

What is a Data Stream?
◼ A real-time, continuous, ordered (implicitly by arrival time of
explicitly by timestamp) sequence of items. It is impossible to
control the order in which items arrive, nor it is feasible to
locally store a stream in its entirety.
1. continous and sequential input
2. typically unpredictable input rate
3. can be large amounts of data
4. not error free

Database Management Vs Data
Stream Management

General Stream Processing Model
Processor
Limited
Working
Storage
. . . 1, 5, 2, 7, 0, 9, 3
. . . a, r, v, t, y, h, b
. . . 0, 0, 1, 0, 1, 1, 0
time
Streams Entering.
Each is stream is
composed of
elements/tuples
Ad-Hoc
Queries
Output
Archival
Storage
Standin
g
Queries
*
assume it is not
possible to
answer queries
from the archival
store
summaries or parts of streams
may be placed, and which can be
used for answering queries
11/10/2022 Mining Data Streams 14

Stream Queries
◼Two ways that queries get asked about streams
▪ standing queries
▪are stored, permanently executing, and produce outputs
at appropriate times
▪ ad-hoc queries
▪a question asked once about the current state of a
stream or streams.

Issues in Stream Processing
◼ Streams often deliver elements very rapidly.
▪ We must process elements in real time, or we lose the opportunity to
process them at all, without accessing the archival storage
◼ Streams are “slow
▪ Requirements of all the streams together can easily exceed the
amount of available main memory

Problems on Data Streams
◼Types of queries one wants on answer on
a data stream:
▪Sampling data from a stream
▪Construct a random sample
▪Queries over sliding windows [ad-hoc queries]
▪Number of items of type x in the last k elements
of the stream
*

Problems on Data Streams
◼Types of queries one wants on answer on
a data stream:
▪Filtering a data stream
▪Select elements with property x from the stream
▪Counting distinct elements
▪Number of distinct elements in the last k elements
of the stream
▪Estimating moments
▪Estimate avg./std. dev. of last k elements
▪Finding frequent elements
*

Sampling from a Data Stream:
Sampling a fixed proportion
As the stream grows the sample also gets
bigger

Sampling from a Data Stream
◼Since we can not store the entire stream,
one obvious approach is to store a sample
◼Two different problems:
▪(1) Sample a fixed proportion of elements
in the stream (say 1 in 10)
▪(2) Maintain a random sample of fixed size
over a potentially infinite stream
▪At any “time” k we would like a random sample
of s elements
▪What is the property of the sample we want to maintain?
For all time steps k, each of k elements seen so far has
equal prob. of being sampled
*

Sampling a Fixed Proportion
◼Problem 1: Sampling fixed proportion
◼Scenario: Search engine query stream
▪Stream of tuples: (user, query, time)
▪Answer questions such as: How often did a user
run the same query in a single day?
▪Have space to store 1/10th
of query stream
◼Naive solution:
▪Generate a random integer in [0..9] for each query
▪Store the query if the integer is 0, otherwise
discard
*

Problem with Naive Approach
◼
*

Solution: Sample Users
Solution:
◼ Pick 1/10th
of users and take all their searches in the sample
◼ Use a hash function that hashes the user name or user id
uniformly into 10 buckets
*

Generalized Solution
◼Stream of tuples with keys:
▪Key is some subset of each tuple’s components
▪e.g., tuple is (user, search, time); key is user
▪Choice of key depends on application
◼To get a sample of a/b fraction of the
stream:
▪Hash each tuple’s key uniformly into b buckets
▪Pick the tuple if its hash value is at most a
Hash table with b buckets, pick the tuple if its hash value is at most a.
How to generate a 30% sample?
Hash into b=10 buckets, take the tuple if it hashes to one of the first 3 buckets
*

Sampling from a Data Stream:
Sampling a fixed-size sample
As the stream grows, the sample is of
fixed size

Maintaining a fixed-size sample
◼Problem 2: Fixed-size sample
◼Suppose we need to maintain a random
sample S of size exactly s tuples
▪E.g., main memory size constraint
◼Why? Don’t know length of stream in advance
◼Suppose at time n we have seen n items
▪Each item is in the sample S with equal prob. s/n
How to think about the problem: say s = 2
Stream: a x c y z k c d e g…
At n= 5, each of the first 5 tuples is included in the sample S with equal prob.
At n= 7, each of the first 7 tuples is included in the sample S with equal prob.
Impractical solution would be to store all the n tuples seen
so far and out of them pick s at random
*

Solution: Fixed Size Sample
◼Algorithm (a.k.a. Reservoir Sampling)
▪Store all the first s elements of the stream to S
▪Suppose we have seen n-1 elements, and now
the nth
element arrives (n > s)
▪With probability s/n, keep the nth
element, else discard it
▪If we picked the nth
element, then it replaces one of the
s elements in the sample S, picked uniformly at random
◼Claim: This algorithm maintains a sample S
with the desired property:
▪After n elements, the sample contains each
element seen so far with probability s/n
*

Reservoir Sampling Example

Proof: By Induction
◼We prove this by induction:
▪Assume that after n elements, the sample contains
each element seen so far with probability s/n
▪We need to show that after seeing element n+1
the sample maintains the property
▪Sample contains each element seen so far with
probability s/(n+1)
◼Base case:
▪After we see n=s elements the sample S has the
desired property
▪Each out of n=s elements is in the sample with
probability s/s = 1
*

Proof: By Induction
◼
Element n+1
discarded
Element n+1
not discarded
Element in the
sample not picked
*

Queries over a
(long) Sliding Window

Sliding Windows
◼ A useful model of stream processing is that queries
are about a window of length N –
the N most recent elements received
◼ Interesting case: N is so large that the data cannot
be stored in memory, or even on disk
▪ Or, there are so many streams that windows
for all cannot be stored
◼ Amazon example:
▪ For every product X we keep 0/1 stream of whether that
product was sold in the n-th transaction
▪ We want answer queries, how many times have we sold
X in the last k sales
*

Sliding Window: 1 Stream
◼Sliding window on a single stream:
q w e r t y u i o p a s d f g h j k l z x c v b n m
Past Future
N = 6
*

Exercise

Solution
◼ Use hash function to generate random numbers [0..19] for
each incoming tuple
◼ Store tuple if random number = 0 else discard
A)Estimate the average number of students in a course

More algorithms for streams
▪(1) Filtering a data stream: Bloom filters
▪Select elements with property x from stream
▪(2) Counting distinct elements: Flajolet-Martin
▪Number of distinct elements in the last k elements
of the stream
▪(3) Estimating moments: AMS method
▪Estimate std. dev. of last k elements
▪(4) Counting frequent items
*

Filtering Data Streams
◼Each element of data stream is a tuple
◼Given a list of keys S
◼Determine which tuples of stream are in S
◼Obvious solution: Hash table
▪But suppose we do not have enough memory to
store all of S in a hash table
▪E.g., we might be processing millions of filters
on the same stream
38
*

Applications
◼Example: Email spam filtering
▪We know 1 billion “good” email addresses
▪If an email comes from one of these, it is NOT
spam
◼Publish-subscribe systems
▪You are collecting lots of messages (news articles)
▪People express interest in certain sets of keywords
▪Determine whether each message matches user’s
interest
39
*

Bloom Filter

First Cut Solution - Bloom Filter
◼Given a set of keys S that we want to filter
◼Create a bit array B of n bits, initially all 0s
◼Choose a hash function h with range [0,n)
◼Hash each member of s S
∈ to one of
n buckets, and set that bit to 1, i.e., B[h(s)]=1
◼Hash each element a of the stream and
output only those that hash to bit that was set
to 1
▪Output a if B[h(a)] == 1
41
*

First Cut Solution (2)
◼Creates false positives but no false negatives
▪If the item is in S we surely output it, if not we may
still output it
42
Filter
Ite
m
0010001011000
Output the item since it may be in S.
Item hashes to a bucket that at least
one of the items in S hashed to.
Hash
func
h
Drop the item.
It hashes to a bucket set
to 0 so it is surely not in S.
Bit array
B
*

First Cut Solution (3)
◼|S| = 1 billion email addresses
|B|= 1GB = 8 billion bits
◼If the email address is in S, then it surely hashes
to a bucket that has the big set to 1,
so it always gets through (no false negatives)
◼Approximately 1/8 of the bits are set to 1, so
about 1/8th
of the addresses not in S get through
to the output (false positives)
▪Actually, less than 1/8th
, because more than one
address might hash to the same bit
43
*

Analysis: Throwing Darts (1)
◼More accurate analysis for the number of
false positives
◼Consider: If we throw m darts into n equally
likely targets, what is the probability that
a target gets at least one dart?
◼In our case:
▪Targets = bits/buckets
▪Darts = hash values of items
44
*

◼We have m darts, n targets
◼What is the probability that a target gets at
least one dart?
45
(1 – 1/n)
Probability some
target X not hit
by a dart
m
1 -
Probability at
least one dart
hits target X
n( / n)
Equivalent
Equals 1/e
as n →∞
1 – e–m/n
*

◼Fraction of 1s in the array B =
= probability of false positive = 1 – e-m/n
◼Example: 109
darts, 8 10
∙ 9
targets
▪Fraction of 1s in B = 1 – e-1/8
= 0.1175
▪Compare with our earlier estimate: 1/8 = 0.125
46
*

Bloom Filter
◼Consider: |S| = m, |B| = n
◼Use k independent hash functions h1 ,…, hk
◼Initialization:
▪Set B to all 0s
▪Hash each element s S
∈ using each hash function hi,
set B[hi(s)] = 1 (for each i = 1,.., k)
◼Run-time:
▪When a stream element with key x arrives
▪If B[hi(x)] = 1 for all i = 1,..., k then declare that x is in S
▪That is, x hashes to a bucket set to 1 for every hash function hi(x)
▪Otherwise discard the element x
(note: we have a
single array B!)
*

Bloom Filter -- Analysis
◼What fraction of the bit vector B are 1s?
▪Throwing k m
∙ darts at n targets
▪So fraction of 1s is (1 – e-km/n
)
◼But we have k independent hash functions
and we only let the element x through if all k
hash element x to a bucket of value 1
◼So, false positive probability = (1 – e-km/n
)k
*

Bloom Filter – Analysis (2)
◼m = 1 billion, n = 8 billion
▪k = 1: (1 – e-1/8
) = 0.1175
▪k = 2: (1 – e-1/4
)2
= 0.0493
◼What happens as we
keep increasing k?
◼“Optimal” value of k: n/m ln(2)
▪In our case: Optimal k = 8 ln(2) = 5.54 ≈ 6
▪Error at k = 6: (1 – e-1/6
)2
= 0.0235
Number of hash functions, k
False
positive
prob.
*

Bloom Filter: Wrap-up
◼Bloom filters guarantee no false negatives,
and use limited memory
▪Great for pre-processing before more
expensive checks
◼Suitable for hardware implementation
▪Hash function computations can be parallelized
◼Is it better to have 1 big B or k small Bs?
▪It is the same: (1 – e-km/n
)k
vs. (1 – e-m/(n/k)
)k
▪But keeping 1 big B is simpler
*

(2) Counting Distinct Elements

Counting Distinct Elements
◼Problem:
▪Data stream consists of a universe of elements
chosen from a set of size N
▪Maintain a count of the number of distinct
elements seen so far
◼Obvious approach:
Maintain the set of elements seen so far
▪That is, keep a hash table of all the distinct
elements seen so far
52
*

Applications
◼How many different words are found among
the Web pages being crawled at a site?
▪Unusually low or high numbers could indicate
artificial pages (spam?)
◼How many different Web pages does each
customer request in a week?
◼How many distinct products have we sold in
the last week?
53
*

Using Small Storage
◼Real problem: What if we do not have space
to maintain the set of elements seen so far?
◼Estimate the count in an unbiased way
◼Accept that the count may have a little error,
but limit the probability that the error is large
54
*

Flajolet-Martin Approach
◼Pick a hash function h that maps each of the
N elements to at least log2 N bits
◼For each stream element a, let r(a) be the
number of trailing 0s in h(a)
▪r(a) = position of first 1 counting from the right
▪E.g., say h(a) = 12, then 12 is 1100 in binary, so r(a) = 2
◼Record R = the maximum r(a) seen
▪R = maxa r(a), over all the items a seen so far
◼Estimated number of distinct elements = 2R
55
*

Why It Works: Intuition
◼Very very rough and heuristic intuition why
Flajolet-Martin works:
▪h(a) hashes a with equal prob. to any of N values
▪Then h(a) is a sequence of log2 N bits,
where 2-r
fraction of all as have a tail of r zeros
▪About 50% of as hash to ***0
▪About 25% of as hash to **00
▪So, if we saw the longest tail of r=2 (i.e., item hash
ending *100) then we have probably seen
about 4 distinct items so far
▪So, it takes to hash about 2r
items before we
see one with zero-suffix of length r
*

Why It Works: More formally
◼
57
*

◼
58
Prob. that given h(a) ends
in fewer than r zeros
Prob. all end in
fewer than r zeros.
*

◼Note:
◼Prob. of NOT finding a tail of length r is:
▪If m << 2r
, then prob. tends to 1
▪ as m/2r
→ 0
▪So, the probability of finding a tail of length r tends to 0
▪If m >> 2r
, then prob. tends to 0
▪ as m/2r
→ ∞
▪So, the probability of finding a tail of length r tends to 1
◼Thus, 2R
will almost always be around m!
59
*

Why It Doesn’t Work
◼
60
*

Generalization: Moments
◼Suppose a stream has elements chosen
from a universal set A
◼Assume the universal set is ordered so we
can speak of the ith
element for any i.
◼Let mi be the number of times value i occurs
in the stream
◼The kth
-order moment or kth
moment is the
sum over all i of (mi)k
62
*

Special Cases
◼0th
moment = sum of 1 for each mi i.e > 0
▪ a count of the number of distinct elements in a
stream
◼1st
moment = sum of the mi’s which must be the
length of the stream [Easy to compute]
◼2nd
moment = sum of the squares of the mi’s
▪Called surprise number as it measures how uneven
the distribution of elements in the stream is
63
*

Example: Surprise Number
◼We have a stream of length 100 in which 11
different elements appear
◼Item counts: 10, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9
Surprise number is S = 102
+ 10 × 92
= 910
◼Item counts: 90, 1, 1, 1, 1, 1, 1, 1 ,1, 1, 1
Surprise number is S = 902
+ 10 × 12
= 8,110
*

Problem Computing Moments
◼ No problem computing moments of any order if we can
aﬀord to keep in main memory a count for each element that
appears in the stream.
◼ If we cannot aﬀord to use that much memory, then we need
to estimate the kth
moment by keeping a limited number of
values in main memory and computing an estimate from
these values
◼ Let’s see another form of value that is useful for second and
higher moments.

The Alon-Matias-Szegedy Algorithm
for Second Moments
◼ Let us assume that a stream has a particular length
n.
◼Suppose we do not have enough space to count all
the mi
’s for all the elements of the stream
◼ We can still estimate the second moment of the
stream using a limited amount of space; the more
space we use, the more accurate the estimate will
be.
◼ We compute some number of variables.
◼For each variable X, we store: X.element and X.value

Alon-Matias-Szegedy Algorithm for Second
Moments
◼To determine the value of a variable X,
▪ choose a position in the stream between 1 and n,
uniformly and at random
◼ Set X.element to be the element found there and
initialize X.value to 1.
◼As we read the stream,
▪ add 1 to X.value each time we encounter another
occurrence of X.element.

Example : Alon-Matias-Szegedy Algorithm
◼Suppose the stream is a,b,c,b,d,a,c,d,a,b,d,c,a,a,b
◼The length of the stream is n = 15
◼ Since a appears 5 times, b appears 4 times, and c
and d appear three times each, the second moment
for the stream is 52
+42
+32
+32
= 59

Alon, Matias and Szegedy (AMS)
Method
◼
*

One Random Variable (X)
◼
*

Example
◼ Stream a,b,c,b,d,a,c,d,a,b,d,c,a,a,b
◼ Let’s keep three variables, X1, X2, and X3
◼ “At random” we pick the 3
rd
, 8
th
and 13
th
positions to deﬁne these three variables.
◼ When we reach position 3, we ﬁnd element c, set X1.element =
c and X1.value = 1.
◼ At position 7, X1.element = c and X1.value = 2
◼ At position 8, X2.element = d and X2.value = 1
◼ At position 13, X3.element = a and X3.value = 1
◼ At position 14, X3.element = a and X3.value = 2
◼ Final value : X1.value = 3, X2.value = 2, X3.value = 2
◼ We can derive an estimate of the second moment from any

Example (Contd..)
◼X = n(2X.value − 1) [X1.value = 3, X2.value = 2,
X3.value = 2]
◼X1 =15(2X1.value – 1) = 15(2*3-1) = 75
◼X2 =15(2X2.value – 1) = 15(2*2-1) = 45
◼X3 =15(2X3.value – 1) = 15(2*2-1) = 45
◼ Average of estimates = (75 + 45 +45) / 3
= 165/3 = 55
(a fairly close approximation) to the
True value of the second moment for this stream is
59

Expectation Analysis
◼
a a a a
1 3
2 ma
b b b b
Stream:
Count:
*

Expectation Analysis
◼
Time t
when
the last i is
seen (ct=1)
Time t when
the penultimate
i is seen (ct=2)
Time t
when
the first i is
seen
(ct=mi)
Group
times
by the
value
a a a a
1 3
2 ma
b b b b
Count:
Stream:
mi … total count of
item i in the stream
(we are assuming
stream has length n)
*

Higher-Order Moments
◼
*

Combining Samples
◼
76
*

Streams Never End: Fixups
◼ (1) The variables X have n as a factor –
keep n separately; just hold the count in X
◼ (2) Suppose we can only store k counts.
We must throw some Xs out as time goes on:
▪Objective: Each starting time t is selected with
probability k/n
▪Solution: (fixed-size sampling!)
▪Choose the first k times for k variables
▪When the nth
element arrives (n > k), choose it with
probability k/n
▪If you choose it, throw one of the previously stored
variables X out, with equal probability
*

Exercise
◼Compute the surprise number (second
moment) for the stream 3, 1, 4, 1, 3, 4, 2, 1, 2
What is the third moment of this stream?

Solution
◼stream 3, 1, 4, 1, 3, 4, 2, 1, 2

Counting Ones in a
Window

81
Counting Bits (1)
◼Problem:
▪Given a stream of 0s and 1s
▪Be prepared to answer queries of the form
How many 1s are in the last k bits? where k ≤ N
◼Obvious solution:
Store the most recent N bits
▪When new bit comes in, discard the N+1st
bit
0 1 0 0 1 1 0 1 1 1 0 1 0 1 0 1 1 0 1 1 0 1 1 0
Past Future
Suppose N=6
*

Counting Bits (2)
◼You can not get an exact answer without
storing the entire window
◼Real Problem:
What if we cannot afford to store N bits?
▪E.g., we’re processing 1 billion streams and
N = 1 billion
◼But we are happy with an approximate
answer
82
0 1 0 0 1 1 0 1 1 1 0 1 0 1 0 1 1 0 1 1 0 1 1 0
Past Future
*

An attempt: Simple solution
◼
0 1 0 0 1 1 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 1 1 1 0 0 1 0 1 0 1 1 0 0 1 1 0 1 0
N
Past Future
*

The Datar-Gionis-Indyk-Motwani (DGIM)
Algorithm
◼
[Datar, Gionis, Indyk,
Motwani]
*

Datar-Gionis-Indyk-Motwani (DGIM)
Algorithm – Contd..
◼ Divide the window into buckets, consisting of:
1. The timestamp of its right (most recent) end
2. The number of 1’s in the bucket. This number must be a
power of 2, and we refer to the number of 1’s as the size of
the bucket
3. To represent a bucket, we need log2 N bits to represent the
timestamp (modulo N) of its right end

Six rules that must be followed when
representing a stream by buckets
1. The right end of a bucket is always a position with a 1.
2. Every position with a 1 is in some bucket.
3. No position is in more than one bucket.
4. There are one or two buckets of any given size, up to
some maximum size
5. All sizes must be a power of 2.
6. Buckets cannot decrease in size as we move to the left
(back in time).

A bit-stream divided into buckets following the
DGIM rules
1 0 1 1 0 1 1 0 0 0 1 0 1 1 1 0 1 1 0 0 1 0 1 1 0

Query Answering in the DGIM Algorithm
◼ Suppose stream is 1 0 1 1 0 1 1 0 0 0 1 0 1 1 1 0 1 1 0 0 1 0 1 1 0
and k = 10. Then the query asks for the number of 1’s in the ten
rightmost bits 0 1 1 0 0 1 0 1 1 0
No. of buckets?
2 buckets of size 1, a bucket of size 2, half the bucket of size 4
that is partially within range
1 0 1 1 0 1 1 0 0 0 1 0 1 1 1 0 1 1 0 0 1 0 1 1 0

Maintaining DGIM Conditions
◼Suppose we have a window of length N represented by
buckets satisfying DGIM conditions. Then a new bit
comes in:
▪ Check the leftmost bucket. If its timestamp is not
currentTimestamp – N, the remove
▪ If new bit is 0, do nothing
▪ If new bit is 1, create a new bucket of size 1
▪ If there are now only 2 buckets of size 1, stop
▪ Otherwise, merge previous buckets of size 1 into bucket of size 2
▪If there are now only 2 buckets of size 2, stop
▪Otherwise, merge previous buckets of size 2 into buckets of size 4
▪Etc …
89
Time: At most
log(N), since
there are log(N)
different sizes

Example: Updating Buckets
Slide by Jure Leskovec: Mining Massive Datasets 90
1001010110001011010101010101011010101010101110101010111010100010110010
0010101100010110101010101010110101010101011101010101110101000101100101
0010101100010110101010101010110101010101011101010101110101000101100101
0101100010110101010101010110101010101011101010101110101000101100101101
0101100010110101010101010110101010101011101010101110101000101100101101
0101100010110101010101010110101010101011101010101110101000101100101101

Example
91
• What happens if the next 3 bits are 1,1,1?
… 1 0 1 1 0 1 1 0 0 0 1 0 1 1 1 0 1 1 0 0 1 0 1 1 0
Two of
size 1
One of
size 2
Two of
size 4
0ne of
size 8

Reducing the Error
◼Instead of allowing 1 or 2 of each bucket size, allow r-1
or r of each bucket size for sizes 1, 2, 4, 8, … (and an
integer r>2)
◼Of the smallest and largest size present, we allow there
to be any number of buckets, from 1 to r
▪ Use similar propagation algorithm to that of before
◼Buckets are smaller, so there is tighter bound on error
▪ Can prove that the error is at most 1/r
92

Idea: Exponential Windows
◼Solution that doesn’t (quite) work:
▪Summarize exponentially increasing regions
of the stream, looking backward
▪Drop small regions if they begin at the same point
as a larger region
0 1 0 0 1 1 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 1 1 1 0 0 1 0 1 0 1 1 0 0 1 1 0 1 0
N
?
0
1
1
2
2
3
4
10
6
We can reconstruct the count of the last N bits, except we
are not sure how many of the last 6 1s are included in the N
Window of
width 16
has 6 1s
*

What’s Good?
◼
*

95
What’s Not So Good?
◼As long as the 1s are fairly evenly distributed,
the error due to the unknown region is small
– no more than 50%
◼But it could be that all the 1s are in the
unknown area at the end
◼In that case, the error is unbounded!
0 1 0 0 1 1 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 1 1 1 0 0 1 0 1 0 1 1 0 0 1 1 0 1 0
0
1
1
2
2
3
4
10
6
N
?
*

Fixup: DGIM method
◼Idea: Instead of summarizing fixed-length
blocks, summarize blocks with specific
number of 1s:
▪Let the block sizes (number of 1s) increase
exponentially
◼When there are few 1s in the window, block
sizes stay small, so errors are small
96
1001010110001011010101010101011010101010101110101010111010100010110
010 N
[Datar, Gionis, Indyk,
Motwani]
*

97
DGIM: Timestamps
◼
*

DGIM: Buckets
◼ A bucket in the DGIM method is a record
consisting of:
▪(A) The timestamp of its end [O(log N) bits]
▪(B) The number of 1s between its beginning and
end [O(log log N) bits]
◼ Constraint on buckets:
Number of 1s must be a power of 2
▪ That explains the O(log log N) in (B) above
98
1001010110001011010101010101011010101010101110101010111010100010110
010 N
*

Representing a Stream by Buckets
◼Either one or two buckets with the same
power-of-2 number of 1s
◼Buckets do not overlap in timestamps
◼Buckets are sorted by size
▪Earlier buckets are not smaller than later buckets
◼Buckets disappear when their
end-time is > N time units in the past
*

Example: Bucketized Stream
N
1 of
size 2
2 of
size 4
2 of
size 8
At least 1 of
size 16. Partially
beyond window.
2 of
size 1
1001010110001011010101010101011010101010101110101010111010100010110
010
Three properties of buckets that are maintained:
- Either one or two buckets with the same power-of-2 number of 1s
- Buckets do not overlap in timestamps
- Buckets are sorted by size
*

Updating Buckets (1)
◼When a new bit comes in, drop the last
(oldest) bucket if its end-time is prior to N
time units before the current time
◼2 cases: Current bit is 0 or 1
◼If the current bit is 0:
no other changes are needed
101
*

Updating Buckets (2)
◼ If the current bit is 1:
▪(1) Create a new bucket of size 1, for just this bit
▪ End timestamp = current time
▪(2) If there are now three buckets of size 1,
combine the oldest two into a bucket of size 2
▪(3) If there are now three buckets of size 2,
combine the oldest two into a bucket of size 4
▪(4) And so on …
102
*

Example: Updating Buckets
103
1001010110001011010101010101011010101010101110101010111010100010110
010
0010101100010110101010101010110101010101011101010101110101000101100
101
001010110001011010101010101011010101010101110101010111010100010110
0101
01011000101101010101010101101010101010111010101011101010001011001011
01
010110001011010101010101011010101010101110101010111010100010110010
1101
01011000101101010101010101101010101010111010101011101010001011001011
01
Current state of the stream:
Bit of value 1 arrives
Two orange buckets get merged into a yellow bucket
Next bit 1 arrives, new orange bucket is created, then 0 comes, then 1:
Buckets get merged…
State of the buckets after merging
*

104
How to Query?
◼ To estimate the number of 1s in the most
recent N bits:
1. Sum the sizes of all buckets but the last
(note “size” means the number of 1s in the bucket)
2. Add half the size of the last bucket
◼ Remember: We do not know how many 1s
of the last bucket are still within the wanted
window
*

Example: Bucketized Stream
N
1 of
size 2
2 of
size 4
2 of
size 8
At least 1 of
size 16. Partially
beyond window.
2 of
size 1
1001010110001011010101010101011010101010101110101010111010100010110
010
*

Error Bound: Proof
◼Why is error 50%? Let’s prove it!
◼Suppose the last bucket has size 2r
◼Then by assuming 2r-1
(i.e., half) of its 1s are
still within the window, we make an error of
at most 2r-1
◼Since there is at least one bucket of each of
the sizes less than 2r
, the true sum is at least
1 + 2 + 4 + .. + 2r-1
= 2r
-1
◼Thus, error at most 50%
106
1111111100000000111010101010110101010101011101010101110101000101100
10 N
At least 16 1s
*

Further Reducing the Error
◼Instead of maintaining 1 or 2 of each size
bucket, we allow either r-1 or r buckets (r > 2)
▪Except for the largest size buckets; we can have
any number between 1 and r of those
◼Error is at most O(1/r)
◼By picking r appropriately, we can tradeoff
between number of bits we store and the
error
*

108
Extensions
◼Can we use the same trick to answer queries
How many 1’s in the last k? where k < N?
▪A: Find earliest bucket B that at overlaps with k.
Number of 1s is the sum of sizes of more recent
buckets + ½ size of B
◼Can we handle the case where the stream is
not bits, but integers, and we want the sum
of the last k elements?
100101011000101101010101010101101010101010111010101011101010001011
0010 k
*

Extensions
◼
ci …estimated count for i-th bit
2 5 7 1 3 8 4 6 7 9 1 3 7 6 5 3 5 7 1 3 3 1 2 2 6
2 5 7 1 3 8 4 6 7 9 1 3 7 6 5 3 5 7 1 3 3 1 2 2 6 3
2 5 7 1 3 8 4 6 7 9 1 3 7 6 5 3 5 7 1 3 3 1 2 2 6 3 2
2 5 7 1 3 8 4 6 7 9 1 3 7 6 5 3 5 7 1 3 3 1 2 2 6 3 2
5
Idea: Sum in each
bucket is at most
2b
(unless bucket
has only 1 integer)
Bucket sizes:
1
2
8
16 4
*

Counting Itemsets
◼New Problem: Given a stream, which items
appear more than s times in the window?
◼Possible solution: Think of the stream of
baskets as one binary stream per item
▪1 = item present; 0 = not present
▪Use DGIM to estimate counts of 1s for all items
111
0 1 0 0 1 1 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 1 0 1 1 0 1 1 1 0 0 1 0 1 0 1 1 0 0 1 1 0 1 0
N
0
1
1
2
2
3
4
10
6
*

Extensions
◼In principle, you could count frequent
pairs or even larger sets the same way
▪One stream per itemset
◼Drawbacks:
▪Only approximate
▪Number of itemsets is way too big
112
*

Exponentially Decaying Windows
◼
*

Example: Counting Items
◼
114
*

Sliding Versus Decaying Windows
◼
1/c
. . .
*

Example: Counting Items
◼
116
*

Extension to Itemsets
◼Count (some) itemsets in an E.D.W.
▪What are currently “hot” itemsets?
▪Problem: Too many itemsets to keep counts of
all of them in memory
◼When a basket B comes in:
▪Multiply all counts by (1-c)
▪For uncounted items in B, create new count
▪Add 1 to count of any item in B and to any itemset
contained in B that is already being counted
▪Drop counts < ½
▪Initiate new counts (next slide)
*

Initiation of New Counts
◼Start a count for an itemset S B
⊆ if every
proper subset of S had a count prior to arrival
of basket B
▪Intuitively: If all subsets of S are being counted
this means they are “frequent/hot” and thus S has
a potential to be “hot”
◼Example:
▪Start counting S={i, j} iff both i and j were counted
prior to seeing B
▪Start counting S={i, j, k} iff {i, j}, {i, k}, and {j, k}
were all counted prior to seeing B
118
*

How many counts do we need?
◼Counts for single items < (2/c) (avg. number
∙
of items in a basket)
◼Counts for larger itemsets = ??
◼But we are conservative about starting
counts of large sets
▪If we counted every set we saw, one basket
of 20 items would initiate 1M counts
119
*

Summary
◼Sampling a fixed proportion of a stream
▪Sample size grows as the stream grows
◼Sampling a fixed-size sample
▪Reservoir sampling
◼Counting the number of 1s in the last N
elements
▪Exponentially increasing windows
▪Extensions:
▪Number of 1s in any last k (k < N) elements
▪Sums of integers in the last N elements
*

Real-Time Analytics Platform
(RTAP) Applications – A Case
study : Real Time Sentiment
Analysis For Stock Market
Predictions

Sentiment Analysis
◼ The study of automated techniques for extracting sentiments from
written languages.
◼ Interpretation and classification of emotions (positive, negative
and neutral) within text data using text analysis techniques.
◼ Sentiment analysis tools allow businesses to identify
customer sentiment toward products, brands or services in online
feedback.

Steps in Sentiment Analysis

Sentiment Analysis in Stock Market for Right
Prediction
◼ Sentiment analysis and stock market is a well-researched problem
◼ Maybe due to negative sentiment, the stock price goes down or if there is
any positive sentiments the stock prices increased because of this
optimistic sentiments.

Social Media Contents Based for Real-Time
Sentiment Analysis in Stock Market
Predictions
◼Although, there is no single technique to predict the
stock movement accurately, so researchers have
done lots of experiments to get better results.
◼Social media is playing a key role in sentiment
analysis on the stock market.
◼Even, over the past few years, the influence of social
media sites on everyday life has become so large
that even information on large and small incidents or
disasters is obtained through social media sites.

Social Media Impact on the Stock Market
◼ Contents are created
according to the user’s
intentions
◼ Time of creation
becomes an important
factor in social media
contents based on real-
time sentiment analysis
in stock prediction
◼ People make judgments
about the world around
them when they are
living in the society.
◼ They make positive and
negative attitudes about
people, products, places
and events - sentiments

CHhhjjuiiiiiiiiiiS18003 Unit 3 Class ppt.pptx

More Related Content

Similar to CHhhjjuiiiiiiiiiiS18003 Unit 3 Class ppt.pptx (20)

Recently uploaded (20)

CHhhjjuiiiiiiiiiiS18003 Unit 3 Class ppt.pptx

Editor's Notes