Stream Reasoning: a summary of ten years of research and a vision for the next decade

Stream Reasoning:
a summary of ten years of research and
a vision for the next decade
http://content.iospress.com/articles/data-science/ds006
Emanuele Della Valle
DEIB - Politecnico di Milano
@manudellavalle
emanuele.dellavalle@polimi.it
http://emanueledellavalle.org
Linköping University, Sweden - 5.10.2017

Me
• Assistant Professor at DEIB
Politecnico di Milano
• Expert in semantic technologies
and stream computing
• Brander of stream reasoning: an
approach to master the velocity and
variety dimension of Big Data
• https://scholar.google.com/scholar?hl=
en&q="stream+reasoning"
• 16 years of experience in research
and innovation projects
• Startupper:
• http://www.fluxedo.com
2

INTRODUCTION
SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 3

It's a streaming world …
• Off-shore oil operations
• Smart Cities
• Power turbine
• Social networks
• Generate data streams!
E. Della Valle, S. Ceri, F. van Harmelen, D. Fensel It's a Streaming World! Reasoning upon
Rapidly Changing Information. IEEE Intelligent Systems 24(6): 83-89 (2009)

… looking for reactive answers …
• When a sensor on a drill in an oil-rig indicates that it is
about to get stuck, how long can I keep drilling?
• Where am I likely going to run into a
traffic jam during my commute tonight?
• Which electricity-producing turbine has
sensor readings similar to any turbine that
subsequently had a critical failure?
• Who is driving the discussion
about the top 10 emerging topics ?
• Require continuous processing
and reactive answer

… and many more
conflicting requirements
A system able to answer those queries must be able to
• handle volume
• handle velocity
• handle variety
• cope with incompleteness
• cope with noise
• provide reactive answers
• support fine-grained access
• integrate complex domain models
• offer high-level languages

Grand challenge
• Volume + Velocity + Variety = hard deal
SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org
volume
months days hours min. sec. ms.
velocity
ZB
EB
PB
TB
GB
MB
KB
Variety
7

A good reason to embrace it!
• ++ Variety  ++ value
value
ms. sec. min. hours days months years
velocity
Variety
8

From challenges to opportunities
• Formally data streams are :
– unbounded sequences of time-varying data elements
• Less formally, in many application domains, they are:
– a “continuous” flow of information
– where recent information is more relevant as it describes the
current state of a dynamic system
• Opportunities
– Forget old enough information
– Exploit the implicit ordering (by recency) in the data
time

State-of-the-art: DSMS and CEP
• A paradigmatic change!
• Continuous queries registered over streams that
are observed trough windows
window
input streams streams of answerRegistered
Continuous
Query
Dynamic
System

DSMS and CEP vs. requirements
Requirement
DSMS
CEP
volume
velocity
variety
incompleteness
noise
reactive answers
fine-grained information access
complex domain models
high-level languages

State of the art: Semantic Web
• Given ontology O and query Q, use O to rewrite Q
as Q’ so that, for any set of ground facts A contained in multiple
databases:
– answer(Q,O,A) = answer(Q’,,A)
The answer of the query Q using the ontology O for any set of ground facts A
is equal to answer of a query Q’ without considering the ontology O
• Use mapping M to map Q’ to multiple SQL queries to the various
databases
Rewrite
O
Q
Q’
Map
SQL
M
answer
A
UiO, Norway - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 12

Solutions vs. requirements
Requirement
DSMS
CEP
Sem
Web
volume
velocity
variety
incompleteness
noise
reactive answers

Stream Reasoning
• Research question
– is it possible to make sense in real time of
multiple, heterogeneous, gigantic and inevitably noisy and
incomplete data streams in order to support the decision
processes of extremely large numbers of concurrent users?
Emanuele Della Valle: On Stream Reasoning. PhD thesis, Vrije Universiteit Amsterdam, 2015.
Available online at http://dare.ubvu.vu.nl/handle/1871/53293 .

Is this feasible?
• Proposed approach: cascading Stream Reasoning
Complexity
Raw Stream Processing
Semantic Streams
DL-Lite
DLAbstraction
Selection
Interpretation
Reasoning
Querying
Re-writing
Change Frequency
PTIME
NEXPTIME
104 Hz
1 Hz
Complexity vs. Dynamics
AC0
H. Stuckenschmidt, S. Ceri, E. Della Valle, F. van Harmelen: Towards Expressive Stream Reasoning. Proceedings of
the Dagstuhl Seminar on Semantic Aspects of Sensor Networks, 2010.

A model to describe stream reasoning
D. Dell’Aglio, On Unified Stream Reasoning, PhD thesis, Politecnico di Milano, 2016.
Stream Processing (DSMS)
Event Processing (CEP)Window merge
Window operator
Streams
Graph-level entailment
Window-level entailment
Stream-level entailment
Application
StreamReasoning

GRAPH LEVEL
UNDER SIMPLE ENTAILMENT REGIME

State-of-the-art: RDF model
• RDF: Resource Description Framework
– It allows to make statements about resources
in the form of subject-predicate-object expressions
• In RDF terminology triples
• E.g.
@TimBernersLee posts "This is for everyone"
– A collection of RDF statements represents a labelled,
directed graph
• In RDF terminology a graph
• E.g., the tweet above by Tim Berners Lee is connected to
– Thousands of twitter user profiles via retweets
– Thousands of twitter user profiles via favorite
– …
subject predicate object
UiO, Norway - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 18

RDF stream Models
• RDF Stream (the C-SPARQL way)
– Unbound sequence of time-varying triples
– each represented by a pair made of an RDF triple and its
timestamp
– Timestamp are non-decreasing (allowing for simultaneity)
…
TimBernersLee posts "This is for everyone", 10:16.55 PM 22 Aug 2012
@Alice posts "RT: This is for everyone", 10:17.03 PM 22 Aug 2012
…
D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus: Querying RDF streams with
C-SPARQL. SIGMOD Record 39(1): 20-26 (2010)
subject predicate object timestamp

RDF stream Models
• RDF Stream (the Streaming Linked Data way)
– Unbound sequence of time-varying graphs
– each represented by a pair made of an RDF graph and its
timestamp (all triples in a graph are simultaneous)
– Timestamps (if present) are monotonically increasing
– Graphs act as a form of punctuation
A. Mauri, J-P Calbimonte, D.Dell'Aglio, M.Balduini, M.Brambilla, E.Della Valle, K. Aberer:
TripleWave: Spreading RDF Streams on the Web. ISWC (2) 2016: 140-149
20
D.F. Barbieri, E. Della Valle: A Proposal for Publishing Data Streams as Linked Data –
A Position Paper. LDOW (2010)

Continuous-SPARQL

Continuous-SPARQL
Who are the opinion makers? i.e., the users who are
likely to influence the behavior their followers
REGISTER STREAM OpinionMakers COMPUTED EVERY 5m AS
CONSTRUCT { ?opinionMaker sd:about ?resource }
FROM STREAM <http://…> [RANGE 30m STEP 5m]
WHERE {
?opinionMaker ?opinion ?res .
?follower sioc:follows ?opinionMaker.
?follower ?opinion ?res.
FILTER (cs:timestamp(?follower ?opinion ?res) >
cs:timestamp(?opinionMaker ?opinion ?res) )
}
HAVING ( COUNT(DISTINCT ?follower) > 3 )
SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 22D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus: Querying RDF streams with

Continuous-SPARQL
Who are the opinion makers? i.e., the users who are
likely to influence the behavior their followers
REGISTER STREAM OpinionMakers COMPUTED EVERY 5m AS
CONSTRUCT { ?opinionMaker sd:about ?resource }
FROM STREAM <http://…> [RANGE 30m STEP 5m]
WHERE {
?opinionMaker ?opinion ?res .
?follower sioc:follows ?opinionMaker.
?follower ?opinion ?res.
FILTER (cs:timestamp(?follower ?opinion ?res) >
cs:timestamp(?opinionMaker ?opinion ?res) )
}
HAVING ( COUNT(DISTINCT ?follower) > 3 )
Query registration
(for continuous execution)
FROM STREAM clause
WINDOW
RDF Stream added as
new ouput format
Builtin to access
timestamps
SR 2015, Austria - 3.11.2015 @manudellavalle - http://emanueledellavalle.org 23D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus: Querying RDF streams with

More than modeling,
it's for reactive answers!
• C-SPARQL engine time window-based selection outperforms
SPARQL filter-based selection (Jena-ARQ)
D. Barbieri, D. Braga, S. Ceri, E. Della Valle, Y. Huang, V. Tresp, A.Rettinger, H. Wermser:
Deductive and Inductive Stream Reasoning for Semantic Social Media Analytics
IEEE Intelligent Systems, 30 Aug. 2010.
Our In-memory
RDF stream
processing
engine

Alternatives to C-SPARQL
• SPARQLStream
– What: window in the past, focus on RDF to Stream operators
– Ref: Calbimonte, J.-P., Corcho, O., & Gray, A. J. G. Enabling ontology-based
access to streaming data sources. In ISWC, 2010, pages 96–111.
• CQELS
– What: STREAM clause, focus on new answer
– Ref: Le-Phuoc, D., Dao-Tran, M., Xavier Parreira, J., & Hauswirth, M.
A native and adaptive approach for unified processing of linked streams
and linked data. In ISWC 2011, pages 370–388.
• EP-SPARQL
– What: focus on event specific operators
– Ref: Anicic, D., Fodor, P., Rudolph, S., & Stojanovic, N. EP-SPARQL: a unified
language for event processing and stream reasoning. In WWW 2011, pages
635–644.
• TEF-SPARQL
– What: adds "facts" as first class elements
– Ref: https://www.merlin.uzh.ch/publication/show/8467

Work in progress
• RSP-QL
– Syntax
• https://github.com/streamreasoning/RSP-QL/blob/master/RSP-
QL%20Sample%20Queries.md
– Proposed semantics
• D.Dell'Aglio, E.Della Valle, J.-P.Calbimonte, Ó. Corcho: RSP-QL
Semantics: A Unifying Query Model to Explain Heterogeneity of
RDF Stream Processing Systems. Int. J. Semantic Web Inf. Syst.
10(4): 17-44 (2014)
– Semantics (work in progress)
• https://github.com/streamreasoning/RSP-
QL/blob/master/Semantics.md
– Quick ref.
• D. Dell'Aglio, J.-P. Calbimonte, E. Della Valle, Ó. Corcho: Towards a
Unified Language for RDF Stream Query Processing. ESWC
(Satellite Events) 2015: 353-363

GRAPH LEVEL
UNDER OTHER ENTAILMENT REGIME

continuous deductive reasoning
• DL Ontology Stream ST
– A ontology stream with respect to a static Tbox T is a
sequence of Abox axioms ST(i)
• A Windowed Ontology Stream ST(o,c]
– A windowed ontology stream with respect to a static
Tbox T is the union of the Abox axioms ST(i) where
o<i≤c
• Reasoning on a Windowed Ontology Stream
ST(o,c] is as reasoning on a static DL KB
Emanuele Della Valle, Stefano Ceri, Davide Francesco Barbieri, Daniele Braga, Alessandro
Campi: A First Step Towards Stream Reasoning. FIS 2008: 72-81

discusses discusses discusses
discusses discusses
discusses
discusses
Example of
continuous deductive reasoning
What impact has been my micropost p1 creating in the last hour?
Let’s count the number of microposts that discuss it …
REGISTER STREAM ImpactMeter AS
SELECT (count(?p) AS ?impact)
FROM STREAM <http://…/fb> [RANGE 60m STEP 10m]
WHERE {
:Alice posts [ sr:discusses ?p ]
}
p1 p3 p5 p8
p2 p4 p7
p6
Transitive
property
Alice posts p1 .

Not so naïve approach to
stream reasoning
• The problem is that materialization (the result of data-driven
processing) are very difficult to decrement efficiently.
– State-of-the-art: DRed algorithm
• Over delete
• Re-derive
• Insert
Reasoner
Inferred
data
ontology
window
insertions
deletions
Incremental !!!
SPARQL

Is DRed needed?
• DRed works with random insertions and deletions
• In a streaming setting, when a triple enters the window,
given the size of the window, the reasoner knows already
when it will be deleted!
• E.g.,
– if the window is 40 minutes
long, and,
– it is 10:00, the triple(s)
entering now
– will exit on 10:40.
• Conclusion
– deletions are predictable
Time
Enter
window
Exit
window
Explicitly in
window
Infer
win
10:00 AßB
10:10 BßC
10:20 AßE
10:30 EßC
10:40 AßB
10:50 BßC
11:00 AßE
A B
A B C A
A B C
E
A
A B C
E
A
A C
E
A
A B C
E
A
C
E

IMaRS algorithm
• Idea:
– add an expiration time to each triple and
– use an hash table to index triples by their expiration time
• The algorithm
1. deletes expired triples
2. Adds the new derivations that are consequences of
insertions annotating each inferred triple with an
expiration time (the min of those of the triple it is
derived from), and
3. when multiple derivations occur, for each multiple
derivation, it keeps the max expiration time.

IMaRS algorithm
• Incremental Reasoning on RDF streams (IMaRS): new reasoning
algorithm optimized for reactive query answering
D.F. Barbieri, D. Braga, S.Ceri, E. Della Valle, M. Grossniklaus: Incremental Reasoning on
Streams and Rich Background Knowledge. ESWC (1) 2010: 1-15
D. Dell'Aglio, E. Della Valle: Incremental Reasoning on RDF Streams. In A.Harth, K.Hose,
R.Schenkel (Eds.) Linked Data Management, CRC Press 2014, ISBN 9781466582408
§ Re-materialize after each window slide
§ Use DRed
§ IMaRS
% of deletions w.r.t. the content of the window

IMaRS algorithm
• comparison of the average time needed to answer
a C-SPARQL query, when 2% of the content exits the window each
time it slides, using
– A backward reasoner on the window content
– DRed + standard SPARQL on the materialization
– IMaRS + standard SPARQL on the materialization

Some alternatives to IMaRS
• TROWL
– How: DRed in the context of approximate reasoning
– Ref: Y. Ren, J. Z. Pan. Optimising ontology stream reasoning
with truth maintenance system. In CIKM (2011)
• Sparkwave
– How: extended RETE algorithm for windows and RDFS
– Ref: Sparkwave: Continuous Schema-Enhanced Pattern
Matching over RDF Data Streams. Komazec S, Cerri D. DEBS
2012
• The Backward/Forward Algorithm
– How: optimizing DRed
– B. Motik, Y. Nenov, R.E.F. Piro, I. Horrocks: Incremental Update
of Datalog Materialisation: the Backward/Forward Algorithm.
AAAI 2015: 1560-1568

WINDOW LEVEL
ENTAILMENT

window level entailment
• Graph-level entailment considers data item contents, but it
does not use the temporal annotations
• Window-level entailment applies the inference process on
the non-merged stream items.
• E.g.,
– A door cannot be open and close at the same time
– A window contains: door A is open @1, door A is close @2
– At graph-level the reasoner tells that there is an inconsistency in
the window because it ignore the parts in italics
– At window-level the reasoner does not
• Best approaches I saw so far
– ÖL Özçep, R Möller. Ontology Based Data Access on Temporal and
Streaming Data. Reasoning Web, 2014
– D. Dell'Aglio et al. :A Query Model to Capture Event Pattern Matching
in RDF Stream Processing Query Languages EKAW 2016: 145-162

STREAM LEVEL
ENTAILMENT

• Window-level entailment only considers a recent portion of
the stream
• Stream-level entailment aims at considering the entire
stream
• This is not just a theoretical dream, CEP does so
– E.g., rise C for every A that follows a B without a C in the middle
• Best approaches I saw so far
– ETALIS
• Anicic, D., Rudolph, S., Fodor, P., & Stojanovic, N. Stream reasoning and
complex event processing in ETALIS Semantic Web,3(4), 2012, 397–407.
– LARS
• H. Beck, M. Dao-Tran, T. Eiter, M. Fink: LARS: A Logic-Based Framework
for Analyzing Reasoning over Streams. AAAI 2015: 1431-1438H.

OPEN PROBLEMS AND CHALLENGES
FOR THE NEXT YEARS

Requirements vs. state-of-the-art
Requirement SotA@2017Q2
volume
velocity
variety
incompleteness
noise
reactive answers

Towards queries that capture user needs
• Capturing a wider set of tasks in queries
– Descriptive analytics
• Which electricity-producing turbine has sensor readings similar
(i.e., Pearson correlated by at least 0.75) to any turbine that
subsequently had a critical failure in the past year?
– Predictive analytics (Machine Learning)
• Where am I likely going to run into a traffic jam during my
commute tonight and how long will it take, given current weather
and traffic conditions?
– Geo-spatial reasoning
– Including preferences
– Exploiting the graph nature of the data

• Tightening CEP and ontological reasoning
– ETALIS/EP-SPARQL is the corner stone with high-
expressivity but very low efficiency
– Work in progress:
CEP
DL
RDF STREAM PROCESSING
RAW STREAM PROCESSING
Abstraction
Selection
Annotation
Reasoning
Querying
Rewriting
R. Tommasini, P. Bonte, E. Della Valle, E. Mannens, F. De Turck, F. Ongenae: Towards
Ontology-Based Event Processing. OWLED 2016: 115-127

• Forgetting knowledge
– Is IMaRS semantics what users want?
– Consumption ≠ expiration
• Consumed facts are not useful for processing
• Expired facts are not true anymore
– Should we introduce
• a notion of semantic importance?
• a notion of time-annotated facts?
• …

Towards sophisticated Stream Reasoning
• Extending the range of logical formalisms
– Pioneers:
• Metric Temporal logic
– F. Heintz and P. Doherty, DyKnow: An approach to middleware for
knowledge processing, Journal of Intelligent and 11 Fuzzy Systems 15(1)
(2004), 3–13.
• Answer Set Programming (ASP)
– A. Mileo, A. Abdelrahman, S. Policarpio and M. Hauswirth, StreamRule:
A nonmonotonic stream reasoning system for the semantic web, in:
RR, LNCS, Vol. 7994, Springer, 2013, pp. 247–252.
– More alternatives:
• action logics
• step logics
• active logics
• event calculus

• Integrating other types of reasoning/processing
– My contribution
D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, Y. Huang, V. Tresp, A. Rettinger, H. Wermser:
Deductive and Inductive Stream Reasoning for Semantic Social Media Analytics.
IEEE Intelligent Systems 25(6): 32-41 (2010)

• Integrating other types of reasoning/processing
– Other options
• More on inductive stream reasoning
• Probabilistic reasoning
• Planning
• Natural Language Processing
• Sentiment analysis
• …

• Towards semantic streams
– Describe data stream to make them discoverable,
composable, etc.
– My work in progress
• Y. A. Sedira, R.Tommasini, E. Della Valle: Towards VoIS: A
Vocabulary of Interlinked Streams. DeSemWeb@ISWC 2017
– There is much more to describe, e.g.
• Stream of state (the door is close) vs.
Stream of actions (the door was closed)

• Towards scalable stream reasoners
– Rewriting
• e.g., MorphStream
– How: rewriting in DSMS languages (one at a time)
– Ref: Calbimonte, J.-P., Corcho, O., & Gray, A. J. G. Enabling ontology-
based access to streaming data sources. In ISWC, 2010, pages 96–111.
– Using parallelization and distribution
• e.g., DynamiTE
– How: Truth maintenance for DF (a fragment of RDFS)
– J. Urbani, A. Margara, C. J. H. Jacobs, F. van Harmelen, H.E. Bal:
DynamiTE: Parallel Materialization of Dynamic RDF Data. ISWC (1)
2013: 657-672
– Using approximation
• e.g., TR-OWL
– How: Truth maintenance for EL++ with syntactic approximations
– Ref: Y. Ren, J. Z. Pan. Optimising ontology stream reasoning with truth
maintenance system. In CIKM (2011)

Towards robustness to imperfect data
• Overcoming types of heterogeneity that do not
exist in the static reasoning settings
– It is not just extending OBDA to continuous queries
– There are more sources of heterogeneity than in the
static database settings
• Different execution semantics
• No possibility for a priori homogenous
– Synchronisation*
– discretization
• …
* Pioneered in F.Heintz, DyKnow: A stream Based Knowledge Processing Middleware
Framework, PhD thesis,Linköping University, Sweden, 2009.

Towards robustness to imperfect data
• Copying with noise
– Data as signals instead of data as truths!
– Easy to state, hard to achieve
– Still of paramount importance!
– Notable approaches I run into
• Matthias Nickles, Alessandra Mileo: Web Stream Reasoning Using
Probabilistic Answer Set Programming. RR 2014: 197-205
• Anastasios Skarlatidis, Georgios Paliouras, Alexander Artikis, George A.
Vouros: Probabilistic Event Calculus for Event Recognition. ACM Trans.
Comput. Log. 16(2): 11:1-11:37 (2015)
• Anni-Yasmin Turhan, Erik Zenker: Towards Temporal Fuzzy Query
Answering on Stream-based Data. HiDeSt@KI 2015: 56-69

Conclusions
Stream Processing
Event ProcessingWindow merge
Window operator
Streams
Graph-level entailment
Window-level entailment
Application
StreamReasoning
Very well only some initial studies still very open
New abstractions to capture user needs
Application

Stream Reasoning:
a summary of ten years of research and a
vision for the next decade
http://content.iospress.com/articles/data-science/ds006
Q/A
Emanuele Della Valle
DEIB - Politecnico di Milano
@manudellavalle
emanuele.dellavalle@polimi.it
http://emanueledellavalle.org
Linköping University, Sweden - 5.10.2017

Stream Reasoning: a summary of ten years of research and a vision for the next decade

More Related Content

Similar to Stream Reasoning: a summary of ten years of research and a vision for the next decade (20)

More from Emanuele Della Valle (20)

Recently uploaded (20)

Stream Reasoning: a summary of ten years of research and a vision for the next decade

Editor's Notes