Using Complex Event Processing for Modeling Semantic Requests in Real-Time Social Media Monitoring

INFORMATIK
Using Complex Event Processing for Modeling

FZI FORSCHUNGSZENTRUM
Semantic Requests in Real-Time Social Media
Monitoring
Dominik Riemer, Nenad Stojanovic, Ljiljana Stojanovic
FZI Research Center for Information Technologies, Karlsruhe, Germany

RAMSS 2012, 04 June 2012
Dublin, Ireland

Agenda

Introduction

Use Cases

System Architecture

Semantic Requests

Tools

Discussion

Why is Social Media Monitoring important?

 Social Media Monitoring
 Continuous process of monitoring a specific subject of matter in social
media
 Continuous, not necessarily real-time

 Usage of Social Media is emerging
 Wider range of age groups
 New business opportunities for companies

 State of the Art in SMM
 Today: mainly image reputation, early identification of harmful
messages
 Predictions (e.g., earthquake identification)
 Aggregations over crawled data in order to find overall scores for a
brand

Search in Social Media

 Historical Search:
 Search in past social media data
 Social network analysis
 Applications:
 market research

 Real-Time Search:
 Search for patterns when data arrives
 Applications:
 proactive marketing
 real-time spinning

 Proactive Search:
 Forecast future developments
 E.g., movie box office revenues

Complex Event Processing

 Event processing is a form of computing that performs operations on events

 CEP is an enabling technology that supports on the fly, (business-) real-
time processing of huge event streams

 CEP is about a timely (or in head of time) recognition of the situations of
interest and corresponding reaction
 A complex event pattern describes a situation of interests

Proactive Marketing

 As-Is:
 Buying decisions are often based on recommendations through social
media channels
 Users share experiences and problems with products in social networks

 Business threads:
 Negative feedback spreads very fast in social networks

 Business opportunities:
 New customers
 More satisfied customers

 Proactive Marketing
 Establishing relations to consumers and customers based on detected
situations
 Real-time advertising, real-time problem solving
 Example: automatic helpdesk ticket generation

Real-Time Spinning

 As-Is:
 TV debates before elections are an important event for political
candidates
 Direct feedback of users from social networks

 Business threads
 Early opinions of voters are the basis for future developments

 Business opportunities
 Direct feedback to voters can influence their opinion

 Real-Time spinning
 Additional links to information sources
 Early identification of opinions, increased awareness for specific topics

Objectives
Some User Requirements Approach

 Real-Time Processing
 Enrichment of social data (Distributed) CEP
 High throughput
 Advanced filtering and pattern detection

 Advanced Pattern Definitions
 Usage of domain knowledge Semantic Requests
 Pattern definition hides technical details

 Triggering of Actions & Visualizations
Visual Pattern
 e.g., Notifications Modeling

Challenges of Stream Search
Web search Stream search
Order of occurrence Data->Query Query->Data
Scope Information Needs Notification Needs

Data Collection Crawling Real-Time Sources

Optimization Priority Data Query

Goal Fast Processing of Queries Fast Processing of Data

Challenge:

How can we simplify the user-centric
process of pattern definition?

Conceptual Model

 Semantic Requests
 event pattern extensions
 make use of background knowledge
 should simplify definition of situations of interest
 design-time translation to non-semantic queries

 Relevancy
 Advanced filtering based on background knowledge
 Users are often interested in generalized concepts (e.g., automotive
manufactures)
 Importance
 Requires extensions with characteristic terms of a domain
 What is when important?
 Unusuality
 Computation of pattern fulfilment degree (white-box event processing)
 Makes use of data mining techniques

Modeling Semantic Requests

 Event pattern definition (Esper)
select * from UpdateEvent where content like „%apple%“ or content
like „%ipad%“ or content like „%iphone%“ or content like „%ios%“

 Semantic request-enhanced pattern definition:
select * from UpdateEvent where sr:anythingRelated(content, like,
Company, [hasName:Apple]

 Semantic requests:
 sr:anything: Find any instances of a given concept
 sr:anythingRelated: Find any related instances of a given concept
 sr:all: Find all instances of a given concept
 sr:allRelated: Find all related instances of a given concept
 sr:geo: Find all related locations that are related to a given concept

Semantic Requests: Pros and Cons

 Pros:
 No extension of the underlying engine required
 Background knowledge separated from pattern definition
 Easier to maintain, knowledge is defined at a single point
 Faster definition of similar, but frequently changing patterns

 Cons:
 Knowledge base has to be modeled and maintained
 Patterns have to be redeployed after changes in the knowledge base
occured
 Performance loss due to long pattern specifications

Translation Process

 Definition of background knowledge (Ontology)
 Extraction of semantic requests from the original pattern
 Ontology is loaded
 Generation of SPARQL queries based on semantic requests
 Keywords are added to the resulting EPL
 Pattern is deployed to the event processing engine.

Discussion & Future Work

 Semantic Request
select * from TwitterEvent where content like „%apple%“ or content
like „%ipad%“ or content like „%iphone%“ or content like „%ios%“

 Problem
 Patterns probably return many unimportant messages
 Example: links to iTunes, fruit, etc.

 Approach:
 Enhancing semantic requests with importance
 e.g., Apple + technology-related topics
 e.g., Apple + technical or finance-related words

Related Work

 Social Media Monitoring
 Twarql (Mendes et. al. 2010):
 Publishing Twitter Streams as Linked Data
 allows definition of SPARQL queries
 Domain-specific Topic Detection (Schirru et. al., 2010)
 weighting of terms and concepts
 uses data mining methods, not real-time computations

 Semantic Requests / Semantic CEP
 Event Query Pre-Processing (Teymourian 2011)
 rewrite queries based upon background knowledge
 not grounded in social media domain

Using Complex Event Processing for Modeling Semantic Requests in Real-Time Social Media Monitoring

More Related Content

Similar to Using Complex Event Processing for Modeling Semantic Requests in Real-Time Social Media Monitoring (20)

Recently uploaded (20)

Using Complex Event Processing for Modeling Semantic Requests in Real-Time Social Media Monitoring