SlideShare a Scribd company logo
CrowdSem 2013
1st International Workshop on
“Crowdsourcing the Semantic Web”
Sydney, Australia
21 October 2013
Follow us on Twitter: @crowdsem 
Spread the news from today: #crowdsem2013
Monday, October 21, 13
Who are we
Maribel Acosta, Institute AIFB, Karlsruhe Institute of
Technology, Germany
Lora Aroyo, VU University Amsterdam, the Netherlands
Abraham Bernstein, University of Zurich, Switzerland
Jens Lehmann, University of Leipzig, Germany
Natasha Noy, Stanford University, US
Elena Simperl, University of Southampton, United Kingdom
Monday, October 21, 13
Workshop Program
“Dr. Detective”: combining gamification techniques and crowdsourcing to create a gold standard in medical
text, Anca Dumitrache, Lora Aroyo, Chris Welty, Robert-Jan Sips and Anthony Levas , (Discussant: Natasha
Noy)
11:00 – 11:30 Crowdsourced Semantics with Semantic Tagging: “Don’t just tag it, LexiTag it!”, Csaba
Veres, (Discussant: Lora Aroyo)
11: 30 – 12:00 SLUA: Towards Semantic Linking of Users with Actions in Crowdsourcing, Umair Ul Hassan,
Sean O’Riain and Edward Curry, (Discussant: Maribel Acosta)
12:00 – 12:30 Experiment paper:  Crowdsourced Entity Markup, Lili Jiang, Yafang Wang, Johannes Hoffart
and Gerhard Weikum, (Discussant: Abraham Bernstein)
12:30 – 12:45 Position paper: A Role for Provenance in Social Computation, Milan Markovic, Peter Edwards
and David Corsar, (Discussant: Lora Aroyo)

Monday, October 21, 13
Workshop Program
1:45 – 2:15 Experiment paper: Frame Semantics Annotation Made Easy with DBpedia, Marco Fossati, Sara
Tonelli and Claudio Giuliano, (Discussant: Maribel Acosta)
2:15 – 2:45 Experiment paper: Developing Crowdsourced Ontology Engineering Tasks: An iterative
process, Jonathan Mortensen, Mark Musen and Natasha F. Noy, (Discussant: Elena Simperl)
2:45 – 3:15 Content and Behaviour Based Metrics for Crowd Truth, Guillermo Soberon, Lora Aroyo, Chris
Welty, Oana Inel, Manfred Overmeen and Hui Lin, (Discussant: Abraham Bernstein)
3:15 – 3:30 Position paper: Information Reputation, Peter Davis and Salman Haq, (Discussant: Elena
Simperl)

Papers accessible at: http://ceur-ws.org/Vol-1030/

Monday, October 21, 13
Invited Talk
4:00 – 5:00 Big Data meets Big Social: Social Machines and the Semantic Web
David De Roure, University of Oxford
This talk will discuss this opportunity for Semantic Web approaches to underpin the design
and construction of future Social Machines against a backdrop of increasing scale and
increasing automation.
David De Roure is Professor of e-Research at University of Oxford and
Director of the Oxford e-Research Centre. Throughout his career he has
worked closely in and across multiple disciplines to foster innovation in
digital methods and scholarship. His current projects range from Web
Science (Social Machines and Web Observatories) to Digital
Humanities (computational musicology). He has a coordinating role in
Digital Humanities at Oxford and is a UK strategic adviser for data
resources in social media.

Monday, October 21, 13
Crowdsourcing Exercise
Task: Gather structured information about the three keynotes at ISWC2013, e.g.
find pictures
find research interests
find publications
etc.
Form groups of experienced and novices in practical crowdsourcing experience
Choose whether you would want to run the task on CrowdFlower or AMT
Create a simple design of the task (20 mins)
One group member presents the design to the whole workshop (5 mins)
Selected tasks will be performed on either AMT or CrowdFlower and we will discuss the
results at the wrap-up session

Monday, October 21, 13
Initial Designs ...

Monday, October 21, 13
Initial Designs ...

Monday, October 21, 13
Initial Designs ...

Monday, October 21, 13
Stay tuned on Twitter
for the results

Follow us on Twitter: @crowdsem 
Spread the news from today: #crowdsem2013
Monday, October 21, 13

More Related Content

What's hot (19)

PPTX
Seeding A Culture of Collaboration
Museums Computer Group
 
ODP
Glyn Moody: from open source to open research
glynmoody
 
PPT
11 7 2007 EVA
Stuart Dunn
 
PPTX
Executable Music Documents
David De Roure
 
PPTX
Social Machines Paradigm
David De Roure
 
PPT
Technology enhanced learning in higher education
jimson99
 
PPTX
Kathryn eccles digital humanities
oiisdp
 
PPTX
Valedictory
Stephen Town
 
PPT
Jankowski, Vks E Research Slidecast, 26 June2008
Nick Jankowski
 
PPTX
The Future of Reading:
David Lisa
 
PPTX
Taking IT for Granted
David De Roure
 
PPTX
Of Libraries and Labs: Effecting User-Driven Innovation
Alex Humphreys
 
PDF
ARCC National Perspective Panel: XSEDE (Towns)
John Towns
 
PPTX
Cae brussels 28-1 presentation
Nicholas Anastasopoulos, PhD
 
PPTX
The cloud and the commons
Tom Ipri
 
PDF
The future of online learning in higher education, MOOCs and more
Welten Institute-Open Universiteit Nederland
 
PPT
Presentation
laurien
 
DOC
B communities of practice reference material - draft
ocasiconference
 
PDF
Digital Transformations: keynote talk to Listening Experience Database Sympos...
Andrew Prescott
 
Seeding A Culture of Collaboration
Museums Computer Group
 
Glyn Moody: from open source to open research
glynmoody
 
11 7 2007 EVA
Stuart Dunn
 
Executable Music Documents
David De Roure
 
Social Machines Paradigm
David De Roure
 
Technology enhanced learning in higher education
jimson99
 
Kathryn eccles digital humanities
oiisdp
 
Valedictory
Stephen Town
 
Jankowski, Vks E Research Slidecast, 26 June2008
Nick Jankowski
 
The Future of Reading:
David Lisa
 
Taking IT for Granted
David De Roure
 
Of Libraries and Labs: Effecting User-Driven Innovation
Alex Humphreys
 
ARCC National Perspective Panel: XSEDE (Towns)
John Towns
 
Cae brussels 28-1 presentation
Nicholas Anastasopoulos, PhD
 
The cloud and the commons
Tom Ipri
 
The future of online learning in higher education, MOOCs and more
Welten Institute-Open Universiteit Nederland
 
Presentation
laurien
 
B communities of practice reference material - draft
ocasiconference
 
Digital Transformations: keynote talk to Listening Experience Database Sympos...
Andrew Prescott
 

Similar to CrowdSem 2013 Workshop @ISWC2013 (20)

PPTX
WOW2013 Introduction
David De Roure
 
PDF
Tech talk about iswc2013
Rodrigo Senra
 
PDF
2014_WWW_BTOR
Dongpo Deng
 
PDF
Social Data on the Web 2008
Alexandre Passant
 
PPT
Social Information Processing (Tin180 Com)
Tin180 VietNam
 
PDF
Human computation and the Semantic Web (examples)
Elena Simperl
 
PDF
The Semantic Web Iswc 2008 7th International Semantic Web Conference Iswc 200...
rapeapiwnik
 
PDF
Lecture 3: Vocabularies & Data Formats on the Social Web (2014)
Lora Aroyo
 
PDF
The Semantic Web ISWC 2014 13th International Semantic Web Conference Riva de...
mangeirombs52
 
PPT
Introduction to the Social Data on the Web Workshop
John Breslin
 
PPTX
One does not simply crowdsource the Semantic Web
Elena Simperl
 
PDF
Wimmics Research Team 2015 Activity Report
Fabien Gandon
 
PPT
Crick Presentation. Valencia 28_feb_2010
Luis Torres-Yepez
 
PPTX
Interlinking semantics, web2.0, and the real-world
The Open University
 
PDF
Tutorial: Social Semantic Web and Crowdsourcing - E. Simperl - ESWC SS 2014
eswcsummerschool
 
PDF
ESWC 2014 Tutorial part 2
Miriam Fernandez
 
PPTX
ESWC SS 2012 - Monday Introduction John Domingue: The 2nd ESWC Summer School
eswcsummerschool
 
PPTX
Social semantic web
Vlad Posea
 
PDF
Towards a classification framework for social machines
SOCIAM Project
 
PDF
Towards a classification framework for social machines
Elena Simperl
 
WOW2013 Introduction
David De Roure
 
Tech talk about iswc2013
Rodrigo Senra
 
2014_WWW_BTOR
Dongpo Deng
 
Social Data on the Web 2008
Alexandre Passant
 
Social Information Processing (Tin180 Com)
Tin180 VietNam
 
Human computation and the Semantic Web (examples)
Elena Simperl
 
The Semantic Web Iswc 2008 7th International Semantic Web Conference Iswc 200...
rapeapiwnik
 
Lecture 3: Vocabularies & Data Formats on the Social Web (2014)
Lora Aroyo
 
The Semantic Web ISWC 2014 13th International Semantic Web Conference Riva de...
mangeirombs52
 
Introduction to the Social Data on the Web Workshop
John Breslin
 
One does not simply crowdsource the Semantic Web
Elena Simperl
 
Wimmics Research Team 2015 Activity Report
Fabien Gandon
 
Crick Presentation. Valencia 28_feb_2010
Luis Torres-Yepez
 
Interlinking semantics, web2.0, and the real-world
The Open University
 
Tutorial: Social Semantic Web and Crowdsourcing - E. Simperl - ESWC SS 2014
eswcsummerschool
 
ESWC 2014 Tutorial part 2
Miriam Fernandez
 
ESWC SS 2012 - Monday Introduction John Domingue: The 2nd ESWC Summer School
eswcsummerschool
 
Social semantic web
Vlad Posea
 
Towards a classification framework for social machines
SOCIAM Project
 
Towards a classification framework for social machines
Elena Simperl
 
Ad

More from Lora Aroyo (20)

PDF
NeurIPS2023 Keynote: The Many Faces of Responsible AI.pdf
Lora Aroyo
 
PDF
CATS4ML Data Challenge: Crowdsourcing Adverse Test Sets for Machine Learning
Lora Aroyo
 
PDF
Harnessing Human Semantics at Scale (updated)
Lora Aroyo
 
PDF
Data excellence: Better data for better AI
Lora Aroyo
 
PDF
CHIP Demonstrator presentation @ CATCH Symposium
Lora Aroyo
 
PDF
Semantic Web Challenge: CHIP Demonstrator
Lora Aroyo
 
PDF
The Rijksmuseum Collection as Linked Data
Lora Aroyo
 
PDF
Keynote at International Conference of Art Libraries 2018 @Rijksmuseum
Lora Aroyo
 
PDF
FAIRview: Responsible Video Summarization @NYCML'18
Lora Aroyo
 
PDF
Understanding bias in video news & news filtering algorithms
Lora Aroyo
 
PDF
StorySourcing: Telling Stories with Humans & Machines
Lora Aroyo
 
PDF
Data Science with Humans in the Loop
Lora Aroyo
 
PPTX
Digital Humanities Benelux 2017: Keynote Lora Aroyo
Lora Aroyo
 
PDF
DH Benelux 2017 Panel: A Pragmatic Approach to Understanding and Utilising Ev...
Lora Aroyo
 
PDF
Crowdsourcing ambiguity aware ground truth - collective intelligence 2017
Lora Aroyo
 
PDF
My ESWC 2017 keynote: Disrupting the Semantic Comfort Zone
Lora Aroyo
 
PDF
Data Science with Human in the Loop @Faculty of Science #Leiden University
Lora Aroyo
 
PDF
SXSW2017 @NewDutchMedia Talk: Exploration is the New Search
Lora Aroyo
 
PDF
Europeana GA 2016: Harnessing Crowds, Niches & Professionals in the Digital Age
Lora Aroyo
 
PDF
"Video Killed the Radio Star": From MTV to Snapchat
Lora Aroyo
 
NeurIPS2023 Keynote: The Many Faces of Responsible AI.pdf
Lora Aroyo
 
CATS4ML Data Challenge: Crowdsourcing Adverse Test Sets for Machine Learning
Lora Aroyo
 
Harnessing Human Semantics at Scale (updated)
Lora Aroyo
 
Data excellence: Better data for better AI
Lora Aroyo
 
CHIP Demonstrator presentation @ CATCH Symposium
Lora Aroyo
 
Semantic Web Challenge: CHIP Demonstrator
Lora Aroyo
 
The Rijksmuseum Collection as Linked Data
Lora Aroyo
 
Keynote at International Conference of Art Libraries 2018 @Rijksmuseum
Lora Aroyo
 
FAIRview: Responsible Video Summarization @NYCML'18
Lora Aroyo
 
Understanding bias in video news & news filtering algorithms
Lora Aroyo
 
StorySourcing: Telling Stories with Humans & Machines
Lora Aroyo
 
Data Science with Humans in the Loop
Lora Aroyo
 
Digital Humanities Benelux 2017: Keynote Lora Aroyo
Lora Aroyo
 
DH Benelux 2017 Panel: A Pragmatic Approach to Understanding and Utilising Ev...
Lora Aroyo
 
Crowdsourcing ambiguity aware ground truth - collective intelligence 2017
Lora Aroyo
 
My ESWC 2017 keynote: Disrupting the Semantic Comfort Zone
Lora Aroyo
 
Data Science with Human in the Loop @Faculty of Science #Leiden University
Lora Aroyo
 
SXSW2017 @NewDutchMedia Talk: Exploration is the New Search
Lora Aroyo
 
Europeana GA 2016: Harnessing Crowds, Niches & Professionals in the Digital Age
Lora Aroyo
 
"Video Killed the Radio Star": From MTV to Snapchat
Lora Aroyo
 
Ad

Recently uploaded (20)

PDF
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
PDF
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
PDF
NewMind AI - Journal 100 Insights After The 100th Issue
NewMind AI
 
PDF
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
PDF
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
PDF
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
PDF
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
PDF
Blockchain Transactions Explained For Everyone
CIFDAQ
 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
PDF
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
PDF
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
PDF
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
PPTX
Q2 FY26 Tableau User Group Leader Quarterly Call
lward7
 
PDF
From Code to Challenge: Crafting Skill-Based Games That Engage and Reward
aiyshauae
 
PDF
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
PPTX
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
PDF
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
NewMind AI - Journal 100 Insights After The 100th Issue
NewMind AI
 
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
Blockchain Transactions Explained For Everyone
CIFDAQ
 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
Q2 FY26 Tableau User Group Leader Quarterly Call
lward7
 
From Code to Challenge: Crafting Skill-Based Games That Engage and Reward
aiyshauae
 
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 

CrowdSem 2013 Workshop @ISWC2013

  • 1. CrowdSem 2013 1st International Workshop on “Crowdsourcing the Semantic Web” Sydney, Australia 21 October 2013 Follow us on Twitter: @crowdsem  Spread the news from today: #crowdsem2013 Monday, October 21, 13
  • 2. Who are we Maribel Acosta, Institute AIFB, Karlsruhe Institute of Technology, Germany Lora Aroyo, VU University Amsterdam, the Netherlands Abraham Bernstein, University of Zurich, Switzerland Jens Lehmann, University of Leipzig, Germany Natasha Noy, Stanford University, US Elena Simperl, University of Southampton, United Kingdom Monday, October 21, 13
  • 3. Workshop Program “Dr. Detective”: combining gamification techniques and crowdsourcing to create a gold standard in medical text, Anca Dumitrache, Lora Aroyo, Chris Welty, Robert-Jan Sips and Anthony Levas , (Discussant: Natasha Noy) 11:00 – 11:30 Crowdsourced Semantics with Semantic Tagging: “Don’t just tag it, LexiTag it!”, Csaba Veres, (Discussant: Lora Aroyo) 11: 30 – 12:00 SLUA: Towards Semantic Linking of Users with Actions in Crowdsourcing, Umair Ul Hassan, Sean O’Riain and Edward Curry, (Discussant: Maribel Acosta) 12:00 – 12:30 Experiment paper:  Crowdsourced Entity Markup, Lili Jiang, Yafang Wang, Johannes Hoffart and Gerhard Weikum, (Discussant: Abraham Bernstein) 12:30 – 12:45 Position paper: A Role for Provenance in Social Computation, Milan Markovic, Peter Edwards and David Corsar, (Discussant: Lora Aroyo) Monday, October 21, 13
  • 4. Workshop Program 1:45 – 2:15 Experiment paper: Frame Semantics Annotation Made Easy with DBpedia, Marco Fossati, Sara Tonelli and Claudio Giuliano, (Discussant: Maribel Acosta) 2:15 – 2:45 Experiment paper: Developing Crowdsourced Ontology Engineering Tasks: An iterative process, Jonathan Mortensen, Mark Musen and Natasha F. Noy, (Discussant: Elena Simperl) 2:45 – 3:15 Content and Behaviour Based Metrics for Crowd Truth, Guillermo Soberon, Lora Aroyo, Chris Welty, Oana Inel, Manfred Overmeen and Hui Lin, (Discussant: Abraham Bernstein) 3:15 – 3:30 Position paper: Information Reputation, Peter Davis and Salman Haq, (Discussant: Elena Simperl) Papers accessible at: http://ceur-ws.org/Vol-1030/ Monday, October 21, 13
  • 5. Invited Talk 4:00 – 5:00 Big Data meets Big Social: Social Machines and the Semantic Web David De Roure, University of Oxford This talk will discuss this opportunity for Semantic Web approaches to underpin the design and construction of future Social Machines against a backdrop of increasing scale and increasing automation. David De Roure is Professor of e-Research at University of Oxford and Director of the Oxford e-Research Centre. Throughout his career he has worked closely in and across multiple disciplines to foster innovation in digital methods and scholarship. His current projects range from Web Science (Social Machines and Web Observatories) to Digital Humanities (computational musicology). He has a coordinating role in Digital Humanities at Oxford and is a UK strategic adviser for data resources in social media. Monday, October 21, 13
  • 6. Crowdsourcing Exercise Task: Gather structured information about the three keynotes at ISWC2013, e.g. find pictures find research interests find publications etc. Form groups of experienced and novices in practical crowdsourcing experience Choose whether you would want to run the task on CrowdFlower or AMT Create a simple design of the task (20 mins) One group member presents the design to the whole workshop (5 mins) Selected tasks will be performed on either AMT or CrowdFlower and we will discuss the results at the wrap-up session Monday, October 21, 13
  • 10. Stay tuned on Twitter for the results Follow us on Twitter: @crowdsem  Spread the news from today: #crowdsem2013 Monday, October 21, 13