An introduction
to open data
Anders Pedersen @anpe / @okfn[.org]
CC-By v3 Licensed (all jurisdictions)
Government data: Not a new thing
Government data: Not a new thing
http://dabrownstein.files.wordpress.com/2013/02/visualizing-slavery-carpenter.jpg
Fast
forward: Is it
safe
to cycle?
Where Does Our
Money Go?
We notice when government data
collection fails
• US government is
prohibited by law to
collect data on
gun related deaths
• Argentina’s inflation
statistics cannot be trusted
• Greek financial figures
until 2011 were unreliable
• April 2014: Nigeria adjust their size of GDP by 80 per cent
3 global
challenges
Data Locked Up
Data Hard to Use
No-one to Do It
We are an global network working for since 2004 to
open up data and see it used to empower citizens
and organizations to answer questions that matter
and drive positive change
“Central” team of more than 35 on 4 continents.
Community network including civil servants, civil-
society researchers and citizens with presence in more
than 40 countries - including the Philippines, Indonesia,
South Korea, Japan and China!
Advocacy and Expertise in
Opening up Data
Create tools, skills and
communities
Data
=>
Knowledge & Insight
OKFestival
Open Knowledge Festival
Largest open data event in the world
Berlin 2014
What?
Data is open if anyone is free to use, reuse
and redistribute it for any purpose without
restriction or charge
http://OpenDefinition.org/
Why?
Challenge: Exploding
Complexity of information
In 1820 all UK bank clearing done in a single room in
London once a day
Today, billions of transactions a minute.
=> componentization to divide and conquer complexity
Opportunity: Information
Technology
Smart phone = system for the Apollo moon landings
1TB of storage < $100 - in 1994 $450,000.
Mass participation in information access, processing and
production. Decentralization.
Image: ItoWorld
OpenStreetmap Edits
Many Eyes Will Bring
Knowledge To Society
And Government
Source: http://data.gov.uk/data/openspending-
report/index
£200m potential
saving for NHS
Credit: Ben Goldacre &
Open Healthcare UK
http://openprescribing.org/
http://prescribinganalytics.com/
Who committed aid for Yolanda?
The Many Minds Principle
Best Thing to Do With Your Data Will Be
Thought of By Someone Else
£200m potential
saving for NHS
Credit: Ben Goldacre &
Open Healthcare UK
http://openprescribing.org/
http://prescribinganalytics.com/
3 E’s
Economy
Efficiency
Empowerment
Huge Growth in Open Data in
Last Few Years
Especially for Government Data
Data Catalogs Around the World as of July 2012
Source: http://datacatalogs.org/
Data Catalogs Around the World as of July 2012
Is open data simply “open
washing”?
Open washing was coined by Christian Villum at Open Knowledge in this blog post:
http://blog.okfn.org/2014/03/10/open-washing-the-difference-between-opening-your-
data-and-simply-making-them-available/
Open Data Census/Index
“Russian officials use the ranking on the
Index as one of KPIs of data openness. It’s
been very helpful for us - open data activists
- to promote open data and open knowledge
in Russia.”
Ivan Begtin, Ambassador,
Open Knowledge Russia
“Since the Index came out, a number of
countries - including the Russian,
Indonesian, German and Belgian
Governments - are using it as a yardstick for
their achievements or lack of it.”
Andrew Stott, former Director for
Transparency and Digital
Engagement for the UK
Government, & Open Data Advisor
at the World Bank
In October 2013, ahead of the annual Open Government
Partnership (OGP) summit in London, Open Knowledge
launched the Open Data Index, the first major assessment of
the state of open government data in the world.
The Index ranked 70 countries according to the availability
and accessibility of data in ten key categories, and is based
on peer reviewed submissions from the Open Data Census.
No natural order:
leaders are those who act
Source: Daily Nation, Kenya, November 10, 2012, http://www.nation.co.ke/business/news/Open-data-initiative-has-hit-a-dead-end/-
/1006/1617026/-/n18uhrz/-/index.html
Walk the walk on open data
● Better data
● Publish data that matters - local and
granular
● Release in open formats
● Know how well you score
Better data
● Improve formats eg. PDF → CSV
● Documentation: What is in your data
● Low data quality and closed formats will
result in reduced reuse
For driving local data use:
Granularity is key
Release data that matters:
eg. grades at school level
Source: Twaweza, Tanzania, List of worst schools,
http://www.shule.info/schools/worst
Tip: the story is almost always buried in
granular data
Source: Mapumental
Case I: fair distribution of
government subsidies
Government:
Provincial breakdown shows
“equal distribution”
Breakdown by postal code =
rich gained 20 times higher
subsidies than poor areas
Case II: US Farm subsidy
data
Data is open if anyone is free to use, reuse
and redistribute it for any purpose without
restriction or charge
http://OpenDefinition.org/
Without open formats data
will not be reusable and truly
open data
Example: geodata portal
Source: national.census.okfn.org
Know your baseline:
assess how to improve
In doubt about priorities?
Release raw data first...
CSOs or journalists will then
take care of the rest
Tackling questions about
open data:
The global top 3 excuses
why data should not be
opened up
1) We cannot release data
due to privacy or commercial
sensitivity
2) Data quality is not good enough. It’s
complicated!
Response: Just clean it up!
3) We do not have the resources
Response: You do not need to release nice
visualisations, just release the bare CSVs.
So besides all these tips:
What makes an open data
initiative successful?
Change makers in government working
together to make it happen!
http://okfn.org/
http://ckan.org/
http://schoolofdata.org/

An introduction to open data