Meeting today’s dissemination challenges – Implementing International Standards in OECD.Stat MSIS Dublin 14-16 April 2014

MEETING TODAY‟S
DISSEMINATION CHALLENGES:
Implementing international standards in .Stat
Prepared by Jonathan Challener, OECD
For MSIS, April 2014 - Dublin, Ireland

Doesn’t non-standard power supplies make things difficult?

What happens when standards are not applied well?

Picture: ‘The day Sweden changed from left-hand drive to right’
Confusion entails

and inefficiencies!
“A little like the grade 8 student who doesn‟t
pay attention in class all year”.

What is .Stat?
.Stat is the central repository ("warehouse")
of validated statistics and related metadata
.Stat is the central hub connecting data production,
sharing & dissemination processes
It is the corporate source of data for
data sharing and dissemination purposes

What is .Stat?
.Stat is the central repository ("warehouse")
of validated statistics and related metadata
.Stat is the central hub connecting data production,
sharing & dissemination processes
It is the corporate source of data for
data sharing and dissemination purposes
“.Stat is now being used and shared with 10 organisations
including the OECD, as part of the Statistical Information
System Collaboration Community (SIS-CC)”.

.Stat Positioning in Statistical Information System
DATA DELIVERY
INTERNAL DATA
SHARING
DATA DISSEMINATION
DATA PRODUCTION
.STAT

.Stat Positioning in Statistical Information System
DATA DELIVERY
INTERNAL DATA
SHARING
DATA DISSEMINATION
DATA PRODUCTION
.STAT
“The diagram illustrates the .Stat contribution to the SIS processes.
.Stat’s core value-added lies in “Data Delivery”, a set of functions that
enable dissemination and data sharing, and “Data Upload”, a set of
functions interfacing data production processes into a single upload
mechanism to feed dissemination channels”.

.Stat Functional Representation
.STAT DATA DELIVERY ENGINE
DATA PRODUCTION
DATA SHARING DATA DISSEMINATION
SEARCH
ENGINES
DATA ANALYSIS
TOOLS
P
C
WEBSITES, APPS
PUBLICATIONS.STAT BROWSER
.STAT DATA
UPLOAD ENGINE
FILE
UPLOAD
SDMX
IMPORT
DATA PRODUCTION
TOOLS
TABLE & CHART
EXTRACTION SERVICES
RELEASE MGT
SERVICES
.STAT BROWSER
CONFIGURATION
DATA
EXTRACTION SERVICES
SDMX
INPUT
E
P
BATCH
UPLOAD
SDMX
GLOBAL
REGISTRY
PUBLISHING
BACK
OFFICE
DATA
MAPPING
SDMX
OUTPUT
X
X
.Stat
Component
Process
Human user
Data Producer
Data Editor
Data Consumer
API or
Webservice
Other
SDMX hubs

.STAT DATA DELIVERY ENGINE
DATA PRODUCTION
DATA SHARING DATA DISSEMINATION
SEARCH
ENGINES
DATA ANALYSIS
TOOLS
P
C
WEBSITES, APPS
PUBLICATIONS.STAT BROWSER
.STAT DATA
UPLOAD ENGINE
FILE
UPLOAD
SDMX
IMPORT
DATA PRODUCTION
TOOLS
TABLE & CHART
EXTRACTION SERVICES
RELEASE MGT
SERVICES
.STAT BROWSER
CONFIGURATION
DATA
EXTRACTION SERVICES
SDMX
INPUT
E
P
BATCH
UPLOAD
SDMX
GLOBAL
REGISTRY
PUBLISHING
BACK
OFFICE
DATA
MAPPING
SDMX
OUTPUT
X
X
.Stat
Component
Process
Human user
Data Producer
Data Editor
Data Consumer
API or
Webservice
Other
SDMX hubs
“The grey shaded boxes in the figure below show a visual
representation of how .Stat fits within a broader Data Dissemination
Information System of organisations; the boxes with dotted lines
represent other components of the Data Dissemination Information
System that are not supported by .Stat but are enabled by it”.

In particular, .Stat provides the following 3 key functional areas…

.Stat Functional Representation.Stat Data Upload Engine

.Stat Functional Representation.Stat Data Delivery Engine

.Stat Functional Representation.Stat Data Browser

.Stat Positioning in GSBPM Reference Model
.Stat contributes to
Planned additions
Archive incorporated into the over-
arching process of data and metadata
management

.Stat Positioning in GSBPM Reference Model
.Stat contributes to
Planned additions
Archive incorporated into the over-
arching process of data and metadata
management
“.Stat can be mapped today to the Generic Statistical Business
Process Model (GSBPM) under “Disseminate” and “Build”. In the
future it will also incorporate archive functions as part of the over-
arching process for data and metadata management”.

Multipurpose SDMX within .Stat…

For dissemination and data eXchange
SDMXWS and RESTful API
• SDMX 2.0 compliant
• SOAP + REST
• Pull
• SDMX-ML
• SDMX Structural metadata
created on the fly

For „Open Data‟ dissemination
SDMX-JSON (beta)
SDMX-TWG agreed in mid 2013 on proposal for data and their
structural metadata (inc. flat & sliced layouts) and referential
metadata (dataset, series, obs) as annotations.
Further enhancements to come: Complete data structures and
referential metadata

For data reporting
SDMX-Reference
Infrastructure (RI)*
• SDMX 2.0 and 2.1 compliant
• SOAP + REST
• SDMX Common APIs
(SdmxSource.NET)
• Pull + Push
• SDMX-ML, GESMES , CSV
• Structural metadata stored in
mini registry
• One web service - several
mapped database instances
Mapping
Store DB
XXX.Stat
Data
warehouse
SDMX-RI
Web Service
Dissemination
Mapping
Assistant
SDMX-RI
* The integration of SDMX-RI in .Stat is based on collaboration with Eurostat,
provider of the SDMX-RI component with ISTAT taking the lead on behalf of
the OECD‟s Statistical Information System Collaboration Community.

For internal data sharing
DirectAccess
• Restful SDMX query
• Flat data, flags, units
• Referential metadata
Excel-add-in
• DirectAccess (Rest SDMX)
• Native Excel pivot table
• Wizard to select data

For a decentralised publishing environment
DataHub*
• One interface to the publishing
tools
• Centralised reporting and auditing
• SDMX based structural metadata,
and referential metadata
management
• Flexible load tool that promotes
‘self publish’ for data custodians
• In-built checks and safeguards to
minimise errors
• Manages security and access
rights
• Can be extended to manage other
outputs and not limited to .Stat
* DataHub has been developed and integrated with .Stat by
Statistics NZ, with an additional connection to the Fusion Registry
for managing structural metadata through the definition of DSDs.

SDMX-RDF data cube vocabulary pilot

SDMX-RDF data cube vocabulary pilot
“Explore further semantic web/linked data
opportunities (SDMX-RDF data cube
vocabulary). To be taken forward by ISTAT and
ABS under the SIS-CC umbrella”.

• Lower technology adoption costs
• Increased development consistency, simplicity and
predictability
• Improved code reuse
• Reduced cost, time and effort to transition between
different solutions
We all know the…
• Reduced focus on infrastructure
• Ability to create composite interfaces that are tailored to the needs of
specific task
• Improved application portability
• Enable faster time to market because it is easier to use off the shelf
components and applications that can integrate and provide features for the
solution

References
1. Operationalising .Stat in a decentralised publishing
environment (DataHub) by Tony Breen SNZ :
https://community.oecd.org/docs/DOC-68362
2. Building a scalable architecture (.Stat) by Jens Dosse OECD:
3. SDMX-RI and .Stat integration by Francesco Rizzo Istat:
4. SDMX-JSON API: http://stats.oecd.org/opendataapi/Index.htm

Jonathan Challener, OECD
jonathan.challener@oecd.org
@Challener
MSIS - Dublin, 14-16 April 2014
Meeting today’s dissemination challenges: Implementing international standards in .Stat
Thank you

Meeting today’s dissemination challenges – Implementing International Standards in OECD.Stat MSIS Dublin 14-16 April 2014

More Related Content

What's hot (20)

Viewers also liked (13)

Similar to Meeting today’s dissemination challenges – Implementing International Standards in OECD.Stat MSIS Dublin 14-16 April 2014 (20)

Recently uploaded (20)

Meeting today’s dissemination challenges – Implementing International Standards in OECD.Stat MSIS Dublin 14-16 April 2014