20100512 Workflow Ramage

Workshop on Workflows on Earth Observation Steven Ramage, OGC [email_address] 21 June 2010, Sir Clive Granger Building, University of Nottingham

The importance of workflow Workflows are important because real jobs rarely involve few, simple tasks. Creating value in almost any area of endeavor requires the sequencing and organisation of human efforts, or of digital or mechanical processes, e.g. Supply chain manufacturing Insurance claims processing Building design All can be studied as workflows. The geospatial realm is no different. Image processing and spatial modeling can also be characterized as workflows.

Description of workflow In information systems workflows are a series of coordinated analytical and information processing steps Typically to transform data into information or knowledge Workflows could be described as Web-based scripts or macros that automate serial information tasks.

OGC Interoperability Program Collaborative, standards-based interoperability initiatives Sponsors define operational issues where standards can help and initiatives are created Technology providers come together to prototype solutions to tackle sponsors’ interoperability problems. OGC Web Services (OWS) initiatives underway for years OWS-5 had geospatial workflows as a primary focus, a detailed video report is available at: http://www.opengeospatial.org/projects/initiatives/ows-5

OGC Web Services standards OGC has four main OGC Web services standards for geospatial data sharing and processing: OGC Web Mapping Service Interface Standard (WMS) defines a Web API for requesting a picture of data, such as a PNG, JPEG, or GIF. OGC Web Feature Service Interface Standard (WFS) defines a Web API for accessing raw vector data in formats such as GML, KML or GeoRSS and adding, updating and deleting data.

OGC Web Services standards OGC has four main OGC Web services standards for geospatial data sharing and processing: OGC Web Coverage Service Interface Standard (WCS) defines a web API for accessing raster data in formats such as GeoTIFF, JPEG2000, or HDF WCS also supports adding, updating and deleting data. OGC Web Processing Service Interface Standard (WPS) defines a web API for running an algorithm or model, specifying data inputs and outputs

“ Chaining” Web Services for decision support … WCS WPS – Classification WPS - WCTS WFS Internet Web servers OGC Interfaces Service chaining creates value-added products Decision Support Client Geoprocessing worklow developed in OGC testbeds since 2004 Assess Wildfire Activity

BPEL workflow in OWS-5 Participants chained combinations of services together to address the requirements of geospatial data conflation and image processing. It also addressed an important gap, which was the lack of SOAP (Simple Object Access Protocol) and WSDL (Web Service Description Language) descriptions for those APIs.

BPEL workflow in OWS-5 Both W3C standards are critical for creating and managing workflows using industry standards such as OASIS’s BPEL (Business Process Execution Language). BPEL is the workflow description language used in many end-user service chaining tools such as Oracle BPEL Designer and George Mason University’s BPEL Power.

Conflation workflow architecture

Conflation workflow architecture The OWS-5 geo-processing workflow thread bound together WFS and WPS services in BPEL scripts to model data conflation in a consistent, repeatable fashion. This successful prototype signaled the utility of this approach for a host of geospatial tasks that require the combination of many different geospatial services and operations. Fusion activities developed in OWS-7 and will have more focus in this area for OWS-8.

OGC Web Processing Service (WPS) WPS GetCapabilities Execute DescribeProcess Algorithms Repository … … Algorithm 1 Data Handler Repository … … Data Handler A Communication over the web using HTTP WPS-client Web Processing Service

REST-oriented workflow in OWS-5 The SOAP/WSDL/BPEL approach to chaining Web services is one of the most widely used ways to implement service oriented architectures (SOA). However, other approaches, notably REST (Representational State Transfer) services, are also used to implement services in an enterprise. Some feel that Web services do not need the extra layer of description provided by SOAP and WSDL to build reliable workflows.

REST-oriented workflow in OWS-5 REST uses HTTP and other parts of the basic Web platform. In the SWE (Sensor Web Enablement) thread of OWS-5, participants used a REST-based architecture and WfXML to create workflows that allowed raw earth observation data to be acquired from assets, such as the EO-1 satellite, and then utilized by different WPS. Relevant data were then published and distributed to end-users in formats such as GeoAtom (Atom with GeoRSS extensions) and KML.

Integrating smoke WPS into WfXML-R workflow GMU GRASS SOAP/WSDL GRASS Image Algebra EO-1 SOS Google Earth Execute Inputs: URL Grid 1 URL Grid 2 OutputType EO Sensor Web Workflow GMU WCS-T GetCapabilities DescribeProcess NGC EO-1 Smoke WPS Calculated smoke geotiff Classified smoke geotiff Call calculation Call classifier Processes: FineSmoke CoarseSmoke Inputs: URL Grid 1 URL Grid 2 OutputType Output: URL Grid 3

OWS-6 WPS Grid Processing Bastian Baranski and Bastian Schäffer, Univ. Muenster Institute for Geoinformatics (IfGI) and 52° North, Germany Andrew Woolf, Science and Technology Facilities Council (STFC), UK Lan-Kun Chung, GIS Center, Feng Chia University, Taiwan

OWS-6 WPS grid processing Grid processing profiles of WPS - specification WPS grid processing profile integrated with grid computing infrastructure: Job Submission Description Language (JSDL) High Performance Computing (HPC) Basic Profile (HPC-BP) Simple API for Grid Applications (SAGA) Data Access and Integration set of specifications (WS-DAI-*) Web Services Resource Framework (WSRF)

OWS-6 WPS grid processing OGF enabled WPS - implementation WPS to benefit from and integrate with distributed computing resources and technologies Two potential ways to make use of OGF specifications, concepts and their implementations where identified encapsulating other resources integration alongside other services

Grid-enabled WPS implementations Airport Scenario Trajectory Service (WPS JSDL Profile) Science and Technology Facilities Council (STFC), UK Plume Rendering Service (WPS JSDL Profile) Institute for Geoinformatics (IfGI), University of Münster, Germany

Grid-enabled WPS implementations Debris Flow Scenario Rainfall Data Interpolation (WPS HPC Basic Profile) Institute for Geoinformatics (IfGI), University of Münster, Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan Geophone Data Analysis (WPS HPC Basic Profile) Institute for Geoinformatics (IfGI), University of Münster, Germany Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan

OGC online resources OWS-6 Geoprocessing Workflow Architecture Engineering Report 0.3.0 09-053r5 Bastian Schäffer 2009-10-09 This document covers geo-processing workflow best practices and methods in a SOA environment.

Debris flow – scenario A debris flow is a fast moving mass of unconsolidated, saturated debris that looks like flowing concrete. Based on existing debris flow monitoring system (proprietary and monolithic software infrastructure) in Taiwan (running since 2003). Develop and implement open (OGC) standards-based service oriented architecture (SOA) for debris flow monitoring.

Debris flow – sensors and geo-processing Landslides and flooding are a threat on the mountainous island of Taiwan, due to typhoons and earthquakes. The Geographic Information Systems Research Center, Feng Chia University (GIS.FCU) in Taiwan has implemented OGC services for use in workflow, detecting and analyzing sensor data for emergency response. There is a demonstration showing the working network of debris flow sensors and examples of distributed services performing analysis and processing the sensor data.

Debris flow operational scenario

Encapsulation vs. integration Integration requires a WSRF binding for WPS

GEOSS AIP-2 flood prediction and response Led by NASA, Spot Image, Northrop Grumman, ERDAS From portal select desired theme and area of interest Wizard picks appropriate workflow for desired result Wizard Mozambique Disaster Management Information System (DMIS) Workflows Estimated rainfall accumulation and flood prediction model Flood Model Selected workflow automatically activates needed assets and models Baseline water level, flood waters and predicted flooding

GEOSS AIP-2 biodiversity & climate change Led by CNR, Univ of Colorado, GBIF IP3 Client & Workflow engine IP3 Distributed Community Catalog/Mediator WCS - T Other Non-OGC Services GBIF Non-OGC Services OGC WPS Access to Model CSW GEOSS Portal WCS WFS req req req resp resp resp req req resp resp req resp req resp req resp Broker Research Scientist req resp

SWE and geo-processing workflow SOS ! SAS Sensor Net Mission Control Center Access & Processing Node WPS CSW Register Measurement Types SPS CSW SAS SOS

The power of workflow Workflows can be formally described as programmes that invoke Web services that invoke other Web services that invoke other Web services. Analytical models, decision support tools and other complex resources can quickly deliver results in this way and optionally, provide documentation of the steps. There is an OGC Workflow Domain Working Group: this is a forum for describing, discussing and providing best practice guidance related to geospatial workflows using existing IT workflow standards.

20100512 Workflow Ramage

More Related Content

Viewers also liked

Similar to 20100512 Workflow Ramage

More from Steven Ramage

20100512 Workflow Ramage

Editor's Notes