GeoServer for Spatio-temporal Data Handling With Examples For MetOc And Remote Sensing - FOSS4G 2015
2 likes•1,603 views
The document discusses GeoServer's capabilities for handling spatio-temporal data, particularly focusing on integration with NetCDF and other formats for effective image processing and data visualization. Key features include the management of granules, support for multiple dimensions, and RESTful operations for granule indexing and retrieval. It highlights challenges faced during data integration, customization, and configuration in geospatial applications.
,DoubleFileNameExtractorSPI[elevationregex
](elevation)
elevationregex.properties file
regex=(?<=_)(d{3})(?=_)
elevationregex.properties file
regex=[0-9]{8}T[0-9]{9}Z(?!.*[0-9]{8}T[0-9]{9}Z.*)
NCOM_wattemp_020_20081031T0000000_12.tiff](https://image.slidesharecdn.com/gsaadrspatiotemporalfoss4g2015-150921072824-lva1-app6891/85/GeoServer-for-Spatio-temporal-Data-Handling-With-Examples-For-MetOc-And-Remote-Sensing-FOSS4G-2015-9-320.jpg)
![XML indexer : collectors and params
FOSS4G 2015, Seoul
14th-19th September 2015
<collectors>
<collector name="fileDateCollector">
<value>[0-9]{8}</value>
<spi>TimestampFileNameExtractorSPI</spi>
<mapped>fileDate</mapped>
</collector>
<collector name="updatedCollector">
<value>MODIFY_TIME</value>
<spi>RuntimeExtractorSPI</spi>
<mapped>updated</mapped>
</collector>
</collectors>
<parameters>
<parameter name="AbsolutePath" value="true" />
<parameter name="AuxiliaryFile" value="polyphemus-test.xml" />
<parameter name="IndexingDirectories"
value="D:/Training_2.4_multidim_Win64/source_data/polyphemus" />
</parameters>](https://image.slidesharecdn.com/gsaadrspatiotemporalfoss4g2015-150921072824-lva1-app6891/85/GeoServer-for-Spatio-temporal-Data-Handling-With-Examples-For-MetOc-And-Remote-Sensing-FOSS4G-2015-24-320.jpg)
GeoServer for Spatio-temporal Data Handling With Examples For MetOc And Remote Sensing - FOSS4G 2015
- 1. GeoServer for Spatio-temporal Data Handling With Examples For MetOc And Remote Sensing Ing. Simone Giannecchini Ing. Daniele Romagnoli Ing. Andrea Aime GeoSolutions
- 2. GeoSolutions Founded in Italy in late 2006 Expertise • Image Processing, GeoSpatial Data Fusion • Java, Java Enterprise, C++, Python • JPEG2000, JPIP, Advanced 2D visualization Supporting/Developing FOSS4G projects GeoServer, MapStore GeoNetwork, GeoNode, Ckan Clients Public Agencies Private Companies http://www.geo-solutions.it FOSS4G 2015, Seoul 14th-19th September 2015
- 3. Reference scenario FOSS4G 2015, Seoul 14th-19th September 2015
- 4. ImageMosaic – the basics FOSS4G 2015, Seoul 14th-19th September 2015
- 5. Terminology Granule/Tile The individual raster element composing the mosaic (Granule) Index The collection of metadata records describing the location, spatial coverage and other attributes of each single granule Dimensions/Domains The dimensions besides the spatial ones used to distinguish individual granules FOSS4G 2015, Seoul 14th-19th September 2015
- 6. GraNules Assumptions Granules must share the same Coordinate Reference System Granules must share the same ColorModel and SampleModel not anymore in GeoServer 2.8.0, assuming you enable JAI- EXT Granules can overlap as they please, can have different resolutions Granules can be in different file formats (faster if the format is uniform) FOSS4G 2015, Seoul 14th-19th September 2015
- 7. The mosaic index Always present Drives the collection of granules for mosaicking Implemented by default using GeoTools Vector Sources Currently supported/tested sources PostGIS (JNDI) Oracle (JNDI) it’s been a nightmare because of naming! H2 Shapefile Can be customized to support custom granule indexes (e.g. legacy catalog) FOSS4G 2015, Seoul 14th-19th September 2015
- 8. The dimensions Maps to alphanumeric attributes in the index TIME and ELEVATION receive special treatment for WMS and WCS Custom/Additional dimensions Everything besides TIME & ELEVATION Map to DIM_XXX in WMS They can be dynamically discovered FOSS4G 2015, Seoul 14th-19th September 2015
- 9. Parsing dimensions from file names FOSS4G 2015, Seoul 14th-19th September 2015 indexer.properties file TimeAttribute=ingestion ElevationAttribute=elevation Schema=*the_geom:Polygon,location:String,ingestion:java. util.Date,elevation:Double PropertyCollectors=TimestampFileNameExtractorSPI[timereg ex](ingestion),DoubleFileNameExtractorSPI[elevationregex ](elevation) elevationregex.properties file regex=(?<=_)(d{3})(?=_) elevationregex.properties file regex=[0-9]{8}T[0-9]{9}Z(?!.*[0-9]{8}T[0-9]{9}Z.*) NCOM_wattemp_020_20081031T0000000_12.tiff
- 10. Publishing dimensions FOSS4G 2015, Seoul 14th-19th September 2015
- 11. NetCDF FOSS4G 2015, Seoul 14th-19th September 2015
- 12. NetCDF format support FOSS4G 2015, Seoul 14th-19th September 2015 NetCDF support Support COARDS/CF* conventions loosely Expose NetCDF internal data as a set of 2D slices Fast 2D (time, elevation) slice extraction
- 13. NetCDF index support files FOSS4G 2015, Seoul 14th-19th September 2015 NetCDF Internal Index Speeds up 2D slice extraction H2/PostgreSQL + binary file Index location is configurable via –DNETCDF_DATA_DIR Data in a non-writable location Granule Index in a DBMS Individual NetCDF Indexes on a separate directory
- 14. One or more variables FOSS4G 2015, Seoul 14th-19th September 2015 Polyphemus Sample Dataset 1 File Multiple Coverages!
- 15. NetCDF indexing configuration FOSS4G 2015, Seoul 14th-19th September 2015
- 16. NetCDF indexing configuration FOSS4G 2015, Seoul 14th-19th September 2015 NetCDF Indexer drive the indexing Automatically created, can be hand edited to expose some coverages only or parametrize the indexing <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Indexer> <schemas> <schema name="default" > <attributes>the_geom:Polygon,imageindex:Integer,time:java.util.Date, elevation:Double</attributes> </schema> </schemas> <coverages> <coverage> <name>O3</name> <schema ref="default"></schema> </coverage> … </coverages> </Indexer>
- 17. NetCDF GridMapping to EPSG codes FOSS4G 2015, Seoul 14th-19th September 2015
- 18. Assumptions/Limitations FOSS4G 2015, Seoul 14th-19th September 2015 Only WGS84 is supported in GeoServer 2.8.x also custom CRS are supported (GridMapping to EPSG properties) Only NetCDF following COARDS/CF convention are supported NetCDF WCS output is only available for multidimensional data sources (ImageMosaic and NetCDF)
- 19. ImageMosaic – NetCDF integration FOSS4G 2015, Seoul 14th-19th September 2015
- 20. NetCDF/Mosaic integration FOSS4G 2015, Seoul 14th-19th September 2015 ImageMosaic NetCDF integration Allow the ImageMosaic to handle multiple NetCDF files Expose NetCDF internal structure (times, elevations) Make ImageMosaic handle slices of the NetCDF file as granules Mosaic NetCDF 1 NetCDF 2 NetCDF … N …
- 21. XML based indexer file FOSS4G 2015, Seoul 14th-19th September 2015 Definition of Dimensions/Domains Definition of table schema Definition of Coverage Mapping of dimensions and table schema to Coverages PropertyCollector definition Additional Indexing Parameters: Path Behaviour Indexing Directories Aux File
- 22. XML indexer: domains/dimensions FOSS4G 2015, Seoul 14th-19th September 2015 <?xml version="1.0" encoding="UTF-8" standalone="yes" ?> <Indexer> <domains> <domain name="time"> <attributes><attribute>time</attribute></attributes> </domain> <domain name="elevation"> <attributes><attribute>elevation</attribute></attributes> </domain> <domain name="fileDate"> <attributes><attribute ref="fileDateCollector">fileDate</attribute></attributes> </domain> <domain name="updated"> <attributes><attribute ref="updatedCollector">updated</attribute></attributes> </domain> </domains>
- 23. XML indexer : schemas and coverages FOSS4G 2015, Seoul 14th-19th September 2015 <schemas> <schema name="default" > <attributes>the_geom:Polygon,location:String,imageindex:Integer,time:java.util.Date, elevation:Double, fileDate:java.util.Date,updated:java.util.Date </attributes> </schema> </schemas> <coverages> <coverage> <name>V</name> <schema ref="default"></schema> <domains> <domain ref="time" /> <domain ref="elevation" /> <domain ref="fileDate" /> <domain ref="updated" /> </domains> </coverage> </coverages>
- 24. XML indexer : collectors and params FOSS4G 2015, Seoul 14th-19th September 2015 <collectors> <collector name="fileDateCollector"> <value>[0-9]{8}</value> <spi>TimestampFileNameExtractorSPI</spi> <mapped>fileDate</mapped> </collector> <collector name="updatedCollector"> <value>MODIFY_TIME</value> <spi>RuntimeExtractorSPI</spi> <mapped>updated</mapped> </collector> </collectors> <parameters> <parameter name="AbsolutePath" value="true" /> <parameter name="AuxiliaryFile" value="polyphemus-test.xml" /> <parameter name="IndexingDirectories" value="D:/Training_2.4_multidim_Win64/source_data/polyphemus" /> </parameters>
- 25. Multiple coverages per mosaic FOSS4G 2015, Seoul 14th-19th September 2015
- 26. REST configuration of mosaic contents FOSS4G 2015, Seoul 14th-19th September 2015 Granule Index CRUD Operations via REST CREATE curl -u admin:Geos -XPUT -H "Content- type:application/zip" --data-binary @http://localhost:8080/geoserver/rest/workspaces/geosolu tions/coveragestores/temperature/file.imagemosaic READ index schema curl -v -u admin:Geos -XGET http://localhost:8080/geoserver/rest/workspaces/geosolut ions/coveragestores/polyphemus/coverages/NO2/index.xml READ WFS like with CQL filtering and paging curl -v -u admin:Geos -XGET "http://localhost:8080/geoserver/rest/workspaces/geosolu tions/coveragestores/polyphemus/coverages/NO2/index/gran ules.xml?limit=1&filter=time='2013-03-03T00:00:00Z
- 27. REST configuration of mosaic contents FOSS4G 2015, Seoul 14th-19th September 2015 READ (Retrieve specific granule by ID) curl -v -u admin:geoserver -XGET "http://localhost:8080/geoserver/rest/workspaces/topp/cove ragestores/polyphemus- v1/coverages/NO2/index/granules/NO2.2689.xml" UPDATE (harvest) curl -v -u admin:Geos -XPOST -H "Content-type: text/plain" -d "/polyphemus_20130303.nc" "http://localhost:8080/geoserver/rest/workspaces/geosoluti ons/coveragestores/polyphemus/external.imagemosaic" DELETE WFS like with CQL filtering and paging or by ID curl -v -u admin:geoserver -XDELETE "http://localhost:8080/geoserver/rest/workspaces/topp/cov eragestores/polyphemus- v1/coverages/NO2/index/granules.xml?filter=location='poly phemus_20130301.nc'"
- 28. GRIB/GRIB2 Format Support FOSS4G 2015, Seoul 14th-19th September 2015 Based on same UCAR NetCDF Java libs used by NetCDF Format Same capabilities of NetCDF Format Same indexing logic Same ImageMosaic support
- 29. The services - visualize FOSS4G 2015, Seoul 14th-19th September 2015
- 30. WMS FOSS4G 2015, Seoul 14th-19th September 2015 TIME, ELEVATION & Custom dimensions http://localhost:8080/geoserver/geosolutions/wms?... &time=2013-03-1T00:00:00.000Z &elevation=35.0 &DIM_FILEDATE=2013-03-01T00:00:00.000Z &DIM_UPDATED=2013-04-08T08:18:41.597Z
- 31. Rendering transformations FOSS4G 2015, Seoul 14th-19th September 2015 SLD Based transformations On-the-fly contouring On-the-fly poligonalization Current arrows Wind Barbs Pluggable: add your own
- 32. Rendering transformations FOSS4G 2015, Seoul 14th-19th September 2015 windbarbs currents contouring
- 33. WMS-EO FOSS4G 2015, Seoul 14th-19th September 2015 Extending LayerGroup concept Support same style on both raster and vector data Support custom dimensions Alter map on the fly to support band combination Wizard to configure EO layer groups Product layer tree Mix of vector and raster info Raw data, flags (clouds, snow), browse
- 34. WMS-EO FOSS4G 2015, Seoul 14th-19th September 2015 Helper UI to build the tree
- 35. WMS-EO FOSS4G 2015, Seoul 14th-19th September 2015
- 36. The services - download FOSS4G 2015, Seoul 14th-19th September 2015
- 37. WCS 2.0 FOSS4G 2015, Seoul 14th-19th September 2015 Basics Core service KVP binding XML binding Common extensions CRS Scaling Interpolation Range subsetting GeoTiff GML NetCDF Add the output format extensions GeoTIFF GML Grid NetCDF
- 38. WCS 2.0 processing chain FOSS4G 2015, Seoul 14th-19th September 2015 Crop RangeSubset Scale & Interpolate Reproject & Interpolate Encode Core Extension Extension Extension Extension
- 39. WCS 2.0 DescribeCoverage FOSS4G 2015, Seoul 14th-19th September 2015 BBOX TIME ELEVATION CUSTOM
- 40. Download ND NetCDF FOSS4G 2015, Seoul 14th-19th September 2015 http://localhost:8080/geoserver/wcs?request=GetCoverage&service=WCS& version=2.0.1&coverageId=geosolutions__NO2 &Format=application/x-netcdf &subset=http://www.opengis.net/def/axis/OGC/0/Long(5,20) &subset=http://www.opengis.net/def/axis/OGC/0/Lat(40,50) &subset=http://www.opengis.net/def/axis/OGC/0/elevation(300,1250) &subset=http://www.opengis.net/def/axis/OGC/0/time("2013-03- 01T10:00:00.000Z","2013-03-01T22:00:00.000Z") • Can also handle custom dimensions • Described in DescribeCoverage in a vendor metadata section
- 41. WCS EO FOSS4G 2015, Seoul 14th-19th September 2015 Add support for WCS EO metadata in readers Associate each file with EO metadata Include such information in DescribeCoverage/DescribeEODataset EODataset: multidimensional mosaic or NetCDF file Describe the inner structure (granules listing) in DescribeEODataSet
- 42. That’s all folks! Questions? [email protected] FOSS4G 2015, Seoul 14th-19th September 2015