GDAL & WebGL: European Commission, Joint Research Center, Ispra

Joint Research Centre - European Commision, November 22, Ispra, Italy.

Research & Development with
WebGL, Closure, GDAL & MapRank

Petr Pridal <petr.pridal@klokantech.com>

WebGL Earth Closure Tools
3D digital globe. JavaScript Toolset.

MapTiler Cluster
GDAL2Tiles on steroids. GeoSearch with ranking.

!

beta

WebGL Earth
Free and open source 3D digital globe
for web and mobile devices

HTML5 & WebGL

• HTML5 <canvas> WebGL extension
• Based on OpenGL ES
KLOKAN TECHNOLOGIES - Museo 300

• Supported by NVidia, ATI, Intel on desktop 500
KLOKAN TECHNOLOGIES - Museo

• HW acceleration on mobile devices
• Firefox 4+, Chrome 9+, Opera, Safari
• Internet Explorer: with Chrome Frame

WebGL: Shaders in GLSL



Inspired by OpenLayers.Layer.GoogleNG
(done by Andreas Hocevar)

Compliant to Google TOS?

Try the demo application:

www.webglearth.com

Mission of the project

• User friendly (dragging, zooming, ...)
• Keep it simple & maximally web oriented
• Raster data from prerendered tiles, WMS, - Museo 500
KLOKAN TECHNOLOGIES
existing tile APIs and TileCache, GeoServer...
• “NoServer” = o ine usage & cloud
• Calculations from CPU to GPU. Preprocess!
• Open-Source (GPLv3), Community powered

Open Source project

• Simple JavaScript API
• Embed <script src=“...../api.js”>
• Mashup with your own tiles, custom WMS, 500
overlays, markers and popups TECHNOLOGIES - Museo 700
KLOKAN

• The full codebase + demo app sources on
GitHub, object oriented, documented code,
manual describing internals available
• Contributors: KlokanTech, CampToCamp,
AI2 Spain, BIMServer.org, NOAA

JavaScript API: Demo


JavaScript API

• Embedding of the 3D globe
• Custom base maps and overlays with
adjustable transparency (tilesKLOKAN TECHNOLOGIES - Museo 300
+ WMS)

• Camera API, ﬂyTo, bbox KLOKAN TECHNOLOGIES - Museo 700

• User interaction and events
• Markers & popups
• <canvas> is part of HTML DOM

Codebase: we, weapp

• Demo application (www.webglearth.com)
completely open-source, API exports too
• Compiled with Closure Compiler with types: 300
http://blog.klokantech.com/2010/12/closure-compiler-for-openlayers-3x.html

• Documented (google style guideline)
• Technically, anything is possible :-)
• Manual describing the implementation
details: http://data.webglearth.com/doc/

Object hierarchy


“Keep it simple” design

• Sphere with diameter 1.0
• WGS84 coordinates used without ellipsoid
transformation (a la Google Maps, OSM, etc.)
• Textures in Spherical Mercator (EPSG:3857,
EPSG:900913)
• Clip stack for megatextures with lookup
• Segmented plane as the model

Segmented plane


Terrain


Vector data (direct rendering)

• Now: rasterized only (WMS, tiles with
Mapnik, ...). Raster tiles must be there for
aerial data.

• Future: direct rendering (from GeoJSON,

KML, WFS, ...)
• Sharing code for loading data formats with
OpenLayers 3, Leaﬂet, Polymaps?
• Vector Tiles - Google MapsGL

Future: Google MapsGL


Future: Mobile devices


Problems: drivers + CORS

• Long list of blacklisted graphic cards
• June 2011: Introduced restrictions on cross-
domain textures for WebGL :-(

• CORS HTTP Header: Access-Control-Allow-Origin: * - Museo 500
KLOKAN TECHNOLOGIES

• http://enable-cors.org/ or chrome --disable-web-security
• Support only in Chrome 15+ (beta), Firefox 8 (11/2011)

• Workaround for now: HTTP proxy
• Tiles hosted on your own server runs always!

Try it: 10 minut cookbook

• JavaScript API: check examples
• Prepare data (one of):
• WMS / TMS / WMTS in EPSG:900913 - Museo 500
KLOKAN TECHNOLOGIES

• Convert any GeoTIFF with GDAL2Tiles /
MapTiler
• Copy everything on your web server
• Display in any modern web browser

Try it at:
http://www.webglearth.com/

Project website, mailing-list, documentation:
http://www.webglearth.org/
Plans for integration with OpenLayers 3.0

Support, open-source contracted development:
Klokan Technologies: http://www.klokantech.com/

"

Closure Tools
OpenSource JavaScript Toolset
developed by Google Inc.

"

Compiler Library
Templates Linter Stylesheets JS Test
(Coding Style Guidelines)

http://code.google.com/closure/

It is all about compiler...

1. Compiles readable JavaScript into compressed
machine-readable JavaScript: dead code removal,
function inlining, etc.
2. JSDoc Tags: typos, wrong use of a @constructor,
wrong type of a variable, misuse of a ﬁeld
annotated with @private and @protected, etc.
3. A reusable JavaScript library, such as OpenLayers,
exports public API - and the compiler optimizes
the internal code.

http://goo.gl/H6twe

It is all about compiler...

4. Unused parts of the library stripped. Application
dependencies solved during compilation.
5. Constants to remove unwanted functionality: - Museo 300
KLOKAN TECHNOLOGIES
Mobile WebKit only, no Quirks mode support, - Museo 500
KLOKAN TECHNOLOGIES
compilation without support of IE6
6. Debugging with FireBug possible even for the
compiled version
7. Dynamically loadable modules, additional
functionality loaded only when required.

http://goo.gl/H6twe

Hello world

goog.require('goog.dom');

function sayHi() { KLOKAN TECHNOLOGIES - Museo 300
var newHeader = goog.dom.createDom('h1', TECHNOLOGIES - Museo 500
KLOKAN {'style':
'background-color:#EEE'},
'Hello world!');
goog.dom.appendChild(document.body, newHeader);
}

goog.exportSymbol('sayHi', sayHi);

How to start?

• Easiest is to get plovr - where all the tools
are packed into one .jar application.
• Run java -jar
plovr.jar serve config.json and use
it with HTML as <script src="http://localhost: - Museo 700
KLOKAN TECHNOLOGIES
9810/compile?id=hello"></script>

• Plovr: Hello world + skeletons at GitHub:
https://github.com/klokantech/closure-library-plovr-hello-world-skeletons/

• Closure Library websites:
http://code.google.com/closure/library/

Where it excels?

• Complex online applications - any serious
work done in JS
• More developers working on KLOKANsame code 300
the TECHNOLOGIES - Museo

• KLOKAN TECHNOLOGIES - Museo 700
High peformance app + mobile apps
• Libraries and APIs
• :-( no fast copy & paste hacking
• :-( steep learning curve, smaller community

Read more at:
http://code.google.com/closure/

Plovr.jar - the “packed” toolset:
http://plovr.com/KLOKAN TECHNOLOGIES - Museo 500

The book: Closure Library, The Deﬁnitive Guide

Blog post: http://goo.gl/H6twe

#

MapTiler Cluster
GDAL2Tiles on steroids.

MAPTILER - WHAT IS IT?

• MapTiler is an application for rapid online map publishing
• Input: raster geodata (GeoTIFF, JPEG2000, MrSID, ECW, Erdas HFA,
BSB, ...). Output: jpeg/png tiles
• Support for existing georeference (World Files, WKT, EPSG,...).
Transformation (warping) of projections possible.
• Standards: OSGEO Tile Map Service Speciﬁcation - TMS, comming OGC
WMTS:
• Web viewers (OpenLayers, Google Maps API) + applications
• KML ﬁles (Google Earth)
• iPhone + iPad + Android native applications
• ArcGIS - with extensions (ArcBruTile). 10.1+ native WMTS support

NECESSARY TRANSFORMATIONS AND WARPING

GEODETIC COORDINATES SPHERICAL MERCATOR PYRAMID COORDINATES TILE INDEX
LATITUDE LONGITUDE METERS XYZ PIXELS / ZOOM XYZ TILE / ZOOM
WGS 84 (EPSG:4326) EPSG:3785 / EPSG:900913 WEB VIEWERS TILE MAP SERVICE

SPHERICAL MERCATOR:
TILE PROJECTION AND COORDINATES
NOT ONLY IN GOOGLE MAPS!
• Popular on the web: Google Maps, Microsoft Bing
Maps, Yahoo Maps, OpenStreetMap, ...
• Lat/Long WGS84 (GPS devices) datum coordinates
always displayed to users - KML
• Map rasters are projected by a global Mercator
projection: “Spherical Mercator”. EPSG:900913 /
EPSG:3857 (meters)
• Conformal, cylindrical. Cropped to square (maximal
latitude ~ 85.05 degrees), WGS84 ellipsoid is sphere!

OPEN-SOURCE ALTERNATIVES:
PREGENERATING X CACHING TILES

• Generating from raster input: • Caching tiles from WMS servers
(MapServer, GeoServer):
• MapTiler & GDAL2Tiles
• TileCache.org (Python)
• MSR MapCruncher
• GeoWebCache.org (Java Servlet)
• Generating from features (vectors)
• Mapnik

• GMapCreator, • Commercial alternatives
• ... • License per server (ArcGIS Server)

EXTREMELY FAST ONLINE MAPS
FOR THOUSANDS OF PEOPLE

• Easy hosting: only static ﬁles / or raster blobs in a database
• Extremely fast delivery, highest-quality output possible (you
have more time for rendering), SLA for up to 99.99% availability

Our software is used ...
• ... in: web applications, mashups, native
mobile applications (iPhone/iPad/Android),
hardware appliances and 3D visualizations
• ... for: standard base maps, aerial imagery,
coverage maps, thematic maps, marine

navigation, by pilots in airplains, on the
ground during disaster events, ...
• ... by: Google, NOAA, NASA, CERN,
Vodafone, T-Mobile, Orange, Three, United
Nations, and thousands of developers

PARALLELIZED RENDERING:
MAPTILER CLUSTER
• Parallelized rendering on Multi-Core processors or even on
computer clusters (Amazon EC2 supported)
• Speed-up the rendering on modern computers

• Rendering on a computer cluster (Amazon EC2) - price: 0.1 USD / hour / core

• Publishing of tiles on a world-wide CDNs (Amazon S3/CloudFront, Google App
Engine, Azure, SimpleCDN) - fast multi-threaded upload for tiles possible

• Merging of huge raster datasets into a seamless map a la
Google Maps on a computer cluster:
• Fast parallelized rendering and merging of large data collections

s
nt
• Cluster parallelization based on MPI

lie
rc
• Test of MapReduce approach (via Nokia labs Disco project)

fo

MapTiler Cluster vs
GDAL2Tiles
• Fast rendering: multi-core CPU & MPI cluster
• Direct merging of several input ﬁles
• Excellent visual quality
• Heavily optimized PNGs/JPEGs tiles to speed
up the delivery and save disk space
• Precise "data over-zooming"
• Various coordinate systems supported

s
nt
lie
rc
fo

WWW . MAPTILER . ORG
Runs under Windows, Linux and Mac OS X + GDAL2Tiles
You can try it on the FOSS4G LiveDVD!
Community: MapTiler User Group
maptiler@googlegrups.com

MapTiler Cluster, commercial support, integration,
customization
http://www.maptiler.com/

GEOGRAPHICAL SEARCHING
• Research, scientiﬁc articles, work on Ph.D. thesis (Petr Přidal),
Computer Science + Geodesy and Cartography
• Alternative projects tested in OldMapsOnline.org (R&D project in MZK):
• Lucene with GeoTemporal Ext. (LGTE) from DigMap.eu
• Alexandria Digital Library
• TimeMap.net (ECAI.org)
• Google GeoSearch API
• Knowledge of traditional GIS search methods (PostGIS, ...)

Result: Development of a new geographical search engine
for map collections.

CORE FEATURES

• Map-driven interface: Simple. Intuitive. Web 2.0.
Users love it!
• Combined queries: Where, When, Who & What
• Results sorted geographically with an advanced
spatial ranking algorithm.
• Instant search. Immediate delivery of the results by
moving the map or typing.
• It is fast. Even for large databases (million+ records).

METADATA & CUSTOMIZATION
• Indexing metadata in MARC, DublinCore, ISO
19139 formats, ...
• Batch import or harvesting: OAI-PMH or OGC CWS
• Numerical geographical boundaries necessary!
(MARC 034, DublinCore DCMI) - enrichment with
Geoparser or Georeferencer tools
• Possible customization of the user interface,
integration into products or websites
• For developers: REST API (JSON format), queries
following OpenSearch.org

Try it at: http://demo.mapranksearch.com/

Try it at: http://rumsey.mapranksearch.com/

Try it at: http://mapy.mzk.cz/

MAPRANK RANKING &
SCALABILITY
• MapRank ranking: an advanced algorithm calculating
similarity of geographical area of query and area covered
by every document and combining it with map scale, time
range, predeﬁned categories and traditional full text
search.
• Data indexing implemented speciﬁcally to provide ranked
real-time results even for large databases. Traditional R-
Tree indexing or Lucene are failing.
• Server component: FastCGI, C++
• Default user interface developed with Closure JavaScript
Library + Closure Compiler.

ENRICHMENT OF METADATA

Geoparser
Text parsing and semi-automatic assignment of
bounding boxes to map records with detectable
geonames (gazetteer), known physical size and map
scale.

Georeferencer
Crowdsourced georeferencing. Higher accuracy, more
outputs beside geometadata such as visualization, 3D
view, comparing of maps, OGC WMS for use in GIS.

Thank you!
Questions, ideas?
Cooperation on OpenSource and R&D?
Coding week for OpenLayers3 & WebGL Earth?
Interest in applications of MapTiler / MapRank?

Petr Pridal <petr.pridal@klokantech.com>

GDAL & WebGL: European Commission, Joint Research Center, Ispra

More Related Content

Similar to GDAL & WebGL: European Commission, Joint Research Center, Ispra (12)

Recently uploaded (20)

GDAL & WebGL: European Commission, Joint Research Center, Ispra