Putting Your Data on the Map
Geospatial Data Manipulation and

Visualisation for Digital Humanities
Shawn Day - 16 February 2015
Objectives
‣ To briefly explore the concepts behind creation,
manipulation and presentation of scholarly research
using spatial visualisations;
‣ To engage in free and informal discussion about how
these might be employed in your own research;
‣ Most of all: Inspire and Imagine.
Agenda
‣ What Do Great GeospatialVisualisations Look Like?
‣ What Core Concepts UnderlieWorking with Spatial Data?
‣ Where Does the Data Come From?
‣ What DoYou Need to Do to Put it On a Map?
‣ A Smattering of Tools
‣ Discussion
Why areYou Here?
Herodotus Time Map
http://hestia.open.ac.uk/herodotus/basic.html
Google Ancient PlacesVisualisation
http://gap.alexandriaarchive.org/gapvis
GaugeMap
NewspapersWest
‣ RuralWest Initative
‣ Bill Lane Centre
‣ for the
‣ USWest
ORBIS
ORBIS
What is Geospatial?
‣ More than GIS
‣ Its all about Space *and* Place - Context
‣ Geovisualisation?
‣ Geoparsing?
What is Geospatial Not?
‣ Automatic (although that’ll get closer later in the this
afternoon ;-)
Starting with aWee Glossary
‣ Georeference
‣ Geoparse
‣ Geotag
‣ Geocode
‣ Geolocate
‣ GPS
‣ GIS
‣ Projection
‣ Geovisualisation
‣ Geosearch
‣ Gazateer
Georeferencing
‣ Identify a place in space
‣ Usually a minimum of Longitude, Latitude, Altitude
‣ Location: 54.583805, -5.9317343
Geoparsing
‣ Identifying as a place as opposed to another type of
entity within a body of text
‣ To assign geographic identifiers to words and phrases in
unstructured content
‣ Useful Services: Metacarta / Calais / CalaisFull
‣ “Software “reads” documents and web pages in a human-
like way to identify geographic terms and references
using natural language processing (NLP)”
Geotag
‣ Add geographical reference information to media such as
photographs, video, websites, blog posting, etc.
www.geoimgr.com/
Geocoding
‣ Forward Geocoding:
‣ The process of finding geographic coordinates from address
data
‣ Batching?
‣ Reverse Geocoding:
‣ Matching geographic coordinates to augmented information
in a human readable form
Geolocating
‣ Assessing the location of a real world object based on IP-
address, or mobile connection to the internet
Global Positioning Systems
‣ Series of US Defense Department Satellites in Orbit that
allow for accurate positioning in three dimensions
‣ Additional services available from GLONAS (Russia) and
emerging GALILEO (Europe) and COMPASS (China)
Geographic Information Systems (GIS)
‣ GIS can display spatial data hidden in tables and
databases
‣ Create detailed and intelligent maps
‣ Integrate data to reveal trends and relationships that
bring new perspectives to previously held beliefs about
people and places
‣ Research questions in the humanities often involve a
spatial component that only GIS can expose
‣ HistoricalGIS
Projection
‣ Conformal projection primarily preserves shape;
‣ Equidistant projection primarily preserves distance;
‣ Equal-area projection primarily preserves area.
Geovisualisation
‣ A paractice involving the presentation of geographic data
and concepts through interactive visualisation.
‣ It’s what we do today.
Kai Krause
http://kai.subblue.com/images/True-Size-of-Africa-kk-v3.pdf
Gazateer (Digital)
‣ Topographic

Dictionary
‣ Database
‣ List
‣ Concordance
Intermission ….
Geospatial Standards
‣ KML
‣ GeoJSON
‣ GML
‣ GeoRSS
‣ GPSX
‣ Boundary File (shapefile)
Keyhole Markup Language KML
‣ A language for the visualisation of geographic
information
‣ Placemarks
‣ Ground Overlays
‣ Paths
‣ Polygons
‣ Styles
‣ Google Earth
‣ Can be embedded in TEI
Sample KML
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document>
<Placemark>
<name>New York City</name>
<description>New York City</description>
<Point>
<coordinates>-74.006393,40.714172,0</coordinates>
</Point>
</Placemark>
</Document>
</kml>
GeoJSON
‣ A format for encoding a variety of geographic data
structures;
‣ Unlike the other standards mentioned, GeoJSON is
written and maintained by an Internet working group of
developers;
‣ It’s Open;
‣ http://geojson.org.
GeoJSON
‣ Geometry Objects
‣ Positions
‣ Point/MultiPoint
‣ Line/MultiLineString
‣ Polygon/MultiPolygon
‣ Geometry Collection
‣ Feature Objects
‣ Feature Collection Objects
‣ Lists of geometries are
represented by a
GeometryCollection.
‣ Geometries with
additional properties are
Feature objects.
‣ Lists of features are
represented by a
FeatureCollection.
GeoJSON Example
{ "type": "GeometryCollection",
"geometries": [
{ "type": "Point",
"coordinates": [100.0, 0.0]
},
{ "type": "LineString",
"coordinates": [ [101.0, 0.0], [102.0, 1.0] ]
}
]
Geography Markup Language GML
‣ Adopted by the Open Geospatial Consortium (OGC)
‣ Can be embedded in TEI
GML
‣ Feature
‣ Geometry
‣ Coordinate Reference
System
‣ Topology
‣ Time
‣ Dynamic feature Coverage
‣ Unit of measure
‣ Directions
‣ Observations
‣ Map presentation styling
rules
GML Example
<PhotoCollection xmlns="http://www.myphotos.org" xmlns:gml="http://www.opengis.net/gml"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.myphotos.org
MyGoodPhotos.xsd">
<items>
<Item>
<name>Lynn Valley</name>
<description>A shot of the falls from the suspension bridge</description>
<where>North Vancouver</where>
<position>
<gml:Point srsDimension="2" srsName="http://www.opengis.net/def/crs/EPSG/0/4326">
<gml:pos>49.40 -123.26</gml:pos>
</gml:Point>
</position>
</Item>
</items>
</PhotoCollection>
What Sort of Packages Use Geo Standards
‣ OpenLayers;
‣ Leaflet;
‣ MapServer;
‣ Geoforge;
‣ GeoServer;
‣ GeoDjango;
‣ CartoDB
‣ UMap;
‣ Google Maps.
Place and the TEI
‣ Guidelines found in 13.2.3 Place Names and 13.3.4
Places in P5
‣ First concerns naming of place (duh!) and the second the
locating of the place in space (a little more complex)
‣ Placename
‣ <placeName @type/> or <geogName @type/>
‣ Place
‣ <place><placeName @type><location><geo/></location></
placename></place>
Place Naming
‣ placeName contains an absolute or relative place name
to a geo-political reference.
‣ <placeName/ @key>
‣ eg. 

<placeName key=”FO-01”>

<settlement type=””town”>OFFOY

</settlement>

<country type=”nation”>France

</country>

</placeName>
<placeName/>
district
settlement
region
country
bloc
@key @type
Trying it Out
Steps
1. Collect Data
2. Georeference
3. Geoparse
4. Transform
5. Map
6. Munge
7. Analyse
8. Present
9. Refine
Experiment 1
‣ Question: Where do you all come from??
‣ I have a list
‣ I added cities to it
‣ Cities Geoparsed in GPSVisualizer
‣ Export as CSV
‣ Import to UMap
‣ Embed in Blog
‣ Prettify
Step 1 - Acquire Data
Step 2: Geoparsing
Step 3: Import to UMap
Step 3.1: Create a New Layer
Step 3.2: Add Today’s Attendees
Step 3.3: Adjust Properties
Step 3.4: Add a Line/Journey
Step 3.5:Who Came to the LastWorkshop?
Step 4: Share the Map
Step 5 Prettify
‣ http://umap.openstreetmap.fr/en/map/anonymous-edit/
28857%3AZ9AgUoghX7FeLbOWS3Ya-mWKcyE
A Real An Intriguing UMap Example
UMap
‣ Help and More Info
‣ http://umap.openstreetmap.fr
‣ http://wiki.openstreetmap.org/wiki/
UMap#Feedback_and_help
Digital Objects on the Map
Experiment 2
‣ Where Do I Find Belfast and Dublin?
‣ Use Edina UnLock to query Gazateers
‣ Generate a Datafile
‣ Inspect
‣ Place onto UMap
‣ Break into Layers
‣ Adjust Properties
‣ Capture for Sharing
Step 1: Using a Digital Gazetteer
‣
{
‣
"type": "FeatureCollection",
‣
"features": [
‣
{
‣
"type": "Feature",
‣
"id": "4568138",
‣
"bbox": [-0.12883, 51.50051, -0.12883, 51.50051],
‣
"properties": {
‣
"name": "London",
‣
"sourceid": "2643743",
‣
"country": "United Kingdom",
‣
"countrycode": "GB",
‣
"adminlevel1": "England",
‣
"adminlevel2": "Greater London",
‣
"adminlevel3": "",
‣
"adminlevel4": "",
‣
"featuretype": "Capital of a Political Entity",
‣
"unlockFeatureCode": "GN.P.PPLC",
‣
"custodian": "GeoNames",
‣
"gazetteer": "GeoNames",
‣
"scale": "unknown",
‣
"centroid": "-0.12883, 51.50051",
‣
"population": "7556900",
‣
"altidentifiers": [9654368, 9679],
‣
"footprint": "http://unlock.edina.ac.uk/ws/footprintLookup?format=json&identifier=4568138"
‣
}
‣
},
‣
{
‣
"type": "Feature",
‣
"id": "4753",
‣
"bbox": [0.34303968256350553, 51.27823964309304, -0.5145139108357116,
51.697124957394415],
‣
"properties": {
‣
"name": "London Euro Region",
‣
"sourceid": "41428",
‣
"country": "United Kingdom",
‣
"countrycode": "GB",
On the UMap
Experiment 3
‣ I already have data - what can I do with it?
‣ batchgeo.com?
Summarising
‣ Hopefully it seems easier now than it may have at the
outset
‣ It’s not automatic
‣ Tools works together
‣ Data Sharing
‣ Sustainability
‣ Ease of Use
‣ Extensibility
‣ Applicability
Upcoming Seminars
‣ 20 February - Creating Lightweight Digital Maps with
GeoJSON
‣ 5/6 March - DataVisualisation in the Digital Humanities
‣ 23 March - Using Google Tools for Digital Humanities
Scholarship
‣ 20 April - Requirements Engineering for Humanities/
Social Science Scholars
‣ May - Digital Project Management
Thanks
Shawn Day
@iridium
s.day@qub.ac.uk
http://qubdh.co.uk

Putting Your Data on a Map

  • 1.
    Putting Your Dataon the Map Geospatial Data Manipulation and
 Visualisation for Digital Humanities Shawn Day - 16 February 2015
  • 2.
    Objectives ‣ To brieflyexplore the concepts behind creation, manipulation and presentation of scholarly research using spatial visualisations; ‣ To engage in free and informal discussion about how these might be employed in your own research; ‣ Most of all: Inspire and Imagine.
  • 3.
    Agenda ‣ What DoGreat GeospatialVisualisations Look Like? ‣ What Core Concepts UnderlieWorking with Spatial Data? ‣ Where Does the Data Come From? ‣ What DoYou Need to Do to Put it On a Map? ‣ A Smattering of Tools ‣ Discussion
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
    NewspapersWest ‣ RuralWest Initative ‣Bill Lane Centre ‣ for the ‣ USWest
  • 9.
  • 10.
  • 11.
    What is Geospatial? ‣More than GIS ‣ Its all about Space *and* Place - Context ‣ Geovisualisation? ‣ Geoparsing?
  • 12.
    What is GeospatialNot? ‣ Automatic (although that’ll get closer later in the this afternoon ;-)
  • 13.
    Starting with aWeeGlossary ‣ Georeference ‣ Geoparse ‣ Geotag ‣ Geocode ‣ Geolocate ‣ GPS ‣ GIS ‣ Projection ‣ Geovisualisation ‣ Geosearch ‣ Gazateer
  • 14.
    Georeferencing ‣ Identify aplace in space ‣ Usually a minimum of Longitude, Latitude, Altitude ‣ Location: 54.583805, -5.9317343
  • 15.
    Geoparsing ‣ Identifying asa place as opposed to another type of entity within a body of text ‣ To assign geographic identifiers to words and phrases in unstructured content ‣ Useful Services: Metacarta / Calais / CalaisFull ‣ “Software “reads” documents and web pages in a human- like way to identify geographic terms and references using natural language processing (NLP)”
  • 16.
    Geotag ‣ Add geographicalreference information to media such as photographs, video, websites, blog posting, etc. www.geoimgr.com/
  • 17.
    Geocoding ‣ Forward Geocoding: ‣The process of finding geographic coordinates from address data ‣ Batching? ‣ Reverse Geocoding: ‣ Matching geographic coordinates to augmented information in a human readable form
  • 18.
    Geolocating ‣ Assessing thelocation of a real world object based on IP- address, or mobile connection to the internet
  • 19.
    Global Positioning Systems ‣Series of US Defense Department Satellites in Orbit that allow for accurate positioning in three dimensions ‣ Additional services available from GLONAS (Russia) and emerging GALILEO (Europe) and COMPASS (China)
  • 20.
    Geographic Information Systems(GIS) ‣ GIS can display spatial data hidden in tables and databases ‣ Create detailed and intelligent maps ‣ Integrate data to reveal trends and relationships that bring new perspectives to previously held beliefs about people and places ‣ Research questions in the humanities often involve a spatial component that only GIS can expose ‣ HistoricalGIS
  • 21.
    Projection ‣ Conformal projectionprimarily preserves shape; ‣ Equidistant projection primarily preserves distance; ‣ Equal-area projection primarily preserves area.
  • 22.
    Geovisualisation ‣ A paracticeinvolving the presentation of geographic data and concepts through interactive visualisation. ‣ It’s what we do today.
  • 23.
  • 24.
  • 25.
  • 26.
    Geospatial Standards ‣ KML ‣GeoJSON ‣ GML ‣ GeoRSS ‣ GPSX ‣ Boundary File (shapefile)
  • 27.
    Keyhole Markup LanguageKML ‣ A language for the visualisation of geographic information ‣ Placemarks ‣ Ground Overlays ‣ Paths ‣ Polygons ‣ Styles ‣ Google Earth ‣ Can be embedded in TEI
  • 28.
    Sample KML <?xml version="1.0"encoding="UTF-8"?> <kml xmlns="http://www.opengis.net/kml/2.2"> <Document> <Placemark> <name>New York City</name> <description>New York City</description> <Point> <coordinates>-74.006393,40.714172,0</coordinates> </Point> </Placemark> </Document> </kml>
  • 29.
    GeoJSON ‣ A formatfor encoding a variety of geographic data structures; ‣ Unlike the other standards mentioned, GeoJSON is written and maintained by an Internet working group of developers; ‣ It’s Open; ‣ http://geojson.org.
  • 30.
    GeoJSON ‣ Geometry Objects ‣Positions ‣ Point/MultiPoint ‣ Line/MultiLineString ‣ Polygon/MultiPolygon ‣ Geometry Collection ‣ Feature Objects ‣ Feature Collection Objects ‣ Lists of geometries are represented by a GeometryCollection. ‣ Geometries with additional properties are Feature objects. ‣ Lists of features are represented by a FeatureCollection.
  • 31.
    GeoJSON Example { "type":"GeometryCollection", "geometries": [ { "type": "Point", "coordinates": [100.0, 0.0] }, { "type": "LineString", "coordinates": [ [101.0, 0.0], [102.0, 1.0] ] } ]
  • 32.
    Geography Markup LanguageGML ‣ Adopted by the Open Geospatial Consortium (OGC) ‣ Can be embedded in TEI
  • 33.
    GML ‣ Feature ‣ Geometry ‣Coordinate Reference System ‣ Topology ‣ Time ‣ Dynamic feature Coverage ‣ Unit of measure ‣ Directions ‣ Observations ‣ Map presentation styling rules
  • 34.
    GML Example <PhotoCollection xmlns="http://www.myphotos.org"xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.myphotos.org MyGoodPhotos.xsd"> <items> <Item> <name>Lynn Valley</name> <description>A shot of the falls from the suspension bridge</description> <where>North Vancouver</where> <position> <gml:Point srsDimension="2" srsName="http://www.opengis.net/def/crs/EPSG/0/4326"> <gml:pos>49.40 -123.26</gml:pos> </gml:Point> </position> </Item> </items> </PhotoCollection>
  • 35.
    What Sort ofPackages Use Geo Standards ‣ OpenLayers; ‣ Leaflet; ‣ MapServer; ‣ Geoforge; ‣ GeoServer; ‣ GeoDjango; ‣ CartoDB ‣ UMap; ‣ Google Maps.
  • 36.
    Place and theTEI ‣ Guidelines found in 13.2.3 Place Names and 13.3.4 Places in P5 ‣ First concerns naming of place (duh!) and the second the locating of the place in space (a little more complex) ‣ Placename ‣ <placeName @type/> or <geogName @type/> ‣ Place ‣ <place><placeName @type><location><geo/></location></ placename></place>
  • 37.
    Place Naming ‣ placeNamecontains an absolute or relative place name to a geo-political reference. ‣ <placeName/ @key> ‣ eg. 
 <placeName key=”FO-01”>
 <settlement type=””town”>OFFOY
 </settlement>
 <country type=”nation”>France
 </country>
 </placeName> <placeName/> district settlement region country bloc @key @type
  • 38.
  • 39.
    Steps 1. Collect Data 2.Georeference 3. Geoparse 4. Transform 5. Map 6. Munge 7. Analyse 8. Present 9. Refine
  • 40.
    Experiment 1 ‣ Question:Where do you all come from?? ‣ I have a list ‣ I added cities to it ‣ Cities Geoparsed in GPSVisualizer ‣ Export as CSV ‣ Import to UMap ‣ Embed in Blog ‣ Prettify
  • 41.
    Step 1 -Acquire Data
  • 42.
  • 43.
  • 44.
    Step 3.1: Createa New Layer
  • 45.
    Step 3.2: AddToday’s Attendees
  • 46.
    Step 3.3: AdjustProperties
  • 47.
    Step 3.4: Adda Line/Journey
  • 48.
    Step 3.5:Who Cameto the LastWorkshop?
  • 49.
  • 50.
  • 51.
  • 52.
    A Real AnIntriguing UMap Example
  • 53.
    UMap ‣ Help andMore Info ‣ http://umap.openstreetmap.fr ‣ http://wiki.openstreetmap.org/wiki/ UMap#Feedback_and_help
  • 54.
  • 55.
    Experiment 2 ‣ WhereDo I Find Belfast and Dublin? ‣ Use Edina UnLock to query Gazateers ‣ Generate a Datafile ‣ Inspect ‣ Place onto UMap ‣ Break into Layers ‣ Adjust Properties ‣ Capture for Sharing
  • 56.
    Step 1: Usinga Digital Gazetteer ‣ { ‣ "type": "FeatureCollection", ‣ "features": [ ‣ { ‣ "type": "Feature", ‣ "id": "4568138", ‣ "bbox": [-0.12883, 51.50051, -0.12883, 51.50051], ‣ "properties": { ‣ "name": "London", ‣ "sourceid": "2643743", ‣ "country": "United Kingdom", ‣ "countrycode": "GB", ‣ "adminlevel1": "England", ‣ "adminlevel2": "Greater London", ‣ "adminlevel3": "", ‣ "adminlevel4": "", ‣ "featuretype": "Capital of a Political Entity", ‣ "unlockFeatureCode": "GN.P.PPLC", ‣ "custodian": "GeoNames", ‣ "gazetteer": "GeoNames", ‣ "scale": "unknown", ‣ "centroid": "-0.12883, 51.50051", ‣ "population": "7556900", ‣ "altidentifiers": [9654368, 9679], ‣ "footprint": "http://unlock.edina.ac.uk/ws/footprintLookup?format=json&identifier=4568138" ‣ } ‣ }, ‣ { ‣ "type": "Feature", ‣ "id": "4753", ‣ "bbox": [0.34303968256350553, 51.27823964309304, -0.5145139108357116, 51.697124957394415], ‣ "properties": { ‣ "name": "London Euro Region", ‣ "sourceid": "41428", ‣ "country": "United Kingdom", ‣ "countrycode": "GB",
  • 57.
  • 58.
    Experiment 3 ‣ Ialready have data - what can I do with it? ‣ batchgeo.com?
  • 59.
    Summarising ‣ Hopefully itseems easier now than it may have at the outset ‣ It’s not automatic ‣ Tools works together ‣ Data Sharing ‣ Sustainability ‣ Ease of Use ‣ Extensibility ‣ Applicability
  • 60.
    Upcoming Seminars ‣ 20February - Creating Lightweight Digital Maps with GeoJSON ‣ 5/6 March - DataVisualisation in the Digital Humanities ‣ 23 March - Using Google Tools for Digital Humanities Scholarship ‣ 20 April - Requirements Engineering for Humanities/ Social Science Scholars ‣ May - Digital Project Management
  • 61.