openstreetmap on postgresql pgcon 2009 -...

46
OpenStreetMap on PostgreSQL pgcon 2009 May 2009 Steven Singer [email protected] [email protected] IRC: stevenSn

Upload: buidung

Post on 08-Jul-2018

222 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

OpenStreetMap on PostgreSQLpgcon 2009

May 2009

Steven [email protected]@radintl.comIRC: stevenSn

Page 2: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Map of the hour

What is a free map How OpenStreetMap works How to use the data with

PostgreSQL

Page 3: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

What is OpenStreetMap

A project to collect & distribute free geographic data

A wiki-editable map of the world

CC-by-SA licensed

Page 4: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Is Your Map Free?Flicker: izarbeltza

Page 5: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Free to See

flicker:iyllish

Page 6: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Free to give

Publishing Distribution (embedded

devices) Sell

Page 7: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Free to derive

Page 8: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Free to style

Page 9: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the
Page 10: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the
Page 11: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Free to editAdd placesCorrect mistakesAdd new roads

Page 12: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

How it got started

Founded by Steve Coast, Summer of 2004

flicker:davidhc

Page 13: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Crowd Sourcing

Page 14: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

How it works

Page 15: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the
Page 16: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Tags Can be anything

Page 17: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the
Page 18: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

<node id='34009069' lat='-33.4442638' lon='25.746064' user='Firefishy' xapi:users='CoreyBurger,

Firefishy' timestamp='2009-03-23T00:09:07Z' version='2'>

<tag k='elephant' v='many'/> <tag k='elephants' v='yes'/> <tag k='name' v='Addo Elephant National Park'/> <tag k='tourism' v='information'/> </node>

Page 19: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Where does the data come from

100,000+ contributors GPS traces local knowledge Bulk imports Imagery traces

Page 20: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

GPS

Page 21: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

TracingTracing

Page 22: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Data Imports

Copyright compatible datasets TIGER, vmap0 (US), GeoBase(Canada) NAND (Netherlands)

Page 23: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

114,000 users + 22,000,000 + highways 28,000 pubs

Page 24: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Where does the data go?

XML

REST

PNG

Page 25: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

The simple schema

Page 26: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Why PostgreSQL

Pre 0.6 Mysql + MYISAM Errors could mean only half an

edit Solution: Transactions InnoDB has no text search

Page 27: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

2x Intel Xeon Processor E5420 Quad Core 2.5Ghz 2x 73GB (3.5) SAS 15K10x 450GB (3.5) SAS 15K

PostgreSQL 8.3

30GB of RAM

Page 28: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

(pg)-stat

AVG 7.42M tuples fetched per second

AVG 1.04k tuples inserted per second

800GB

Page 29: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Table Size(GB)current_node_tags 99current_nodes 58current_way_nodes 44current_way_tags 20

Historicalnodes 211node_tags 119way_nodes 113gps_points 94way_tags 32ways 14

Page 30: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Planet Dumpshttp://planet.openstreetmap.org

Page 31: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Osmosis, your own API

Populates DB from planet dumps

Page 32: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

But I really want to use PostGIS

Complex Geometries Spatial queries

Page 33: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

osm2pgsql

Populates postgis from planet dumps

Creates spatial objects (lines) Populates attribute columns for

popular tags

planet dump

postgis

osm2pgsql

minute diffs

Page 34: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

schema

Page 35: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

planet_osm_lines

Roads Rivers Trails

Page 36: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

planet_osm_points

POI's amenity place_of_worship name

Page 37: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

planet_osm_polygons

Areas Land use building outlines lakes

Page 38: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Is my school a polygon or a point?

Tags can be anything You can map a school as a

area, buildings, or a single POI Must query all tables (UNION?)

Page 39: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

mapnikpostgis

Map Style<XML>

Tiles(PNG)

SVG

PDF(for printing)

OpenLayers (Slippery Map)

Page 40: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

I can do *GIS too

Page 41: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Nearby Pubs

SELECTp.name,ST_Distance(ST_Transform(p.way,3348),ST_Transform(u.way,3348) )

FROM planet_osm_point p, planet_osm_point u where p.amenity='pub' AND u.name='University of Ottawa' AND ST_Distance(p.way,u.way)<1;

~

Page 42: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

name | distancem-----------------------+------------------

Elephant & Castle | 541D'Arcy McGees | 764Drumlins | 937Meadowlands Pub | 7259Montana's Steakhouse | 9565Rockwell's Sports Bar | 9622Tanglewood Pub | 10339Greenfields Pub | 16710Barley Mow Pub | 17698Cheshire Cat Pub | 27110The Thirsty Moose | 48771Queen's Hotel | 48771 (12 rows)

Page 43: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Spatial queries

ST_Intersects ST_UNION ST_Distance

Page 44: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Things we can do

How free is your map? About OpenStreetMap How to use with PostGIS

Page 45: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

http://www.openstreetmap.org

http://www.mapnik.org

http://postgis.refractions.net

https://www.openjump.org

http://www.qgis.org

Questions?

Steve [email protected]@radintl.comIRC: StevenSn

Page 46: OpenStreetMap on PostgreSQL pgcon 2009 - …docs.huihoo.com/postgresql/pgcon/2009/OpenStreetMap-with... · Map of the hour What is a free map How OpenStreetMap works How to use the

Map Copyright OpenStreetMap & contributors

http://www.flickr.com/photos/izarbeltza/68084109/http://www.flickr.com/photos/jyllish/3497579854/

Photo Credits

NASA

http://www.flickr.com/photos/davidhc/382148384/

http://wiki.openstreetmap.org