• Presentación Smart City y Geolocalización: Héctor García
  • Sample Page
  • ES
  • EN
  • Avda. de Grecia 8. 41012 Sevilla, España
  • |
  • www.geographica.gs

Making tiles with Mapnik

Towards a tiles engine based on Spark

Posted at 10/05/2015 by Alberto Asuero

Why do we make tiles? For a reason of efficiency and speed. To create tiles consists of making a pyramid of the world and rendering the images previously. In this way our maps are faster because they only download the images that had been rendered.

The WMS services were good several years ago but today the needs are a quite different, and there are much better solutions for applications dealing with a large quantity of users. It has it’s uses, but it’s not a solution for scalable applications.

I’d like to tell you our experience:

We tile the orthophoto of Andalusia with an high level of detail (level 18). For that we used gdal2tiles.py. The first issue to deal with was the format, for the year 1956 it was a MrSID and we had to fight a bit with GDAL. The second one, and much harder, was to create the mosaic.

tiles: El rompido, year 1956 on the left and the year 1979 on the right
El rompido, year 1956 on the left and the year 1979 on the right

Check this out at CartoDB. Seville year 1956 – Seville year 1979

Even thought we achieved it, it looked really well but it took too much time (50 days). It was unacceptable so we start working to improve the timing and furthermore to have something that allow us to take data from the most common data sources.

The target was to tile the maps of each of our applications without thinking about the format of the data. We create Equidna – 100% OpenSource -, with this tools we’ve created hundreds of maps for different clients and we’ve been able to reduce the timing thanks to a multiprocessing implementation. It was a large improvement, using gdal2tiles it took 50 days and with Equidna we reduced it to 30 days.

Equidna is based on Mapnik and it allows you to make tiles with a wide variety of data sources: PostGIS, Shapefiles, GeoJSONs, GDAL, raster, etc…

Right now, we’re working on an improvement based on BigData using a Map/Reduce algorithm (despite of I’m not a fan the term “BigData” I think it helps for a better understanding). With this new approach we assign each machine a region of the world.

So, using an elastic platform as Amazon EC2 we can launch N machines; if a process takes 50 days, if we launch 50 machines it will take 1 day (aprox.) and the cost will be the same. We’re developing this platform on Spark.

In a few months we’ll post the results, and of course, it will be OpenSource.

BigDataGIS
GEO_blog_alberto
Posted at 10/05/2015 by
 Alberto Asuero
Chief Technology Officer
SHARE
Posted in: BigData, GIS | Tagged: Equidna, geographica, Mapnik, Spark, Tiles
← Geographica have a new web
Geodesic lines & GIS →

© 2023 Geographica. Todos los derechos reservados.

  • English
  • Español