Hi All

I have been diving into marble documentations these days trying to figure out the rationale behind what is the best way to display and route on the osm map using marblewidget.

1. I have not tried but seen from lots of posts. Is it correct that marble was not initially designed to read .osm, .pbf or .kml maps directly but pre-rendered tiles in .gdml format? Thats why if we load a large .osm extract it would become quite slow for marble to render.

2. If I also want to do routing in addition to just displaying, loading the raw map data is a must? Any advices on the right way to do this?

3. By the way, is now the marblewidget support .pbf file loading and rendering?

Thanks guys.

dalishi wrote:1. I have not tried but seen from lots of posts. Is it correct that marble was not initially designed to read .osm, .pbf or .kml maps directly but pre-rendered tiles in .gdml format? Thats why if we load a large .osm extract it would become quite slow for marble to render.

Any software that renders large-scale maps needs some kind of tiling approach to handle the massive amount of data. Software that allows loading large files does some kind of preprocessing to generate tiles internally. Marble has that sort of preprocessing integrated as well. You can see it for some map themes when opening them for the first time and a progress dialog pops up saying something like "Generating tiles, please wait". This however is for map themes.

When you open an .osm or .kml file with Marble you are not loading a map theme however. This sort of file opening is meant to load some additional data into an existing map theme. Notice that when you open a file everything is displayed on top of the existing map theme. There is a wizard integrated into Marble (right click on the map themes in the map themes panel to start it) which allows an easy creation of new map themes. But so far there is no support to create a map theme from e.g. an .osm file.

There was a GSoC project in 2012 to integrate vector tile maps into Marble. See http://www.danitxu.com/blog/tag/ander-pijoan/ for some information. Unfortunately not everything of it could be integrated and we still lack substantial progress on that.

dalishi wrote:2. If I also want to do routing in addition to just displaying, loading the raw map data is a must? Any advices on the right way to do this?

Marble has no algorithm implemented that does routing itself, it always queries some service (e.g. openrouteservice or osrm) or uses a locally installed routing software. Additional data is only needed for offline routing. Most notably monav does offline routing and is very well integrated into Marble (e.g. Marble lets you download its data from within the routing config dialog).

dalishi wrote:3. By the way, is now the marblewidget support .pbf file loading and rendering?

We first need properly working vector tile support before .pbf support makes sense really. It's not too hard to implement (we have a pbf parser in one of the tools already) and rendering comes for free once the data is parsed -- Marble uses the same internal format for the data it loads, so no further changes are needed.

Earthwings wrote:When you open an .osm or .kml file with Marble you are not loading a map theme however. This sort of file opening is meant to load some additional data into an existing map theme. Notice that when you open a file everything is displayed on top of the existing map theme. There is a wizard integrated into Marble (right click on the map themes in the map themes panel to start it) which allows an easy creation of new map themes. But so far there is no support to create a map theme from e.g. an .osm file.

Hi Earthwings, sorry for the late reply and thank you for your reply which is of great valuable to me to understand how marble works better.

Actually this offline routing is what I was initially planning to do. As our project has new map layers to add to existing osm map data, so we prefer to load a local .osm file and display in marblewidget, instead of using a map theme which request a map from online server.

I am not sure if I understand right. As you mentioned, it seems to me that using a map theme (.gdml) is the only way for marble to request/display maps (in online manner)? There is no way to do offline .osm rendering/display? i.e. to render a local .osm by configuring the map theme? I can take care of the routing by using some offline router on the .osm raw data and paint the route on marblewidget. So just the rendering/display now is a pain for us.

Thanks.

There are a couple of viable options. It depends mostly on your use-case which one is the best.

Without any modifications to Marble you can start it like this:

Code: Select all

marble --map=earth/openstreetmap/openstreetmap.dgml /path/to/overlay1.osm /path/to/overlay2.osm

to load Marble with the OpenStreetMap theme openened and the two local files overlay1.osm and overlay2.osm.

If you prefer to do some modifications, you can create your own map theme. This is useful to e.g. increase the maximum zoom level, have custom texture layers or pre-load .osm or other files. Here's an example map theme myosm.dgml derived from openstreetmap.dgml, see http://nienhueser.de/marble/myosm.png.
You can download the map theme from http://nienhueser.de/marble/myosm.zip Just extract it in ~/.local/share/marble/maps/earth/ and start Marble, it will appear in the list of map themes on the Map View panel. You need the patch from https://git.reviewboard.kde.org/r/124673/ though to have .osm files rendered correctly when they're loaded this way.

Neither of both solves the problem of loading large .osm files. You could use the C++ API as described in https://techbase.kde.org/Projects/Marble/Runners/LoadingOSM to load smaller .osm files when they come into view and remove them when they get out of view.
In essence this is what vector tiling is about, just that Marble takes care of the loading/unloading on its own. We made a lot of progress recently for an Android app of Marble. Vector tiling would be extremely useful for it and I expect some progress on the topic because of that.

Regarding offline usage: Marble maintains a local cache for all map themes. Before querying map tiles (be it textures or vector) from a server, it tries to find a cached version of the tile on the local file system. If it finds a suitable tile, it will use it and not query any server. So offline rendering just needs a local cache that has all tiles that are needed. See the subdirs in ~/.local/share/marble/maps/earth/openstreetmap for an example of the local cache. Inside Marble you can use the File => Download Region menu item to populate the local cache for the current map theme.
Assuming you have enough disk space (or you setup a local OSM server) or the areas you're interested in are limited anyway, you could just populate the local cache and have Marble use it.

So in the end it depends on your use-case mostly. If you just want a base texture layer for some limited region, use the Download Region feature. If you have some custom additions to the map in a local .osm file, load that on top. You can also set Marble to "Work Offline" to avoid that it tries to query missing tiles from a server.

Hi Earthwings, thanks for your detailed reply.

Our use case is simple. We modify open street maps (currently for a small area in .osm format) and then show the modified maps. We use the Qt and marblewidget rather than the marble globe regular program. The reason is we need to integrate into our software. Totally off-line usage.

According to your instructions, what we can do is load the .osm on top of the existing map theme. But I realized that the map theme still need to query from the server probably for the tile images for display, even for the area we load the .osm file, if I disconnect my Internet, the loaded area will not show. So can not be used offline. So my question is:

1. What is the point loading local .osm or .kml file (draw on top) while Marble actually did not render tiles from those loaded .osm or .kml file but still query from server for tiles?

For offline usage another way you mentioned is use the cached tiles images. My question is:

2. Since we will have modified the osm data which might be totally new in everything, so where can I get the image tiles for the modified .osm data? Should I set up my own osm server to generate all the tiles for marble to visualize? Can marblewidget do this for me, I mean, load directly a osm data (no matter in what format, .osm, .kml .postGIS whatever) and visualize? If can, how should I do?

Thank you.

I think there are two sane options. The first is to use only image tiles and then exchange the modified ones. This can be done in the local tile cache, but a better approach would be to setup a local tile server and have it regenerate tiles on changes. Most of the work then happens on that server. The second option is to use only vector rendering (load .osm files) and modify them directly, or reload them on changes. Currently this does not scale for larger areas in Marble, but we plan to work on that in early September. There's also some work that might be interesting for you in a just finished GSoC project, see http://mariusoc.blogspot.ro/2015/08/wra ... gs-up.html