Image to Heightmap converter

Two weeks ago I mentioned one more feature I want to add to make heightmap truly customizable — “get Heightmap from the image” function. I found the idea promising and now I want to describe the way it was implemented.

Let’s start from the idea I had. Currently there are 2 options for a heightmap customization: apply/create a template or paint the map manually using the “free draw” brushes. From my point of view there is too much work to draw a whole map manually, so the proposed use case is a template creation and using brushed only for a fine-tuning and separate features adding.

Heightmap templates work good, but there is a type of cartographers who want to recreate a pre-designed heightmap rather than use a randomized one. Some want to create a map based on existing one, like map of Westeros or map of Europe; others use graphics editors like GIMP or Photoshop to create maps and want either to try them in my generator or use the generator to place rivers/towns rationally. For both these groups I want to build a single solution allowing to load any raster image as an input and get a generator-usable heightmap as an output.

Once we have designated the idea we can start the implementation. First thing is to allow users to upload an image. This can be easily done via native FileReader object. Now we need to somehow process the image into a heightmap.

Loaded images usually have too many points and colors, to simplify image and meanwhile fit it into the underlying map structure we can tessellate the image to Voronoi graph. Process is pretty straightforward:

  • Load image
  • Draw image to canvas
  • Get canvas ImageData
  • Calculate a Voronoi diagram
  • For each diagram point get an appropriate image color of the image
  • Draw each diagram cell with defined color fill
Cheetah (wiki image processed)
A processed photo of Cheetah (source author: schani, licensed under CC)

The result looks like a stained-glass window, quite beautiful but pretty useless. The number of colors is still great, so the next step is to associate these colors with a standard color scheme I use for a heightmap. The scheme is a simple blue-to-red spectral color interpolation and contains colors representing each height from 0 to 1. That means my scheme contains about 100 colors and I can potentially normalize any color to fit it.

Technically it’s easy, the only problem is finding a correct height for a pretty random color. There is no and cannot be ideal solution. An uploaded image may have almost any color scheme and there is no way to assign a right height automatically. The only options here is to assume the height based on some color attribute and let user to fix it manually.

Images people may upload can be split into 4 groups: not maps (nothing to discuss here, the result is unpredictable), maps without clear elevation color coding (at least we can try to separate land from water), maps with colors showing elevation (we can assume height by used hue) and monochrome heightmaps (height is defined by lightness). So we can try to programmatically predict a heightmap for 2 out of 4 groups of images. Not bad.

Lab_color_space from wiki
Lab color space

For each diagram cell we have selected a color from the uploaded image. Color is represented in RGB color model, which is not suitable for calculations based on human vision. Using D3 it’s very easy to convert the color to Lab space. Lab is a color system with 3 dimensions: L for lightness and a and b for the color components green–red and blue–yellow. So L dimension is ideal for monochrome heightmaps interpretation, while for colored maps we can try to use a combination of a and b values.

var imageData = ctx.getImageData(0, 0, mapWidth, mapHeight);
var imgData = imageData.data;

polygons.map(function(i, d) {
  var x = i.data[0], y = i.data[1]; // get Voronoi polygon point coords
  var p = (x + y * mapWidth) * 4; // find appropriate image point
  var r = imgData[p]; // get red light from imageData
  var g = imgData[p + 1]; // get green light from imageData
  var b = imgData[p + 2]; // get blue light from imageData
  var lab = d3.lab("rgb(" + r + ", " + g + ", " + b + ")"); // get lab color
  if (type === "hue") {
    var normalized = normalize(lab.b + lab.a / 2, -50, 200); // normalize by hue
  } 
  if (type === "lightness") {
    var normalized = normalize(lab.l, 0, 100); // normalize by lightness 
  }
  var rgb = color(1 - normalized); // get color from heightmap color scheme
  cells[d].height = normalized; // assign height
  landmass.append("path") // draw cell
    .attr("d", "M" + i.join("L") + "Z")
    .attr("fill", rgb)
    .attr("stroke", rgb); 
});

As you can see normalized value is calculated based either on color hue (Lab a and b dimension) or lightness (Lab L dimension). Hue calculation is kind of magic, the main idea is to clearly separate bluish colors and assign them ocean (< 0.2) height. After image loading user is able to select one of these two types of auto-assignment or assign the colors manually.

Let’s try the auto-assignment on some examples. Heightmap of Tamriel, original on the left and auto-processed by lightness on the right:

As you can see the uploaded files are getting stretched to 16:9 aspect ratio. To get rid of the stretching you need to change the image width/height before uploading. Another example shows the auto-assignment by lightness for the heightmap of Ireland with 16:9 aspect ratio:

Looks quite good. Interpretation by hue works not so consistently, but still acceptable:

Most fantasy RPG maps contain unwanted noise such as labels, icons and so on. While it’s quite difficult to process a created RPG map, usually system is able go get at least continents contours. For example here is processed Map of Gnosis by Maxime Plasse:

Map of Gnosis (small)
Map of Gnosis by Maxime Plasse

Map of Gnosis processed.png

That’s why you have to clean the map before uploading and there should be an ability to fine-tune the processed maps. I also want to add that “quality” of the output is highly depends on the Voronoi graph size. Images above are made on a default graph size with about 8k polygons. The same map with ~70k polygons:

Map of Gnosis processed 70k.png

The manual processing works different. The problem is that usual image has too much colors to be manually associated with a height value. I can imagine users will select an appropriate height for 10-20 colors, but not for hundreds of them. So we need to reduce the number of used colors without significant quality loss. Thankfully there are algorithms doing exactly this thing, known as color quantization.

Quantization algorithms are not so easy and I’m not going to describe them. The one I’m using now is modified median cut quantization (MMQC) algorithm porter to JavaScript by Nick Rabinowitz and available as quantize.js. There are some bugs and I cannot say it fits my need ideally, but it’s small and quite fast. All we need is to provide a list of colors as an input. The output is desired number of quantized colors.

We can get a list of colors directly from the uploaded image, but it has no much sense as finally the map will be rendered as a set of polygons and collecting colors from all image points is a waste of resources. So we need to select just one color for each polygon, add all these colors to the array, remove the duplicates and pass the array to the quantization algorithm. As I said before I find reasonable to have about 10-20 colors, so let it be 12 as a default number. Even a complex not-a-map image is still recognizable being rendered with 12 colors only:

Cheetah (wiki image processed with 12 colors)
A processed photo of Cheetah, 12 colors (source author: schani, licensed under CC)

As the system doesn’t know what king of image will be loaded, a good solution is to draw an image in a “stained-glass” style with reduced number of colors and let user decide whether it’s feasible to apply an auto-assignment function or distribute the colors manually. There is also an ability to change the colors number as there are some cases where significant details are getting lost with 12 colors only, meanwhile some images may have 2-5 colors only.

I have tried to make a UI as easy as I can imagine. Let’s take a look on example map: a fantasy combination of Orkney and Sky islands (topographic maps are taken from Wikipedia):

Image to Heightmap converter UI
Image to Heightmap converter UI

On upload the image is getting rendered in a “stained-glass” style with 12 colors, listed in the “Unassigned colors” section. You can select a color and click on the scale above to assign a height value, or you may click on the buttons above to auto-assign all colors based on lightness or hue. Look at the same map with manually assigned colors (left), colors auto-assigned by lightness (center) and by hue (right):

 

Auto-assignment does not prevent you from manual colors distribution, so the best idea is to use an automatic function and them manually fix the assignment if needed. To simplify the manual assignment the source image can be displayed as semi-transparent overlay. Once assignment is done you can manually fine tune the heightmap or complete it in one click:

Orkney Skye completed
Political map of Orkney-Skye Islands

That’s pretty much all for today. Comparing generated random maps with maps converted from real-world data (like the “drowned Britain” map in the post header) I cannot stop thinking all my efforts with random shapes were in vain. Even UI is still poor I consider Image converter is one one the most useful tools I ever made and hope you will enjoy it.

Other changes made as a part of today’s update (v. 0.52b) are briefly listed in the changelog. Feel free to comment and report new bugs.

18 thoughts on “Image to Heightmap converter

  1. I can’t believe I’m only now coming across this tool. I was following something very similar a while back (it’s disappeared but for this fork: https://github.com/matneyx/Fountain ) and was always dismayed by the apparent lack of just this sort of functionality in planning. I was wondering, though, whether you intend to add biomes in some manner akin to this: http://www-cs-students.stanford.edu/~amitp/game-programming/polygon-map-generation/#biomes

    Like

  2. Oh, excellent! Then the only thing I’m after that I guess you don’t yet have is global map projections, possibly as a means to wrap a globe like in this tool: https://www.maptoglobe.com

    I’m looking forward to seeing where you go with this.

    Like

    1. Not sure a lot of people need it, and it’s really challenging to implement. The linked tool is very cool and I assume author spent months to make it. You may just download Generated fantasy map and use the image in maptoglobe. Looks good

      Like

      1. I’ve seen some demand for it while researching it for myself, but I’m sure it is rather difficult. I’ve got a battery of tools to work around the problem and have found a projection that I can fake well enough for my own work (one that I can edit and then project out to equirectangular for global wrapping). This is the primary tool I’ve used for that: https://github.com/matthewarcus/mmps

        Honestly though, the ability to switch between working with a handful of map projections would be the bees knees. Seems to me its just about the holy grail of cartography for worldbuilding to literally be able to create a world (i.e., a planet).

        Like

  3. Just wanted to say that your work is really awesome, and I’m hoping you’ll continue working on it in the future!

    I was wondering, have you tried messing around with tectonic plates to create the terrain and temperature/wind patterns to shape it?

    Like

    1. Hi. Thanks for the feedback! I’ve tried, but it’s too hard to make interesting shapes via a plausible tectonic simulation. As of wind, I’m planning to add a customizable wind simulation based on boids model. It can take some time and it’s not a priority task.

      Like

      1. After writing my comment I started messing around with tectonic plates (I actually didn’t receive a notification for you reply, so I’ve just seen it now!) and I agree completely with you, I couldn’t achieve any good looking shapes at all! I honestly thought that it was worth the time to get more “scientific”, but in the end realism isn’t always interesting.

        I’m going to try now your approach, which even though it’s simple on paper looks always good!

        Again thank you for you blog posts, they’re really helpful! 🙂

        PS: actually I’ve got another question. I read in another comment in this post that you’re planning on trying 3D maps. Since your map are in scale (i.e. in the standard size 1 pixel is equal to 3 km) are you planning to actually build the terrain for each tile, or just for visualizing the same 2D map but just in 3D?

        Like

  4. I;m missing it, but where can I upload my image to be converted to a heightmap? I can do the image processing like you described, but where do I load it?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s