Monthly Archives: January 2010

ASTER – Not worth it yet

A few months ago NASA caused a stir by releasing a new global height dataset called ASTER. I use an earlier dataset (SRTM) for OpenCycleMap, which has a few problems that ASTER, at least initially, promised to solve. The three of primary interest to me are:

  • Voids – SRTM has “no data” gaps in certain places of the world, where the radar reflections went haywire. These happen in marshes (not of interest) and mountains (of great interest!), especially over the Alps. ASTER is void-filled already, so the clever-but-inaccurate void-filling I use wouldn’t be necessary
  • Resolution – It’s great that SRTM covers the whole world, but I’d love to see it at a higher resolution. ASTER’s nominal resolution is three times greater than SRTM, so it’s very attractive.
  • Arctic coverage – SRTM only goes as far north as 60°N, which is a bit of a problem in Scandanavia. Although there’s GTOPO30 data for these areas, that’s got a horizontal resolution measured in kilometres, so not exactly great for me. ASTER covers those areas too, up to 83°N.

So far so good. But when I started work with ASTER in December, things spiralled rapidly downhill. First is the pointlessly irritating “order a dataset” website, that sucked up hours of going round in circles. It’s like a shopping website from 1999. You need to use a stupid interface to order which 1°x1° tiles you want, and “All” isn’t an option, despite there being 22,600 of them. It seems geared up for people who want a couple of dozen at a time, and the whole thing has a feel of being run by men with beards and sandals who’d rather you didn’t use their website in anything newer than Netscape 4 on HP-UX.

When I read the README alarm bells started ringing. There’s a section on “mole runs” and “pit artefacts” that sounded a bit worrying, but I wasn’t sure how much of an issue they’d be – if they were small and few and far between then that’s not much of a problem. But the biggest thing that caught my eye, buried on page six after pages of confirmation of how good the accuracy was, right at the end of a section as a throwaway comment, was the following statement:

Also, while the elevation postings in the ASTER GDEM are at 1 arc-second, or approximately 30 m, the detail of topographic expression resolvable in the ASTER GDEM appears to be between 100 m and 120 m.

That’s a bit of a bomb-shell – it’s saying that although it’s got a much higher nominal resolution than SRTM, it’s effective resolution is about the same – there’s not any more actual detail, just more pixels. That was almost enough to make me give up there and then, but it’s still void filled and covering more area of the planet, which would be good improvements. So I grabbed the DEM (thankfully, they’re straightforward GeoTIFFs) and got to work over Snowdon. I did some colouring and contours, and they both looked excellent and much better than what I made from SRTM. But then I tried hill-shading, and disaster!

Here’s the area around Snowdon (click through for original size):

ASTER Snowdon

and a detail of Snowdon itself:

ASTER Snowdon detail

The mole runs are everywhere – all across that image, even on the flat bits. And the pit artefacts are huge – the size of quarries, and really, really obvious. I honestly can’t use that – maybe for a “what the world would look like if it looked like the moon” project, but nothing more serious than that. And considering that SRTM has only a handful of single-pixel voids in that area, the guys making ASTER have made something that’s substantially worse than an oversampled SRTM. And considering they were even using SRTM to fill the gaps in the ASTER data, that’s a pretty poor show. I started reading around and found a few people saying similar things. And when I though about it, the “improvements” to the contours I saw could be recreated with SRTM by using gdalwarp to artificially increase the resolution (with some nice smoothing) before generating the contour lines. So I gain nothing from ASTER in the 95% of the planet that doesn’t have significant voids, and in that same 95% it’s not really usuable.

So for now, I’ve given up with ASTER. I might revisit it for the band between 60°N and 83°N, but it also says in the readme they have voids over eurasia for that area (so much for void-filled, eh?). And it would be interesting to see if someone can fill the large SRTM voids with ASTER (which sounds back to front, hey-ho), but I don’t have time for that. However, as they say in the docs, all these artefacts are happening in the boundaries where they have different numbers of original samples, so maybe a future version will have these automatically smoothed out, and it they can figure out how to stop their 15m sampling getting turned into 120m effective resolution, that would be awesome. But for now I would say it’s not worth using it.

Hill-shading on OpenCycleMap.org

It was over 18 months ago that I was originally trying to get hill-shading and hill-colouring working on OpenCycleMap (in fact, it wasn’t even called that back then, but that’s a different story). I eventually dropped the hill-shading part of it due to nasty boundary artefacts between source tiles, and due to the fact that the shading, well, didn’t look as nice as I wanted. It was all a bit grey and manky.

Hill Shade Teaser

So instead I launched just the hill-colouring in August 2008, which I was very happy with, and put hill-shading on the back burner. Time passed. Much time.

A few weeks ago, I rolled up my sleeves and got stuck in to figuring out how to do the hillshading properly. With some pointers from Matt, Mike and the OSM Wiki, I played around for a few days until I liked the end result.
Hill Shading results

Here’s a look at Snowdon before hill-shading. The colours do a good job of showing the lie of the land, but it’s a bit flat:
Snowdon Before Shading
Detail:
Snowdon Before Shading (Detail)
And what Snowdon looks like now. The shading lifts the peaks out from the map, and gives them a more solid-object feel:
Snowdon After Shading
Detail:
Snowdon After Shading (Detail)
It really helps most in complex mountains, like here in the Alps, where the contours would otherwise become a jumble and it’s hard to tell valleys from ridges. With the shading, it’s easy.
Valleys in the Alps

It’s a hard balancing act, since OpenCycleMap is first and foremost a map for cyclists, and too much hill-shading overpowers and distracts from the rest of the map. But then again, too little and it doesn’t seem worth the effort! I went for a subtle approach, where it’s enough to make the hills stand out but little enough you might not consciously notice. Unfortunately the effect is diminished in forested areas, and by dense contours, since it’s only the background height colouring that is shaded and those things start obscuring stuff.

Also, I was never really happy with the “drab grey” approach to shading – just making the shadows grey and the highlights white using alpha-blending – so I settled in the end for “hardlight” compositing. It’s a bit like the evolution of GUI buttons from Windows 3.11 (“right, top and left edges are white, other two edges are black, grey in the middle”) with those from MacOSX (“ooh, shiny”). Compare OpenCycleMap to Google Terrain and other hill-shaded maps, and I’m quite proud of the results.

If you have a map project that could do with some good-looking terrain info, then I’m available for freelance work.