Futuremark's Latest Attempt: 3DMark06 Tested

3DMark06

3DMark is a program that has been around for a while now and Futuremark has just released the newest edition. 3DMark06 has some new features, but it's essentially the same reliable tool that it has been in the past. The benchmarking demos have been updated graphically and look very impressive, and interestingly, there is a playable game included in the program this time around.

This version of 3DMark adds some graphical enhancements to the three demos from 3DMark05, and adds a new demo of an impressively rendered arctic outpost. The other three demos are: "proxycon", a futuristic shoot-out scene; "firefly", a night scene depicting two fireflies in a forest; and "canyonflight", which shows an airship (very reminiscent of the one in The Mummy Returns) encountering a huge sea serpent.







The images above are comparisons between 3DMark05 and 3DMark06, and show the improved SM 2.0, HDR and SM 3.0 enhancements to the demos. The High Dynamic Range additions are particularly impressive, and the new shadow effects in each of the demos look very nice as well.

Of course, Futuremark does a great deal of research when deciding how to implement a feature. Unfortunately, no one can predict what all other game developers will end up doing (let alone the way in which they will go about doing it). The HDR implementation, for example, is based on a ground-up approach with full FP16 render targets. This allows them to render reflections and refraction of HDR light sources with bloom, lens flare, and all those great HDR effects that we've come to know and love. Tone-mapping is applied at the end as a post processing step to render the floating point HDR framebuffer data out to an integer display. While all of this is fine, there are some issues with the approach. First, unless some form of supersample AA is used, only ATI's hardware can perform multisample FSAA on an FP16 render target. For this reason, many game developers have opted to avoid such an approach. Also, while both NVIDIA and ATI hardware can do floating point blends, only NVIDIA hardware can perform hardware filtering on floating point render targets.

On top of that, one of the most effective real world HDR implementations that we've seen so far has relied on a dynamic exposure rather than floating point precision (Valve's Source HDR). There is some overhead involved, but the effect is quite good, while allowing full filtering and FSAA on all hardware without the need for custom shader programs to reinvent the wheel. Arguably, 3DMark06 might show a picture of performance on current hardware after game developers no longer care about making all the features work across the board on this generation of GPU, but this is a bit of a stretch and its likely that much more will have changed by that point.

The game that is included is another addition to 3DMark06, but is so poor that it almost couldn't be called an actual game. It is basically a robot shooter game set in a rocky landscape, which admittedly is well-rendered, where you have to shoot little flying robots that zip around and are frustratingly hard to hit. The movement and controls are incredibly frustrating and the game is so boring and confusing that it doesn't really warrant any playing time at all, and it seems that it was only included as some kind of afterthought or proof of concept.

One thing that we want to touch on is the fact that there is some controversy over 3DMark, specifically whether or not it is best suited for testing performance between different types of graphics cards. We at Anandtech don't typically use 3DMark in our graphics card performance tests because we feel that it is not the best measure of real-world performance. While it does give an accurate depiction of the capabilities of a given card, it stresses the cards in ways that no games really do right now, in an attempt to predict what future games may implement. Because of the fact that video cards are ultimately for playing games, it can be argued that a consumer would have a much better idea of what card to buy for their gaming setup by seeing game benchmark results over 3DMark's results. This is our philosophy, and for comparing graphics hardware, we will rely on real world tests over synthetic benchmarks.

However, all this aside, 3DMark06 is a remarkable program in its own right. Feature analysis, stress testing, and image quality comparisons are all useful applications of 3DMark06. For quite some time, we have used 3DMark in system level tests as well. But that's enough on the software. Let's look at the kind of performance that we see with it.