IncGamers puts AMD’s Mantle to the test with the Star Swarm benchmark

With the release of AMD’s Catalyst 14.1 beta drivers, the Mantle application programming interface (API) is out there and available for testing. That’s exactly what I’ll be delving into in a second, but first let’s address what the heck this Mantle business actually is.

This stuff is a minefield of acronyms and technical detail. APIs, GPUs; you could be forgiven for thinking that Terraflops are what happen when an Al-Qaeda bomb plot goes wrong. But when you break it down, it’s not too difficult to follow. Probably.

An API is a standardised set of instructions which programmers use to interact with software or hardware. When we’re talking about graphics card APIs, the big duo are DirectX and OpenGL. Both of these try to standardise a set of features that work across all graphics cards. It’s up to the manufacturers (AMD and Nvidia) to produce cards that can support these features and be compatible with, say, DirectX 11.

That’s all fine and dandy, but what if one of the manufacturers designed an API that would take advantage of one specific type of graphics card architecture? This should allow for better optimisation and performance, because the API is created for your graphics card and not some wider abstraction of all GPUs.

If DirectX is Esperanto, the handy international language that lets you communicate with everybody in a halting fashion, AMD’s Mantle instead promises fluency in the language of a specific country. The intent is a more nuanced conversation and … err … smoother particle effects? Sorry, I think this metaphor went a bit off course somewhere.

Anyway, thanks to that bumbling layman’s explanation you now know why Mantle should, in theory, be able to coax better performance from any AMD cards which use the “Graphics Core Next” (GCN) structure. That’s any from the R7 series, R9 series and HD 8000/7000 line of GPUs. If you don’t have one of those, Mantle won’t do anything for you.

Right now, there are only a couple of ways to test Mantle side by side with DirectX. Battlefield 4 just added support for it, but that’s tricky for me to look at because I don’t own that game. It’s also a title that has stability problems of its own, so maybe not the best option for attempted benchmarking.

The other choice is Oxide Games’ Star Swarm freebie stress test tool, which throws upwards of 5,000 spaceships into combat and then laughs at your frame-rate. That’s the one I’ll be using to compare DirectX and Mantle results, and this is the system I’ll be using to do so:

• i3-2100 CPU at 3.10 GHz
• Asus 2GB HD 7870
• 8 GB of RAM

A couple of notes about that set-up. AMD says it’s still optimising Mantle performance for the HD 7000 line, and the drivers are beta to begin with so it’s quite likely that whatever figures Star Swarm throws out will only improve with time. In addition, Mantle supposedly does a decent job compensating for slower CPUs by reducing load on them, so hopefully that effect will be noticeable with my box.

For both of these tests, the Star Swarm stress test was run in ‘Follow’ mode, with Extreme settings at 1980×1080. Follow is recommended by the developers as the mode that’ll produce the most consistent numbers.

Here come some of those numbers. Are you excited? I know I am.

DirectX mode:

Test Duration: 360 Seconds
Total Frames: 9955

Average FPS: 27.65
Average Unit Count: 3926
Maximum Unit Count: 5498

At times, the FPS dipped as low as 6.0; but it’s also worth pointing out that the simple act of disabling motion blur (achieved by changing EnableMotionBlur=1 to EnableMotionBlur=0 in the config files) raised the average FPS by significant amounts, to 49.09. Without motion blur the dips were also less severe, closer to 15.00 than 6.0.

Mantle mode:

Test Duration: 360 Seconds
Total Frames: 12709

Average FPS: 35.30
Average Unit Count: 4155
Maximum Unit Count: 5507

An improved average FPS and, interestingly, even with the motion blur effect enabled the frame-rate never dipped below 18.00. Not too desirable for any actual gaming, but a marked improvement over the slideshow DirectX mode was offering.

With motion blur disabled, Mantle managed an average of 53.78. So again, an improvement, albeit a less significant one.

I’m not overwhelmed by those results, but they are a pleasant surprise. Whether you’re comparing the motion blur or non motion blur figures, Mantle squeezed out another 5-8 FPS. That’s not a number which will change my gaming world, but it’s an improvement I’m always going to take if it’s on offer.

As a first outing for AMD’s system, beta drivers and all, it’s not too shabby.

From this point, Mantle’s success and viability are going to depend on the number of developers supporting it in their games. Thief will be the first title to launch with compatibility for the API, all future Frostbite engine titles should support it, and others like Star Citizen are said to be lining up to include it as well; but it’s difficult to predict how many other studios will embrace the system. 3dfx had their own API called Glide back in the 90s, but it fell out of favour when DirectX rose to prominence.

The frame-rate boosts are encouraging, but I worry that studios may lack the resources and desire to code simultaneously for DirectX and Mantle. In order to thrive, Mantle will need that consistent support.

---