The Ampere Altra Max Review: Pushing it to 128 Cores per Socket
by Andrei Frumusanu on October 7, 2021 8:00 AM EST- Posted in
- Servers
- Arm
- Neoverse N1
- Ampere
- Altra Max
Conclusion & End Remarks
Our time with the new Altra Max has been interesting, as it’s very much a chip design that quite polarising and pushing some aspects of core scalability to the very extreme.
When Ampere had talked about their plans to put to market a 128-core variant of the Neoverse N1, a 60% increase in cores over their first generation 80-core attempt, we were of course perplexed on how they would achieve this, especially considering the chip is meant to be used on the very same platform with same memory resources, and also on the same fundamental technology – same core microarchitecture, same mesh IP, and same process node.
The Altra Max is a lot more dual-faced than other chips on the market. On one hand, the increase of core count to 128 cores in some cases ends up with massive performance gains that are able to leave the competition in the dust. In some cases, the M128-30 outperforms the EPYC 7763 by 45 to 88% in edge cases, let’s not mention Intel’s solutions.
On the other hand, in some workloads, the 128 cores of the M128 don’t help at all, and actually using them can result in a performance degradation compared to the Q80-33, and also notable slower than the EPYC competition.
I think what we’re seeing here is that Ampere is hyper-optimising themselves into certain workloads. The Altra Max marketing is especially focused around cloud-computing and hyperscaler deployments of the chip. Ampere’s recent announcement earlier this summer, detailing that the company is working on their own custom CPU microarchitecture with specific plans to target such workloads, and abandon the general use case Neoverse Arm CPUs, with Ampere’s description of “general use case” here being mentioned in a negative context, is telling that this is all a deliberate strategy.
What differs a cloud CPU from a regular CPU? I’ll be frank here in mentioning that I don’t have sufficient background on the matter other than to say that memory does not seem to be a focus-point of such workloads. We’re still working on expanding our test suite with more real-world distributed systems workloads to cover such scenarios. By Ampere’s wording of their announcement this summer, and by the very apparent direction of the new Mystique design performance characteristics, it seems we’ll see even greater such extremes in the future.
On the competitive landscape, Ampere is carving out its niche for the moment, but what happens once AMD or Intel increase their core counts as well? A 50% increase in core counts for next-gen Genoa should be sufficient for AMD to catch up with the M128 in raw throughput, and technologies such as V-cache should make sure the HPC segment is fully covered as well, a segment Ampere appears to have no interest in. Intel now has an extremely impressive smaller core in the form of Gracemont, and they could easily make a large-core count server chip to attack the very segment Ampere is focusing on.
Only time will tell if Ampere’s gamble on hyper-focusing on certain workloads and market segments pays out. For now, the new Altra Max is an interesting and very competent chip, but it’s certainly not for everyone.
Related Reading:
-
The Ampere Altra Review: 2x 80 Cores Arm Server Performance Monster
-
AMD EPYC Milan Review Part 2: Testing 8 to 64 Cores in a Production Platform
-
AMD 3rd Gen EPYC Milan Review: A Peak vs Per Core Performance Balance
-
Intel 3rd Gen Xeon Scalable (Ice Lake SP) Review: Generationally Big, Competitively Small
Facebook
Twitter
RSS
60 Comments
View All Comments
Jurgen B - Thursday, October 7, 2021 - link
Love your thorough article and testing. This is some serious firing power from the Ampere and makes some great competition for Intel and AMD. I really like the 256T runs on the AMD Dual socket EPYCs (they really are serving me well in floating point research computing), but it seems that future holds some nice innovations in the field!mode_13h - Thursday, October 7, 2021 - link
Lack of cache seems to be a serious liability, though. For many, it'll be a deal breaker.Wilco1 - Friday, October 8, 2021 - link
Yet it still beats AMD's 7763 with its humongous 256MB L3 in all the multithreaded benchmarks. Sure, it would be even faster if it had a 64MB L3 cache, however it doesn't appear to be a serious liability. Doing more with far less silicon at a lower price (and power) is an interesting design point (and apparently one that cloud companies asked for).Jurgen B - Friday, October 8, 2021 - link
Yes, Cache will play a role for many. However, people buying such servers likely have a very specific workload in mind. And thus they now have more choices which of the manufacturer options they prefer, and these choices are really good to see. Compared to 10 years ago, when AMD was much less competitive, it is wonderful to see the innovation.schujj07 - Friday, October 8, 2021 - link
That isn't true at all. The SPEC java benchmarks have the Epyc ahead, SpecINT Base Rate-N Estimated they are almost equal (despite having half the cores), FP Base Rate-N Estimated the Epyc is ahead, compiling the Epyc is ahead. Anything that will tax the memory subsystem by not fitting into the small cache of the Altra and the performance is lower for the Altera. Per core performance isn't even close.mode_13h - Saturday, October 9, 2021 - link
Thanks for correcting the record, @schujj07.The whole concept of adding 60% more cores while halving cache is mighty suspicious. In the most charitable view, this is intended to micro-target specific applications with low memory bandwidth requirements. From a more cynical perspective, it's merely an exercise in specsmanship and maybe trying to gin up a few specific benchmark numbers.
Wilco1 - Saturday, October 9, 2021 - link
If you're that cynical one could equally claim that adding *more* cache is mighty suspicious and gaming benchmark numbers. Obviously nobody would spend a few hundred million on a chip just to game benchmarks. The fact is there is a market for chips with lots of cores. Half the SPEC subtests show huge gains from 60% extra cores despite the lower frequency and halved L3. So clearly there are lots of applications that benefit from more cores and don't need a huge L3.Wilco1 - Saturday, October 9, 2021 - link
The Altra Max wins the more useful critical-jOPS benchmark by over 30%. It also wins the LLVM compile test and SPECINT_rate by a few percent. The 7763 only wins SPECFP by 18% (not Altra's market) and max-jOPS by 13%.So yes my point is spot on, the small cache does not look at all like a serious liability. Per-core performance isn't interesting when comparing a huge SMT core with a tiny non-SMT core - you can simply double the number of cores to make up for SMT and still use half the area...
mode_13h - Saturday, October 9, 2021 - link
> Per-core performance isn't interesting when comparing ...Trying to change the subject? We didn't mention that. We were talking only about cache.
> The Altra Max wins the more useful critical-jOPS benchmark by over 30%.
That's really about QoS, which is a different story. Surely, relevant for some. I wonder if x86 CPUs would do better on that front with SMT disabled.
> the small cache does not look at all like a serious liability.
Of course it's a liability! It's just a very workload-dependent one. You need only note the cases where Max significantly underperforms, relative to its 80-core sibling, to see where the cache reduction is likely an issue.
The reason why there are so many different benchmarks is that you can't just seize on the aggregate numbers to tell the whole story.
mode_13h - Saturday, October 9, 2021 - link
Apologies, I now see where schujj07 mentioned per-core performance. I even searched for "per-core" but not "per core".