39

YouTube:https://www.youtube.com/watch?v=Y6uI4p0emiM
Text:
Good to meet you all. Mark Nossokoff, I'm a research director with Hyperion.

Hyperion Research, if you're not familiar with Hyperion Research, we are a market analyst industry, market researcher industry analyst consultancy focusing on the HPC market. And it's HPC and it's pretty broadly defined because it also covers cloud, HPC resources in the cloud, AI, quantum computing, and related technologies that either HPC enables or that inform HPC investments and architectures. We also host and produce the HPC User Forums, which are events and workshops that bring together a pretty broad cross-section of users to facilitate and promote the utilization of HPC across the industry.

I'll fast-forward through a few things. We also do a lot of custom research for clients, for users, and vendors as well. And it's a function largely and a lot of times, how are HPC architectures evolving and what do people want to buy and what should they be looking at and how do they measure and compare between procurements. So it's this balance of all these entities within the architectures coming together for a convergence and a complex intersection for these things.

So I'm going to touch on a little bit, just a few numbers from a business perspective, a market science perspective within the HPC space, and then I'll drill down a little bit further into the disaggregation and composability and where and how that could impact the HPC markets.

Overall, in 2022, there was roughly $37 billion spent on HPC resources. That includes both on-premises infrastructure as well as resources in the cloud. Then within the on-premise market, roughly 40% of this overall market was on servers and the technology there and then storage was another 17%.

And all of this will then factor back in a little bit to composability and disaggregation and how we segment the market. In this case, it's our competitive segments across the entry-level workshop space, workgroup space, excuse me, all the way up to the leadership class supercomputers, ranging from a billion, one and a half billion in the workgroup space, which is relatively flat, maybe slight growth or a little flat.

But the supercomputer space is where we're seeing all the growth that aggregates to a total growth of almost 7% across the entire HPC market.

And feel free, if you have any questions, I'm going to fast-forward through a little bit. We also, as we track this, we talk about the vertical markets. We have all this segmentation and data by manufacturing and bioscience, life sciences, and media and entertainment, if you will.

Beyond just the business numbers, we also do an annual site survey in terms of technology as folks are adopting and implementing these in their infrastructure HPC systems. So, one of the things we look at is the adoption of accelerators and GPUs. And it probably doesn't come as a surprise that the growth and adoption of CPUs is occurring. I think these adoption rates, though, I think are very aggressive but very telling. And just in the last year, it went from almost 83% all the way up to 94% of sites have adopted GPUs in their systems. And then as you look at the scale of how they're being adopted, it's largely--there are some very, very large systems, but the higher percentage ones as far as adoption are in the 100 to 500 GPU range, which at being a GPUs being one of the elements that can be disaggregated and then composed, there's opportunity here in the space.

When we look at the type of accelerators that are being purchased and procured and integrated into the systems, as far as accelerators, as a general class, GPUs are far and away the largest type of accelerators. There's additional ones, the TPUs, FPGAs, and then maybe some we're seeing some smattering of adoption of single-purpose AI-type processors as well.

Then overall, the high growth rate that I mentioned relative to GPUs and accelerators across the market.

From another perspective, looking at the size of the memories that are incorporated in these HPC class systems, we're seeing an average amount across the systems at these sites that we survey, which is spread across academic, industry, and government, that roughly an average of just over a petabyte of memory for their largest systems that are being deployed, which then these large systems drive so much memory in their systems that the median is even quite a bit higher than that.

What's also kind of of interest, too, I think along these lines, usually people want more for a lot less. We also want to identify what people might be willing to spend more on. One of the top two items that they're willing to pay a 10 to 15 percent premium for are faster, higher performance processors and larger, faster memory. One is as these systems and vendors can be enabled to purchase these things, there's opportunity there, as well as how to make these resources more efficient in the systems that they adopt. I think this all plays into a bit where some composability can come into play.

Going forward, we make a series of predictions.

I won't go through all of them, but there are two that really are driving and supporting composability and the application adoption of CXL, and that's in the notion of how the architectures are changing.

We expect growth, and we're showing growth, but the growth won't necessarily be in these very large. The dollars will still be spent in some of these larger systems, but we're seeing a trend more towards smaller, more focused types of systems, or even in the largest systems, how they can be partitioned for various specific workloads, as well as being able to configure between the different types of workloads. As these systems are being adopted and the traditional modeling and simulation and the more modern AI-oriented workloads drive different requirements, and being able to dynamically configure the systems depending on the by workload is of interest.

From an interconnect perspective, we track storage and interconnects as well, both at the inter-node InfiniBand Ethernet kind of area, but also down inside the systems, and obviously that's where PCIe reigns and then where CXL is emerging to help address some of these configuration and capabilities.

Maybe a little tangentially related, we did some recent research around optical I/O and how this could be adopted, and is the HPC community ready for it and wanting of it? The short answer to that is yes, from both users and vendors. There's a strong sentiment that optical I/O is one method, probably the highest most anticipated method to be able to address some of the performance and power related problems. Also as part of this study, related to the optical I/O, the notion and the need, expectation for disaggregation, and being able to do that reliably and consistently by workload-driven architectures and infrastructure is also kind of a pent-up demand from these sites.

Moving forward, some of these may also be a little maybe not quite as technical as you're used to going into this, but more from a market user requirement perspective. But as the workflows within the HPC space are configured today, a user comes in and they want to run a job. It goes into the scheduler. The scheduler identifies what's available and when the resources can be available, the compute, the accelerator, the memory, the storage, et cetera. But they're locked in fairly fixed, not too flexible configurations. And those types of--as those resources get reserved, many times elements of what gets reserved aren't going to be used for a particular workload, so they kind of get wasted and stranded, if you will.

So the notion of compatibility, composability, excuse me, and disaggregation really will help provide the more accelerated time to completion, which is what really counts with the researchers, engineers, and scientists that are utilizing the HPC infrastructure.

So increase the system utilization, getting their jobs done faster. The job not sitting in the queue as long, waiting for a specific type of resource, and then the system administrators can see the benefits relative to reduced system costs if they can buy initial ones in smaller chunks and being able to more modularly expand as well as the system demands warrant.

Some things that we're seeing as unknown in some of the research as we talk to folks about disaggregation is there's some concern about resource impacts. What can--will the codes need to be changed and adopted--adapted to to meet these new architectures? Will there be performance impacts? And maybe not so much--there is a notion of how much latency is required as we're reconfiguring and dynamically changing the configurations, but also how much will it scale? There's questions about--as people are thinking about this and looking at it--is it just within a rack? How far--how many racks, how far within a data center could disaggregation and composability be able to extend and for--up to what type of system size? Are there additional costs relative to another network? There's a perception of that.

And probably one of the biggest areas is really trying to understand, almost similarly as cloud has been adopted in the HPC space, people want to know, you know, what workloads, what types of things should be migrated to the cloud, what types of workloads and items should stay on-premises. Similarly, a notion of, well, what is appropriate from an HPC perspective to--that could be disaggregated and composability. So the, you know, the areas that our folks are looking at trying to understand the utilization for where could composability best help, some of the conditions that are amenable to the disaggregation composable systems from a utilization perspective, you know, overall systems now that are maybe not very highly utilized can fragment and compose the storage, separate them out and allow those resources that aren't being used to be made available elsewhere. If there's mismatched resources between what's there and what's required, being able to expose those as well. The scale of the system, I mentioned it. How far can it scale beyond on the performance? Can the performance be predictable as the systems and resources are being gathered and defined and assigned to the workloads? Can the workloads, when they're being--are being spawned, can they in turn specify the quality of service from a performance perspective that then identifies the resources that are going to be demanded? Lastly, on the--well, then the workloads themselves, are they the highly parallel ones or are less so that are best adapted and can afford the composability? And is there a talent and, you know, skill set that composability can actually help where maybe the users don't need a specifier, don't need to know the type of system that their workload needs, that the ability of the dynamic relocation of the resources takes care of it for them. So all of these are areas that I think as solutions are being developed for composability and CXL-based systems, as these types of things can be defined and quantified from an end user, there's how they can really leverage and benefit from composability are areas to focus solutions on.

One of the areas that, you know, that's very—and I'm preaching to the choir here, that's why you guys are all here—was the really broad and pretty strong ecosystem within the composability space, all the way from the standards through all the elements within the system. It is important to have an interoperable, reliable roadmap of items and put together the solutions from multiple sources for this particular architecture to be broadly adopted.

And I'll actually just wrap up the discussion here with - we are starting our own additional research within the composability space, both broadly available as well as the ability to do any custom research or market understanding for where these can apply or where to maybe target CXL-based composable systems. That is certainly available for what we can do and provide.

I'll conclude there with, you know, thank you for inviting me to share the market, kind of any market perspective, how things are considered specifically within the HPC space relative to composability and disaggregation. You know, any questions, feel free to follow back up or I'll--