283

YouTube:https://www.youtube.com/watch?v=2ymXFhM-58A
Text:
Hi, I'm Bill Gervasi from Wolley, and what I'm going to be talking about today is the imperative to bring CXL out of the data centers and onto the motherboards.

So the good news is that we, for decades, had these fabric wars.We had CCIX and we had Gen Z and all these other fabrics that were competing with one another.And that was really holding up the industry because none of them built up enough market momentum to justify the cost of building support silicon for these architectures.So when CXL came out, it finally came to the point where we could invest in these technologies and try to take these things to a mass market and get a recoup our investments and actually hopefully make a profit.Now, as a little side note, I will say that.In recent times, there have been discussions about NVLink and UALink.And after looking at this, I've come to the conclusion these are complementary to CXL, although CXL Type 2 may go away.So what I'm going to be focusing on here really is the adoption of CXL Type 3, in other words, the CXL.mem protocol over that interface.

So what is it that CXL brings and why do I think that it's a success?Well, I think what's significant as it is, is that CXL brings a unified view of all of the resources in the data center.That's memory, that's storage, that's communications, that's I.O.All of those things now are visible behind one interface.However, there's nothing that is data center specific about this.The difference between a data center on these requirements and desktops, notebooks, workstations.It's just a matter of scale.

So why do we care about moving CXL out of the data center?And this has to do with the fundamentals of the market.That the size of these other markets dwarfs the number of components that can be sold into the data center.Those of us that want to make chips, now it used to be you could build a chip for a million bucks and you could recoup that pretty easily.Now with things like 5 and  7 and 3 nanometer processes, we're talking about anywhere from $25 to $100 million to build a chip.So we need a market that has significant volume in order to recoup our investment and bring the cost down.

Here's a typical view of what DRAM generations look like.We've all seen these pictures time and again.DDR3 will come in, get replaced by DDR4 and DDR5.And these little bulges represent the adoption of them in the percentage of the market.And this is really a misleading view of how things really happen.Because there are a lot of bumps.Not just data centers, but data centers are always the first because they're less price sensitive.So they can afford to pay $100 for a controller chip if they have to.But you're not going to put a $100 controller chip.You're not going to put a $100 controller chip into a notebook.You need to have these costs come down over time to migrate from data center to desktop, maybe two, two and a half years later.To notebooks, maybe a year and a year and a half after the desktops.And when it's the cash in a solid state drive, that might be four or five years after the introduction of the technology.So this transition is really an important thing.And what I'm.Arguing is that CXL adoption is going to follow this same path.

Well, so let's see the problems that we're going to have trying to make this happen.The bad news is that today's CXL solutions are indeed targeted at the data centers.And so they're not even trying to be particularly power sensitive.So the good news is that climate change is happening.And people are rethinking.Bringing nuclear power plants online to support CXL, right?

No, that's not how things are going.Instead, the Department of Energy has this program called EES2 that I've personally been a part of, where we project that if you look at that orange line at the top there, the capacity of the planet Earth is quickly being overtaken by all of our electronic gadgets.You take the data centers.You take 5G moving to 6G networks.You take self-driving cars, all this other stuff that's coming online here.And if you project those out, we're going to run out of energy around 2055, which I think we all agree is a pretty bad thing.So you see those dotted green lines.This is a program that the DOE is putting in place to try to increase the efficiency of data centers.And in order to do that, you have to figure out.Where you're spending your energy.And that's what this second drive, second picture shows here, where you see that kind of flat space in the middle there, that's all your standard processing.That's your HTTP, that's your databases, that's your standard transactions.And then over on the far right, you see those four lines that are completely off scale.And by off scale, I mean, literally these are logarithmic scales.So we're looking at.We're looking at a million times more power being consumed to do artificial intelligence and cryptocurrency.So this is flying in the face of the fact that we need to make things more efficient.

So.Looking at that picture and plotting how CXL is going to move out of the data center.The first thing we have to do is address the power consumption of the CXL solutions.Also, these big add in cards are not going to fit into these smaller machines.They're not good, friendly for a desktop.They're certainly not going to fit a notebook.So we have to shrink the footprint.And then shortening the channel.This is a little bit more subtle, but this is an important part of reducing the power right now.These things are designed for driving a meters worth of cable.And so as a result.The power to drive the the line are very high. We need to bring the whole power envelope down.

There's something that I proposed at the PCI SIG, and that we're in the process of analyzing, doing signal integrity checks on. But fundamentally, what I proposed is that we take the M.2 standard. And if you're familiar with the M.2, it's that little SSD that we all have in our notebooks. The M.2 had some problems, though. For example, one of the things that we did was we interleaved the transmit and receive signals, which increased crosstalk, made it maybe impossible for them to ever hit PCIe Gen 6. So one of the things we can do is take advantage of this to fix the limitations of M.2, put all the transmits on one side, receives on the other, stack.So that we can put good grounding into the channels. With that, we have something that now is a very small form factor module. It only grows the width of the module by about three millimeters. And so we can still fit it in between two adjacent slots on the micro ATX motherboard. So things are starting to look really good here. We can bring the channel length down to about 150 millimeters, which helps us keep the entire envelope of this memory module in under 11 watts. And then, just like M.2, we probably want different lengths, depending on what solution we're putting in place. And so the initial proposal there is basically the same that we have in the M.2 world, which is 30, 60, and 80 millimeter lengths.

And this is kind of what that looks like. So this picture here shows the three footprints.Shows that socket. And then, up at the top right there, I included a picture of a M.2 module. So that's 22 millimeters. So you add about three millimeters to that next to a DDR SO-DIMM. And you get a feel for about how big these new modules are going to be.They're roughly about the same square acreage as an SO-DIMM in the longest form factor. And, significantly smaller in the shorter form factors.Now, one thing that I will point out here for the nerds in the audience is that we're probably going to take the PCB a little thicker. The 0.8 millimeters of the M.2 does not allow routing of a lot of memory channels, which is something we're going to want to enable.So odds are we're going to move this to the 1.37 millimeter thickness that SFF is proposing.

So what goes into this socket if we adopt this? Well, you know, the answer is anything you want.That's one of the beauties of CXL protocol sitting on top of PCIe.You can do unmodified PCIe. You can do CXL type one, two, or three.You can do all combinations of these things. You can do multi-headed devices. And all of these things are going to be And all of these things are going to be And all of these things are going to be supported so that you can have memory, processing, storage, communications, and any combination thereof that plugs into that socket.So this is not just for my company. This is something we want to encourage the industry to jump in on, support, and fill those sockets.

We've all lived in this world for the last 50 years where memory and storage were on separate channels and you could never cross them over.You could access memory sort of over PCIe, but the barriers were so high that it was really inefficient and so nobody could really effectively do that.So with this idea of a flex module, the flex module now blurs that line where you can have memory or storage on each channel. You can picture having a system that has just multiple sockets, a flex, and then you put in whatever you want. Now here you can imagine putting in a memory card, putting in a storage card, putting in an AI coprocessor. Why is that important? AI is on an 18-month S-curve from early adoption through saturation. That's in comparison to seven-year S-curves for other technologies. So it's really significant that now you can imagine having the upgradeability of your AI solution as the engine or the algorithms become obsolete. You also may need to increase your memory with the new AI algorithms. So all of this seems to play together well with the trends in the marketplace that are underway. 

But you can only push a rope so hard. So I wanted to thank my friend Tom Chanel at Dell Technologies who provided this view of why this is interesting.As you all know, we've had a while here with a motherboard having four sockets.That's two memory modules per channel because these CPUs have two DDR5 channels.And that worked great up until DDR5 4800.In other words, 4800 megabits per second.We have a problem, though. When DDR5 goes to 6400, all of a sudden we lose two channels.Why? Because electrically you can't run two DIMMs per channel anymore at 6400, 7200 or the faster bins that are being planned now.So this means that memory capacity is going to be cut in half in order to get that higher DDR5 speed.So, you know, take one way to look at that is that, yes, I can sell you a 64 gigabyte system that runs at 4800 megabits per channel.But if you want to go to 6400, I'm going to drop you down to 32 gigabytes.Not a very tenable marketing situation.And with these AI algorithms and chips changing so quickly, the demand for more memory is actually the dominant characteristic.So basically what we're seeing is that all of these innovations are being stalled by the fact that we have closed designs that don't have an easy expansion.You know, there's some PCI sockets on this motherboard.There's only three of them, and they really are high overhead in terms of what you can pack into them.So we should be looking towards putting six on the motherboard, just like when PCI was introduced in 1992.And what happened was that when PCI was introduced, we still were using external serial channels for mice.We didn't have sound in a motherboard.But PCI allowed sound cards, allowed network adapters, allowed all of this stuff to be introduced into the PC, innovated with PC, and then eventually became consumed into the chipsets.We're at that same point today, is my argument.And it's time now for us to introduce another standard socket into the PC architecture to allow these innovations.

One innovation that I'll point out is the stuff that my company is working on, and that's something called NVMe over CXL.And so now I'd like you to picture how this would fit in your system architecture.What NVMe over CXL does is it combines NAND and DRAM in a unique virtualized way.They appear behind the same CXL interface, but as a dual function device.One that is an NVMe drive.That controls the NAND.And the other is an HDM, or host-directed memory, interface that talks to your memory, which could be DDR, could be LPDDR, could even be MRAM.Now, one way to think of the NVMe protocol is that it is really just a cache manager.And so that's what this view is going to show you, is that you can take this virtualization of that address space, make a host memory address space that looks the size of the NAND, but then squeeze it down to a more affordable block of DRAM.So this is allowing you to, for example, present this as an 8 terabytes of memory, even though only 64 gigabytes are on the module.Then you can put your controller memory buffers for standard NVMe access through it.You can map some of it as just host-directed memory that has no equivalent image on the NAND side.You can also do things like persistent memory.Remember the NVDIMM-N?Some companies made great markets out of that.Now you can have not only NVDIMM-N functionality, but it can be part of the memory.You don't have to back up and restore the whole thing.You only get the parts that you need.And then there's the virtualization.Host-directed memory.Now you have something that looks the full size of the virtual memory space that's in the NAND, but you can map blocks in and out as needed to translate the smaller DRAM into the larger NAND.

So that's kind of it.That's the first picture here is to talk about why CXL helped resolve the fabric wars.The industry now needs to grow beyond that to help this market take off and hit the volumes that it deserves.Keep in mind that different markets phase in over time.It might be five years from the introduction of a new technology before it hits all of these marketplaces.And I predict CXL should be the same.It does need to change, though.The CXL that's good for enterprise is not the same as the CXL for desktops and mobile.And I proposed the Flex M.28 form factor for the desktop and the mobile systems.Now, CXL is great in that it allows resource virtualization.And I've shown you the ideas that we have, which is NVMe over CXL can take this virtualization and combine memory and storage in a unique form factor.But finally, it's a message to you.That this is a call for you guys to jump in with us.To jump in with me.And to help expand this marketplace with your own ideas.And let's make this thing a great market for all of us.

So thank you for your time.I'm happy to respond to questions.There's my email, my LinkedIn information, and so forth.Feel free to scan those and get in contact with me.