102

Good afternoon, everyone. My name is Wai Kiong Poon. And my colleague here is Eddy from NVIDIA and myself from Molex. So today, our title of this presentation is the Partnership to Enable Copper for the Next Generation of Artificial Computing.

So this slide over here shows the NVIDIA's DGX system. And one of these DGX system consists of eight OEM modules. And how these eight OEM modules connect to the UBB is we are using the mezzanine connector. And for each OEM module, there is a two pair of these mezzanine connectors that connect the OEM to the UBB, supplying the signal as well as the power.

So when we developed the mezzanine connector, we get feedback from customers, from NVIDIA and for other customers, as what do they like to see on these mezzanine connectors. So these are some of the feedback that we got from our customers. Of course, number one is we need a high-speed board-to-board mezzanine connector that can supply up from 112 gigabit per second, PAM-4 and beyond. And needs to be low profile, 5mm with the height, with the initial target, SMT termination, because it needs to blend well with other components on the PCB. Of course, have a minimum real estate space, shouldn't cover too much of that, take too much space. Must be mechanically robust, simple in design, and of course, is for manufacturing and must be cost-effective.

So this slide over here shows the roadmap of the mezzanine connector. So we started off with the Mirror Mezz connector with a speed of up to 112 gigabit per second. So the next one is the Mirror Mezz Pro, so up to 112, but it has improved performance, more buffer as compared to Mirror Mezz. And the latest mezzanine connector is actually our Mirror Mezz Enhanced, MME for short, that can go up to a speed of 224 gigabit per second.

So to actually design the next generation of mezzanine connector, we need to solve the problem. To have a solution, we need to understand the problem. So what are the current issues plaguing the current design and preventing it from going up to 224 gigabit per second? What is the limitation? So we need to understand all these situations. And only with understanding this issue, we can design a connector that can go up to 224 gigabit per second.

So to design this connector that can have a higher data rate, we need to have a good connector design as well as a robust assembly process. And the current manufacturing process for the high speed connector that can go up to 112 gigabit per second is done by stamping and stitching. So this will kind of limit the flexibility and there's a lot of sensitivity affecting the tuning as well as making the SI performance challenging. So we need to be able to solve this issue.

So you can see over here, there's some minor change in this dimension will actually affect the performance of the SI signal. And you can see some resonance over here, excursion over here. So very minute change in this dimension and the stability of the manufacturing will actually cause the resonance to go up.

So not only the dimension, but the way the terminals actually deflect, how close it is to the plastic, all this will also affect the SI performance. As you can see over here, the impedance actually drops. It's very sensitive to this deflection.

This slide over here also shows the sensitivity of the terminal's design as well as the deflection of this connector to this SI performance. You can see the resonator over here that fluctuates a lot.

So our solution is actually we came up with this new improved connector, what we call MME, Mirror Mass Enhance. So this can go up to 224 gigabit per second. So what we did is we basically overhauled the manufacturing design, make the design more stable, less sensitive to this kind of variation.

So you can see over here, Mirror Mass Enhance is actually depicted by the green line over here. So compared to the red line and the blue line, which depicts the Mirror Mass and Mirror Mass Pro, it has a better insertion loss, return loss, as well as the impedance compared to the Mirror Mass and Mirror Mass Pro.

So this slide over here shows the power sum near-end cross as well as the far-end crosstalk, showing better performance for Mirror Mass Enhance compared to Mirror Mass Pro as well as Mirror Mass. This I pass the stage to Eddy.

Hi, everyone. My name is Seunghyun Hwang, and I'm NVIDIA high-speed principal SI Lead. So I think NVIDIA now become a little bit more popular than what I started work before. So it is clearly known that NVIDIA GPU power many of the world's fastest supercomputer and AI system. So NVLink is the world's first, actually, NVIDIA proprietary system interconnect technology that allows high-speed interconnection to communicate between GPU to GPU or even CPU to GPU. When we first started this high-speed interconnect, actually, mainly only IBM was supporting for CPU to GPU. But now, I think some of know that we start to build CPU. So we will support NVLink for CPU to GPU connection.

So many people are probably familiar with NVIDIA DGX as well. So DGX-H100 is using the fourth generation of NVLink high-speed interconnect. And this system, similar to previous 800 generations, it contains eight GPU modules. And these eight GPUs, as you can see, are attached to one large baseboard. So we have a pretty critical connection happening between the GPU modules to the baseboard through a connector.

So almost every generation, NVIDIA able to achieve a fairly decent GPU performance improvement. On the slide here on the left, I think this is one of our marketing slides. But I honestly don't even can feel how fast this is. But it is at least twice faster. And to make it twice faster on interconnect speed while maintaining the same footprint, as you can see, we did not really change form factor from 800 to H100. And we maintain almost the same-- actually, exactly the same dimension and exactly the same GPU module size. But yet, we increase speed at least 2 to 6x for the HPC or AI or training application. So this put us a lot of SI challenge because our DGX is mainly-- it's considered as the vehicle engine. So to achieve this significant SI improvement over the next generation from 800 to H100, one of the elements that was important was the connector that WaiKiong explained in the previous session.

So we take very serious about choosing a connector that use NVLink. So one of connector we are using is a mezzanine here. So signal integrity can be simple if we define one is loss, other is crosstalk, and so there is reflection. But these three elements actually dominate the interconnect performance. So on the left side, we show just common comparison between each different vendors, crosstalk performance. So crosstalk can be improved if you make signal to signal further apart. But then if you bring that product, if we cannot put into our system, then we have no product. So it is actually fairly difficult to make a lower crosstalk in similar footprint connector design. So we have very strict restriction about the footprint for the connector vendor to designing their connector. And as you can see, different vendor more like to others, similar footprint based on their capability, the crosstalk changing. I mean, at like a Nyquist 400 gigahertz per second frequency, we see more than 20 degree difference. So this was one of the criteria we pick, choose connector vendor for our H100. And also, another factor is even though performance is really good, because we maintain pretty much the same footprint from 800 to H100, we cannot increase-- we cannot change significant on the PCB interconnect routing. So those are another critical factor that we should be able to route hundreds of differential pair in the same footprint while improving the performance.

So we work very closely with the connector vendor and to help them to achieve our target spec for the higher performance interconnect speed. So left side, it shows insertion loss of the Molex mezzanine connector response. And this looks like just straight line. It is fairly easy to draw, but to make this kind of flat response from 1 gigahertz to 50 gigahertz, it is really extremely challenging. Maybe compared to PCI connector, they are talking about 16 gigahertz. I think some people are familiar, they have resonance beyond 16 gigahertz. So this is considered as a fairly excellent SI performance for connector performance. And the middle slide shows what we call TDR response. Basically, we want flat line that's starting from 90 ohm here. And our speed goes higher. This same line can become more down, lower impedance, and more higher impedance. But as you can see, this mezzanine connector stay fairly close to 90 ohm. And that TDR response usually directly proportional to how low reflection we can expect. And that's the third plot shows return loss, where Molex and NVIDIA did a lot of coding to make it optimized, so that the return where we start is blue line, but we further optimize for as a green plot.

So an outcome of all this collaboration with connector vendor and NVIDIA, left side, you show our 800 GPU module card. And the right side, it's our H100 module card. Only difference is mainly GPU performance improving interconnect speed double. But we achieve all of this without increasing any space or any connector size increase, but mainly improving connector vendors performance and also NVIDIA internal design. So inside of this GPU module, what's happening for interconnect is the left plot shows insertion loss performance of our 800 card, shown as a red line. And you can see simply we don't want this line to be fluctuate much. And you can clearly see from 800 to H100, after 20 gigahertz, H100 improved quite a bit compared to 800. And the right side is a cross-hook relative comparison between 800 and H100. Yeah, we cannot show the number here, but it actually has dramatic reduction through the connector design and other NVIDIA design. So that's for our presentation. Thank you.