From 2b4de506f2ed30042277d52e90dfdaa158a52523 Mon Sep 17 00:00:00 2001 From: Michael Axtmann Date: Mon, 26 Aug 2024 16:03:31 +0000 Subject: [PATCH] topo: Avoid grouping of multiple NICs to trainium accelerator Since a TRN accelerator is composed of multiple cores, the number of trainium accelerators does not necessarily reflect the number of NIC devices that the RDMA protocol should expose to the user. Instead, each core should have a NIC accessible for communication if that many NICs are available. The best approach, for now, is to remove trainium accelerators from the list of accelerators around which NICs are grouped. Consequently, each libfabric NIC is exposed as on NIC device to the user. This provides trainium maximal freedom in routing data over NICs. In the long run, a better solution might be to expose the number of actual cores to the plugin and take that number into account while NIC grouping. Signed-off-by: Michael Axtmann --- src/nccl_ofi_topo.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/src/nccl_ofi_topo.c b/src/nccl_ofi_topo.c index 9c9d48ed8..b73796471 100644 --- a/src/nccl_ofi_topo.c +++ b/src/nccl_ofi_topo.c @@ -17,13 +17,8 @@ #include "nccl_ofi_math.h" #include "nccl_ofi_ofiutils.h" -#if HAVE_CUDA static const uint8_t target_class_id = 0x03; /* Display controller class */ static const unsigned short target_vendor_id = 0x10de; /* NVIDIA */ -#else -static const uint8_t target_class_id = 0x08; /* System peripheral */ -static const unsigned short target_vendor_id = 0x1d0f; /* Amazon */ -#endif /* Maximum length of the device property read from file by function * get_device_property() */