CMS ROOT distribution does not include cuda.pcm

[fwyzard]

The build of ROOT that we distribute with CMSSW does not seem to include cuda.pcm:

ls /cvmfs/cms.cern.ch/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/cuda.pcm
ls: cannot access '/cvmfs/cms.cern.ch/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/cuda.pcm': No such file or directory

This causes ROOT to try and generate cuda.pcm while parsing headers that include cuda_runtime.h.

For a local installation of CMSSW this actually works. As you can see from the timestamps, cuda.pcm was just generated, while the other PCM files come form the RPM installation:

$ ls -l /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/*.pcm | grep -C3 cuda
-rw-r--r--. 1 fwyzard zh   507608 Feb  4 10:18 /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/XMLParser.pcm
-rw-r--r--. 1 fwyzard zh  5075816 Feb  4 10:16 /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/_Builtin_intrinsics.pcm
-rw-r--r--. 1 fwyzard zh    38864 Feb  4 10:16 /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/_Builtin_stddef_max_align_t.pcm
-rw-r--r--. 1 fwyzard zh  4499996 Feb 19 10:08 /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/cuda.pcm
-rw-r--r--. 1 fwyzard zh  1907384 Feb  4 10:16 /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/libc.pcm
-rw-r--r--. 1 fwyzard zh 27663076 Feb  4 10:16 /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/std.pcm

For a regular installation on CVMFS or other central filesystem without write access, writing cuda.pcm fails, and ROOT complains with a long error message:

HeterogeneousCoreCUDATest_xr dictionary payload:62:10: remark: could not acquire lock file for module 'cuda': failed to create unique file /data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/cuda.pcm.lock-a210875f: Permission denied [-Rmodule-build]
#include <cuda_runtime.h>
         ^
HeterogeneousCoreCUDATest_xr dictionary payload:62:10: remark: building module 'cuda' as '/data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/cuda.pcm' [-Rmodule-build]
error: unable to open output file '/data/user/fwyzard/build_ROOT/el8_amd64_gcc12/lcg/root/6.32.09-0a945d6e24dcaabe218b38fa8292785d/lib/cuda.pcm': 'Permission denied'
HeterogeneousCoreCUDATest_xr dictionary payload:62:10: remark: finished building module 'cuda' [-Rmodule-build]
HeterogeneousCoreCUDATest_xr dictionary payload:62:10: fatal error: could not build module 'cuda'
#include <cuda_runtime.h>
 ~~~~~~~~^
----- Begin Fatal Exception 19-Feb-2025 10:20:12 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=edmtest::MissingDictionaryCUDAProducer label='producer'
   [2] Calling edm::TypeWithDict::byName()
   Additional Info:
      [a] Getting TClass for edm::Wrapper<edmtest::MissingDictionaryCUDAObject>
      [b] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class edmtest::MissingDictionaryCUDAObject with content:

#line 1 "HeterogeneousCoreCUDATest_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef CMS_MICRO_ARCH
  #define CMS_MICRO_ARCH "x86-64-v3"
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 120301
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef TBB_PREVIEW_TASK_GROUP_EXTENSIONS
  #define TBB_PREVIEW_TASK_GROUP_EXTENSIONS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_15_0_0_pre3"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_15_0_0_pre3"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <cuda_runtime.h>

#include "DataFormats/Common/interface/Wrapper.h"
#include "HeterogeneousCore/CUDATest/interface/MissingDictionaryCUDAObject.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

Rather than trying to disable the header parsing mechanism, I suspect it would be simpler to generate and include cuda.pcm in our ROOT distribution, for those architectures where CUDA is available.

[fwyzard]

assign core

[cmsbuild]

New categories assigned: core

@Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @fwyzard.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

Given that attempts to disable the header parsing seem to continue to be difficult, I'd be fine with the proposal.

[fwyzard]

Actually, also the distributions of ROOT I can download from https://root.cern/install/all_releases/, e.g. root_v6.32.10.Linux-ubuntu24.04-x86_64-gcc13.3.tar.gz, do not contain the file cuda.pcm.

[makortel]

Tagging @pcanal given the ROOT connection

[smuzaffar]

looks like root only provide https://github.com/root-project/root/blob/master/interpreter/cling/include/cling/cuda.modulemap which cling uses to build a cuda.pcm on fly. For CXXMODULE IBs we were generating pcm for few externals (including cuda). We can try to move that to default IBs and see if that helps