Linking CXX executable ../marian

[raihan0824]

Bug description

Error while Linking CXX

How to reproduce

make -j4

Context

[ 0%] Built target marian_version
[ 3%] Built target nccl_install
[ 15%] Built target libyaml-cpp
[ 20%] Built target sentencepiece_train-static
[ 22%] Built target SQLiteCpp
[ 25%] Built target pathie-cpp
[ 31%] Built target zlib
[ 33%] Built target faiss
[ 34%] Built target intgemm
[ 34%] Built target 3rd_party_installs
[ 50%] Built target sentencepiece-static
[ 52%] Built target spm_encode
[ 52%] Built target spm_train
[ 52%] Built target spm_decode
[ 53%] Built target spm_normalize
[ 54%] Built target spm_export_vocab
[ 60%] Built target marian_cuda
[ 96%] Built target marian
[ 99%] Built target marian_decoder
[ 99%] Built target marian_conv
[ 99%] Built target marian_scorer
[ 99%] Built target marian_vocab
[100%] Linking CXX executable ../marian
/usr/bin/ld: ../local/lib/libnccl_static.a(shm.o): undefined reference to symbol 'shm_open@@GLIBC_2.2.5'
/usr/bin/ld: /lib/x86_64-linux-gnu/librt.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/marian_train.dir/build.make:95: marian] Error 1
make[1]: *** [CMakeFiles/Makefile2:643: src/CMakeFiles/marian_train.dir/all] Error 2
make: *** [Makefile:152: all] Error 2

[NovaYear]

The error message occurs during the final stage of the compilation process, during linking. The error message is caused by the following two problems:

1. undefined reference to symbol 'shm_open@@GLIBC_2.2.5':
   This error means that the shm_open function is not defined. This function is a function used for POSIX shared memory. If this function is not defined, the compilation process will fail.

2. error adding symbols: DSO missing from command line:

This error means that the -lrt (real-time) library is missing. This library is a library used for POSIX shared memory.

To solve these problems, you can follow the steps below:

1. Add the -lrt library to define the shm_open function. This library is a library used for POSIX shared memory.
2. Edit the CMakeLists.txt file and add the -lrt library. For example:
   target_link_libraries(marian ${NCCL_LIBRARIES} -lrt)
3. Run the build process again.

Also, instead of linking the nccl library statically, it may be better to link it dynamically. This allows for easier updating of the nccl library.

For example, edit the CMakeLists.txt file and dynamically link the nccl library:
target_link_libraries(marian ${NCCL_LIBRARIES} -lrt)
instead of
target_link_libraries(marian ${NCCL_LIBRARIES_SHARED} -lrt)
After these changes, run the build process again.

OR

I compiled marin and it was successful. You can use the relevant settings as a reference. Throw away all your previous tests and library installations. Try compiling according to the guide I gave you with a clean installation from the beginning.
The errors you are encountering are probably caused by trying to compile with higher version tools. I experienced similar problems, but was later able to compile using ~2022 versions of many libraries and dependencies.
The information on this page may be useful for compiling. Take a look if you want.

#423
^^ How to compile marian on windows 10/11 with WSL step by step guide (ubuntu20.04)