No module named "_C"

[Songyihu]

Thanks for sharing your great work! But I got a "No module issue" when i run your Code. It occurs in deform.py ,
line 8:
"import dyhead._C as _C".

[maheswarantp]

u might want to cd to build/lib.linux-x86_64-3.7 and then try again

[airytai]

Please first build the project at the parent directory of DynamicHead with
python -m pip install -e DynamicHead

[sourabh-patil]

@airytai I did not miss that step. But while doing that I got following error.

"error: command '/usr/local/cuda-11.0/bin/nvcc' failed with exit status 1"

Is it due version mismatch of local CUDA and CUDA version of my pytorch?

[maheswarantp]

@sourabh-patil if u built torch in a conda env, it doesnt quite matter what ur CUDA version is on local.. conda installs the CUDA toolkit and cudnn librarires when u install pytorch along with the torch libraries

[sourabh-patil]

@coding-geek1711 Thank you for your reply. I do not use conda env. Following are my environment details.

PyTorch version: 1.8.0+cu101
Is debug build: False
CUDA used to build PyTorch: 10.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.25
Python version: 3.6.9 (default, Dec 8 2021, 21:08:43) [GCC 8.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-91-generic-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 10.1.105
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti
GPU 2: GeForce RTX 2080 Ti
GPU 3: GeForce RTX 2080 Ti

Nvidia driver version: 460.91.03
cuDNN version: /usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.8.0+cu101
[pip3] torchvision==0.9.0+cu101
[conda] Could not collect

This time I matched the version of cuda used to build pytorch with cuda runtime version (which is 10.1). This time I also installed ninja (which is mentioned while building detectron2) and tried with and without it. After this I Now I got the following error:

WITH NINJA:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/media/HDD8TB/sourabh/DynamicHead/setup.py", line 66, in <module>
    cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/command/develop.py", line 114, in install_for_development
    self.run_command('build_ext')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 708, in build_extensions
    build_ext.build_extensions(self)
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 538, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1359, in _write_ninja_file_and_compile_objects
    error_prefix='Error compiling objects for extension')
  File "/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1683, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

WITHOUT NINJA:

Running setup.py develop for dyhead
Complete output from command /media/HDD8TB/sourabh/env_detect_2/bin/python -c "import setuptools, tokenize;file='/media/HDD8TB/sourabh/DynamicHead/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" develop --no-deps:
running develop
running egg_info
writing dyhead.egg-info/PKG-INFO
writing dependency_links to dyhead.egg-info/dependency_links.txt
writing top-level names to dyhead.egg-info/top_level.txt
reading manifest file 'dyhead.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'dyhead.egg-info/SOURCES.txt'
running build_ext
building 'dyhead._C' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/media
creating build/temp.linux-x86_64-3.6/media/HDD8TB
creating build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh
creating build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh/DynamicHead
creating build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh/DynamicHead/dyhead
creating build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh/DynamicHead/dyhead/csrc
creating build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/cuda
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/media/HDD8TB/sourabh/DynamicHead/dyhead/csrc -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/TH -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda-10.1/include -I/media/HDD8TB/sourabh/env_detect_2/include -I/usr/include/python3.6m -c /media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/vision.cpp -o build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
In file included from /media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/ATen/Parallel.h:140:0,
from /media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
from /media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
from /media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
from /media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/all.h:13,
from /media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/extension.h:4,
from /media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/cuda/vision.h:3,
from /media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/deform_conv.h:4,
from /media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/vision.cpp:2:
/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/ATen/ParallelOpenMP.h:83:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
#pragma omp parallel for if ((end - begin) >= grain_size)

/usr/local/cuda-10.1/bin/nvcc -DWITH_CUDA -I/media/HDD8TB/sourabh/DynamicHead/dyhead/csrc -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/TH -I/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda-10.1/include -I/media/HDD8TB/sourabh/env_detect_2/include -I/usr/include/python3.6m -c /media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/cuda/SigmoidFocalLoss_cuda.cu -o build/temp.linux-x86_64-3.6/media/HDD8TB/sourabh/DynamicHead/dyhead/csrc/cuda/SigmoidFocalLoss_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++14
/usr/include/c++/7/bits/basic_string.tcc: In instantiation of ‘static std::basic_string<_CharT, _Traits, _Alloc>::_Rep* std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_S_create(std::basic_string<_CharT, _Traits, _Alloc>::size_type, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’:
/usr/include/c++/7/bits/basic_string.tcc:578:28:   required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&, std::forward_iterator_tag) [with _FwdIterator = const char16_t*; _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’
/usr/include/c++/7/bits/basic_string.h:5042:20:   required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct_aux(_InIterator, _InIterator, const _Alloc&, std::__false_type) [with _InIterator = const char16_t*; _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’
/usr/include/c++/7/bits/basic_string.h:5063:24:   required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&) [with _InIterator = const char16_t*; _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’
/usr/include/c++/7/bits/basic_string.tcc:656:134:   required from ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’
/usr/include/c++/7/bits/basic_string.h:6688:95:   required from here
/usr/include/c++/7/bits/basic_string.tcc:1067:16: error: cannot call member function ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_set_sharable() [with _CharT = char16_t; _Traits = std::char_traits<char16_t>; _Alloc = std::allocator<char16_t>]’ without object
       __p->_M_set_sharable();
       ~~~~~~~~~^~
/usr/include/c++/7/bits/basic_string.tcc: In instantiation of ‘static std::basic_string<_CharT, _Traits, _Alloc>::_Rep* std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_S_create(std::basic_string<_CharT, _Traits, _Alloc>::size_type, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’:
/usr/include/c++/7/bits/basic_string.tcc:578:28:   required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&, std::forward_iterator_tag) [with _FwdIterator = const char32_t*; _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’
/usr/include/c++/7/bits/basic_string.h:5042:20:   required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct_aux(_InIterator, _InIterator, const _Alloc&, std::__false_type) [with _InIterator = const char32_t*; _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’
/usr/include/c++/7/bits/basic_string.h:5063:24:   required from ‘static _CharT* std::basic_string<_CharT, _Traits, _Alloc>::_S_construct(_InIterator, _InIterator, const _Alloc&) [with _InIterator = const char32_t*; _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’
/usr/include/c++/7/bits/basic_string.tcc:656:134:   required from ‘std::basic_string<_CharT, _Traits, _Alloc>::basic_string(const _CharT*, std::basic_string<_CharT, _Traits, _Alloc>::size_type, const _Alloc&) [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>; std::basic_string<_CharT, _Traits, _Alloc>::size_type = long unsigned int]’
/usr/include/c++/7/bits/basic_string.h:6693:95:   required from here
/usr/include/c++/7/bits/basic_string.tcc:1067:16: error: cannot call member function ‘void std::basic_string<_CharT, _Traits, _Alloc>::_Rep::_M_set_sharable() [with _CharT = char32_t; _Traits = std::char_traits<char32_t>; _Alloc = std::allocator<char32_t>]’ without object
/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/command/easy_install.py:159: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  EasyInstallDeprecationWarning,
/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  setuptools.SetuptoolsDeprecationWarning,
/media/HDD8TB/sourabh/env_detect_2/lib/python3.6/site-packages/torch/utils/cpp_extension.py:369: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
error: command '/usr/local/cuda-10.1/bin/nvcc' failed with exit status 1

I might be stuck at some silly mistake. I will be really grateful if you can help. Thanks!

[sourabh-patil]

It seems that I solved the issue. The problem was with torch and torchvision version. I had to roll back to torch 1.4.0 and torchvision 0.5.0. The reason might be that deformable convs need torch 1.4. It seems weird though. You need to get Detectron2 first which has mentioned to keep torch version above 1.8 and then it gave me error while installing dyhead. But after getting detectron2 with torch version 1.8, I had to roll back to torch 1.4 to install dyhead. Hope that while running train code it won't throw any errors.

[sevenandseven]

Please first build the project at the parent directory of DynamicHead with python -m pip install -e DynamicHead
hello, I did what you said, but it reports an error, how can I fix it?，thank you.

ERROR: DynamicDhead-master is not a valid editable requirement. It should either be a path to a local
project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).

[sourabh-patil]

@sevenandseven Hello, thank you for replying. I have found the bug. It is related to mismatch of nvcc version, torch cuda version and gcc version. I found out that version mismatch is a critical problem while using detectron2. Thanks to their detailed guide about possible solution to the most faced problems, I could find where exactly are version mismatches. If anyone is facing issues to run Dynamic head then go through Common Issues given on https://detectron2.readthedocs.io/en/latest/tutorials/install.html My issue was """ C++ compilation errors from NVCC / NVRTC, or "Unsupported gpu architecture" """ .So solution number 3 from that worked for me. Hope that helps other people. Thanks!

[xuchen218]

Please first build the project at the parent directory of DynamicHead with python -m pip install -e DynamicHead
hello, I did what you said, but it reports an error, how can I fix it?，thank you.

ERROR: DynamicDhead-master is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).

I have the same problem. And I tried to change it to:
python -m pip install git+git://github.com/microsoft/DynamicHead.git

But the following errors still exist.
ERROR: Command errored out with exit status 128: git clone -q git://github.com/microsoft/DynamicHead.git /tmp/pip-req-build-ude1p9yh Check the logs for full command output.

[aliwaqas333]

Please first build the project at the parent directory of DynamicHead with python -m pip install -e DynamicHead
hello, I did what you said, but it reports an error, how can I fix it?，thank you.

ERROR: DynamicDhead-master is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with svn+, git+, hg+, or bzr+).

You should run the command outside the DynamicHead folder. or run python -m pip install -e .