cupy and cupy-core are linked against cudatoolkit (and CUDA 11.8), disregarding of any installed cuda version > 11.8

[nvandamme]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

Seems that the last version of cupy (py311/win64) on conda-forge has cudatoolkit (11.8) dep and force cupy to use an older cuda runtime, disregarding any cuda runtime already installed (in my case, nvidia::cuda-runtime 12.4 with system CUDA 12.6 update 2).

To ensure cupy is compiling RawKernels correctly (on win11 24h2, CUDA 12.6 SDK), i had to downgrade cupy by removing conda-forge's cudatoolkit (11.8).
cupy 13.3.0-py311h0508009_2 --> 13.3.0-py311hd45bd00_2
cupy-core 13.3.0-py311h83aa944_2 --> 13.3.0-py311h54f43e2_2

cupy/cupy#8734 (comment)

Installed packages

https://github.com/cupy/cupy/issues/8734#issue-2649275811

Environment info

https://github.com/cupy/cupy/issues/8734#issue-2649275811

[jakirkham]

Can you please share a conda create command to generate a fresh environment with this issue?

[nvandamme]

Sure, here you go:

env.yml

channels:
  - pytorch
  - nvidia
  - conda-forge
  - nodefaults
dependencies:
  - python=3.11.10
  - structlog
  - pycocotools
  - tensorboard
  - seaborn
  - rich
  - colorama
  - meshio
  - nptyping
  - numba
  - cupy
  - mypy
  - orjson
  - pandas
  - numpy
  - opencv
  - psutil
  - ipython
  - requests
  - scipy
  - scikit-learn
  - pytorch::pytorch=2.5.1
  - pytorch::torchvision
  - pytorch::torchaudio
  - pytorch::pytorch-cuda=12.4
  - intel-openmp # for pytorch, windows only, fix for "OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized." / fix for fbgemm.dll not initialized
  - tqdm
  - jupyterlab
  - laszip
  - matplotlib
  - pcl
  - pdal
  - python-pdal
  - pyproj
  - tbb
  - tbb-devel
  - pydantic
  - returns
  - pyarrow
  - hdbscan
  - pyjwt
  - openpyxl
  - ultralytics
  - dill
  - onnx
  - onnxruntime
  - rustimport
  - cython
  - polars
  - pip
  - pip:
    - wheel
    - nvidia-pyindex
    - open3d
    - rigid-body-motion
    - more_itertools
    - sklearn-stub

[jakirkham]

Can you please share a reproducer that uses conda-forge alone?

[nvandamme]

No can do, pytorch is essential on my side and requier pytorch and nvidia packages repos for the cuda enabled version.

[leofang]

Mixing conda-forge with other channels has never been supported. This is a general, well-established policy irrelevant of CUDA, PyTorch, CuPy, or any conda packages. Now that

1. PyTorch drops its official channel and recommends conda-forge as the substitute ([Announcement] Deprecating PyTorch’s official Anaconda channel pytorch/pytorch#138506), and
2. the nvidia channel aligns its CUDA packages with those of conda-forge starting CUDA 12.5

you should really start giving it a shot using everything only from conda-forge and see how it goes. If there's still any issue, please report to the respective feedstocks so that the maintainers can help address.

[nvandamme]

@leofang

Just a quick check on my side, the migration to conda-forge only repo is not feaseable yet as both pytorch and nvidia hasn't published conda variants on conda-forge, forcing using pytorch and nvidia repos in the meantime.

Glad this whole repo situation is being addressed tough for pytorch, thanks for your help.

[hmaarrfk]

Conda-forge is a community led effort.

Yes it is true we still don't have pytorch on windows.

See conda-forge/pytorch-cpu-feedstock#268 (comment)
conda-forge/pytorch-cpu-feedstock#231