You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am new to MONAI-label and would like to seek help regarding the same.
I am trying to run the segmentation model on my data.
I have uploaded my data in ImageTr folder with the labels in the subfolder "labels/final"
When I try to run the
monailabel start_server --app apps/radiology --studies dataset/ImageTr --conf models segmentation
I get an error
Using PYTHONPATH=/usr:
Extension horovod.torch has not been built: /usr/local/lib/python3.8/dist-packages/horovod/torch/mpi_lib/_mpi_lib.cpython-38-x86_64-linux-gnu.so not found
If this is not expected, reinstall Horovod with HOROVOD_WITH_PYTORCH=1 to debug the build error.
Warning! MPI libs are missing, but python applications are still avaiable.
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: version = False
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: app = /Oliver/apps/radiology
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: studies = /Oliver/PCAT/MONAI/ImageTr
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: verbose = INFO
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: conf = [['models', 'segmentation']]
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: host = 0.0.0.0
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: port = 8000
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: uvicorn_app = monailabel.app:app
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_keyfile = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_certfile = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_keyfile_password = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_ca_certs = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: workers = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: limit_concurrency = None
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: access_log = False
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: root_path =
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: log_level = info
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: log_config = None
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: dryrun = False
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: action = start_server
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:296) -
Allow Origins: ['*']
[2024-05-30 03:38:11,834] [12522] [MainThread] [INFO] (uvicorn.error:82) - Started server process [12522]
[2024-05-30 03:38:11,834] [12522] [MainThread] [INFO] (uvicorn.error:48) - Waiting for application startup.
App Init...
[2024-05-30 03:38:11,834] [12522] [MainThread] [INFO] (monailabel.interfaces.utils.app:37) - Initializing App from: /Oliver/apps/radiology; studies: /Oliver/PCAT/MONAI/ImageTr; conf: {'models': 'segmentation'}
CUDA initialization: The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org/ to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
[2024-05-30 03:38:11,856] [12522] [MainThread] [INFO] (monailabel.utils.others.class_utils:57) - Subclass for MONAILabelApp Found: <class 'main.MyApp'>
[2024-05-30 03:38:11,865] [12522] [MainThread] [ERROR] (uvicorn.error:121) - Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 732, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/usr/lib/python3.8/contextlib.py", line 171, in aenter
return await self.gen.anext()
File "/usr/local/lib/python3.8/dist-packages/monailabel/app.py", line 49, in lifespan
instance = app_instance()
File "/usr/local/lib/python3.8/dist-packages/monailabel/interfaces/utils/app.py", line 50, in app_instance
app = c(app_dir=app_dir, studies=studies, conf=conf)
File "/Oliver/apps/radiology/main.py", line 49, in init
for c in get_class_names(lib.configs, "TaskConfig"):
File "/usr/local/lib/python3.8/dist-packages/monailabel/utils/others/class_utils.py", line 164, in get_class_names
module = importlib.import_module("." + name, package=current_module_name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/Oliver/apps/radiology/lib/configs/segmentation.py", line 17, in
import lib.trainers
File "/Oliver/apps/radiology/lib/trainers/init.py", line 12, in
from .deepedit import DeepEdit
File "/Oliver/apps/radiology/lib/trainers/deepedit.py", line 45, in
from monailabel.tasks.train.basic_train import BasicTrainTask, Context
File "/usr/local/lib/python3.8/dist-packages/monailabel/tasks/train/basic_train.py", line 23, in
import ignite
File "/usr/local/lib/python3.8/dist-packages/ignite/init.py", line 2, in
import ignite.distributed
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/init.py", line 1, in
from ignite.distributed.auto import *
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/auto.py", line 11, in
from ignite.distributed import utils as idist
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/utils.py", line 7, in
from ignite.distributed.comp_models import (
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/comp_models/init.py", line 4, in
from ignite.distributed.comp_models.horovod import has_hvd_support
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/comp_models/horovod.py", line 27, in
class _HorovodDistModel(ComputationModel):
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/comp_models/horovod.py", line 160, in _HorovodDistModel
"SUM": hvd.mpi_ops.Sum,
AttributeError: module 'horovod.torch' has no attribute 'mpi_ops'
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hello all,
I am new to MONAI-label and would like to seek help regarding the same.
I am trying to run the segmentation model on my data.
I have uploaded my data in ImageTr folder with the labels in the subfolder "labels/final"
When I try to run the
monailabel start_server --app apps/radiology --studies dataset/ImageTr --conf models segmentation
I get an error
Using PYTHONPATH=/usr:
Extension horovod.torch has not been built: /usr/local/lib/python3.8/dist-packages/horovod/torch/mpi_lib/_mpi_lib.cpython-38-x86_64-linux-gnu.so not found
If this is not expected, reinstall Horovod with HOROVOD_WITH_PYTORCH=1 to debug the build error.
Warning! MPI libs are missing, but python applications are still avaiable.
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: version = False
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: app = /Oliver/apps/radiology
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: studies = /Oliver/PCAT/MONAI/ImageTr
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: verbose = INFO
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: conf = [['models', 'segmentation']]
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: host = 0.0.0.0
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: port = 8000
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: uvicorn_app = monailabel.app:app
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_keyfile = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_certfile = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_keyfile_password = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: ssl_ca_certs = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: workers = None
[2024-05-30 03:38:10,973] [12522] [MainThread] [INFO] (main:285) - USING:: limit_concurrency = None
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: access_log = False
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: root_path =
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: log_level = info
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: log_config = None
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: dryrun = False
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:285) - USING:: action = start_server
[2024-05-30 03:38:10,974] [12522] [MainThread] [INFO] (main:296) -
Allow Origins: ['*']
[2024-05-30 03:38:11,834] [12522] [MainThread] [INFO] (uvicorn.error:82) - Started server process [12522]
[2024-05-30 03:38:11,834] [12522] [MainThread] [INFO] (uvicorn.error:48) - Waiting for application startup.
App Init...
[2024-05-30 03:38:11,834] [12522] [MainThread] [INFO] (monailabel.interfaces.utils.app:37) - Initializing App from: /Oliver/apps/radiology; studies: /Oliver/PCAT/MONAI/ImageTr; conf: {'models': 'segmentation'}
CUDA initialization: The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org/ to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
[2024-05-30 03:38:11,856] [12522] [MainThread] [INFO] (monailabel.utils.others.class_utils:57) - Subclass for MONAILabelApp Found: <class 'main.MyApp'>
[2024-05-30 03:38:11,865] [12522] [MainThread] [ERROR] (uvicorn.error:121) - Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 732, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/usr/lib/python3.8/contextlib.py", line 171, in aenter
return await self.gen.anext()
File "/usr/local/lib/python3.8/dist-packages/monailabel/app.py", line 49, in lifespan
instance = app_instance()
File "/usr/local/lib/python3.8/dist-packages/monailabel/interfaces/utils/app.py", line 50, in app_instance
app = c(app_dir=app_dir, studies=studies, conf=conf)
File "/Oliver/apps/radiology/main.py", line 49, in init
for c in get_class_names(lib.configs, "TaskConfig"):
File "/usr/local/lib/python3.8/dist-packages/monailabel/utils/others/class_utils.py", line 164, in get_class_names
module = importlib.import_module("." + name, package=current_module_name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/Oliver/apps/radiology/lib/configs/segmentation.py", line 17, in
import lib.trainers
File "/Oliver/apps/radiology/lib/trainers/init.py", line 12, in
from .deepedit import DeepEdit
File "/Oliver/apps/radiology/lib/trainers/deepedit.py", line 45, in
from monailabel.tasks.train.basic_train import BasicTrainTask, Context
File "/usr/local/lib/python3.8/dist-packages/monailabel/tasks/train/basic_train.py", line 23, in
import ignite
File "/usr/local/lib/python3.8/dist-packages/ignite/init.py", line 2, in
import ignite.distributed
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/init.py", line 1, in
from ignite.distributed.auto import *
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/auto.py", line 11, in
from ignite.distributed import utils as idist
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/utils.py", line 7, in
from ignite.distributed.comp_models import (
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/comp_models/init.py", line 4, in
from ignite.distributed.comp_models.horovod import has_hvd_support
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/comp_models/horovod.py", line 27, in
class _HorovodDistModel(ComputationModel):
File "/usr/local/lib/python3.8/dist-packages/ignite/distributed/comp_models/horovod.py", line 160, in _HorovodDistModel
"SUM": hvd.mpi_ops.Sum,
AttributeError: module 'horovod.torch' has no attribute 'mpi_ops'
[2024-05-30 03:38:11,865] [12522] [MainThread] [ERROR] (uvicorn.error:59) - Application startup failed. Exiting.
Beta Was this translation helpful? Give feedback.
All reactions