Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resample operations should fail when no channels #6809

Closed
kretes opened this issue Jul 31, 2023 · 5 comments · May be fixed by #6819
Closed

Resample operations should fail when no channels #6809

kretes opened this issue Jul 31, 2023 · 5 comments · May be fixed by #6819
Assignees
@a-parida12
Labels
Contribution wanted Feature request

Comments

@kretes
Copy link
Contributor

kretes commented Jul 31, 2023

Describe the bug
resample operations like ResampleToMatch or 'Spacing` (probably all that use same underlying mechanism) behave wrong when passed data has no channels.

To Reproduce
Steps to reproduce the behavior:
Run this piece of code:

from monai.data import MetaTensor
from monai.transforms import Spacing, EnsureChannelFirst
import torch
import monai

print(monai.__version__)

img = MetaTensor(torch.rand(10,10,5), affine=torch.eye(4), meta={"original_channel_dim": torch.nan})
sr = Spacing(pixdim=(2,2,2))

double_spacing = sr(img)

#expected shape (6,6,3) or an exception, but the results is (10,6,3), so first dim is treated as channel
print(double_spacing.shape)

img_chan = EnsureChannelFirst()(img)
double_spacing = sr(img_chan)

#expected shape (1,6,6,3), all fine
print(double_spacing.shape)

Expected behavior
Spatial Transformations should lean towards detecting inconsistencies and throw an exception when number of spatial_dims doesn't match.
In this case - image is 3D, it's affine matrix is 4x4, and passed pixdim has 3 entries. Since Spacing requires explicit channels dimensions - it should raise an exception, since affine and pixdim suggest the image is 3D, while tensor isn't CWHD.

Similar thing happens with ResampleToMatch where resampling is continued for a 3D volume without channels, but produces wrong results.

I think this is a better way as we do not hide a potential issue in user's code and forcing the user to find out why resampling isn't working well.

Environment

python -c 'import monai; monai.config.print_debug_info()'

================================
Printing MONAI config...
================================
MONAI version: 1.2.0
Numpy version: 1.24.3
Pytorch version: 2.0.1
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: c33f1ba588ee00229a309000e888f9817b4f1934
MONAI __file__: /raid/shared/tbartczak/conda/envs/tb_xlungs2/lib/python3.11/site-packages/monai/__init__.py

Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
ITK version: 5.3.0
Nibabel version: 5.1.0
scikit-image version: 0.20.0
Pillow version: 9.4.0
Tensorboard version: 2.13.0
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.15.2
tqdm version: 4.65.0
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.9.5
pandas version: 2.0.2
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: 1.0.0

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies


================================
Printing system config...
================================
System: Linux
Linux version: Ubuntu 20.04.6 LTS
Platform: Linux-5.4.0-149-generic-x86_64-with-glibc2.31
Processor: x86_64
Machine: x86_64
Python version: 3.11.3
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 128
Num logical CPUs: 128
Num usable CPUs: 8
CPU usage (%): [1.2, 0.3, 0.3, 100.0, 0.6, 0.3, 0.0, 0.3, 100.0, 0.0, 0.6, 0.3, 0.3, 0.6, 0.9, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.6, 0.6, 0.6, 0.3, 0.6, 0.0, 0.3, 0.6, 0.3, 0.3, 0.3, 11.4, 10.4, 12.2, 7.5, 11.7, 10.8, 9.9, 5.7, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 99.7, 100.0, 100.0, 100.0, 44.7, 100.0, 1.2, 1.2, 0.0, 100.0, 0.3, 0.9, 0.0, 1.5, 1.2, 3.6, 100.0, 0.9, 0.9, 100.0, 1.2, 2.1, 1.2, 0.6, 0.3, 0.3, 0.3, 0.3, 0.6, 1.5, 2.4, 1.2, 1.8, 0.3, 0.3, 0.6, 0.3, 0.3, 1.2, 0.6, 0.6, 0.0, 0.0, 0.3, 0.0, 0.6, 0.3, 0.6, 0.9, 0.3, 0.3, 0.9, 0.0, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.0, 0.9, 0.3, 0.6, 0.6, 0.6, 0.0, 0.6, 0.3, 2.1]
CPU freq. (MHz): 2865
Load avg. in last 1, 5, 15 mins (%): [28.7, 29.0, 29.1]
Disk usage (%): 100.0
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 1007.7
Available memory (GB): 784.3
Used memory (GB): 213.3

================================
Printing GPU config...
================================
Num GPUs: 1
Has CUDA: True
CUDA version: 11.8
cuDNN enabled: True
cuDNN version: 8700
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90', 'compute_37']
GPU 0 Name: NVIDIA A100-SXM4-40GB
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 108
GPU 0 Total memory (GB): 39.4
GPU 0 CUDA capability (maj.min): 8.0
@a-parida12
Copy link
Contributor

@wyli if no one is working on it, Can you please assign the issue to me. I will take a look at it and suggest changes.

My suggestion would be to not raise an Exception as the transforms can also be used in scenario where deep nueral networks are not used. Ideally, if affine is 4x4 and input is 3D, it should assume no channel is provided and act accordingly.

What do you think? @kretes @wyli

@a-parida12
Copy link
Contributor

@wyli The trouble mostly comes from peek_pending_shape() method on

MONAI/monai/transforms/spatial/array.py

Line 489 in 6f5cea3

data_array.peek_pending_shape() if isinstance(data_array, MetaTensor) else data_array.shape[1:]
.

Which according to

MONAI/monai/data/meta_tensor.py

Line 479 in 6f5cea3

def peek_pending_shape(self):
assumes that it is channel first image.

As the changes to MetaTensor objects have far wider implications maybe that should be a different issue that needs to be solved before this.

@wyli
Copy link
Contributor

wyli commented Aug 3, 2023

Sure, I think the main ask of this ticket is to improve the input validation -- when pixdim is configured to have 3 elements and at the same time the input image is not in the format channel-first spatially 3d, the user would get some warning or error msgs. (The affine is in 4x4 so that we can potentially track 2d spatial slices/projections from 3d volumes)

@a-parida12
Copy link
Contributor

Got you! Added a userwarning whenever truncation of the affine matrix happens so that it may be an indication of missing channel first inputs.

@a-parida12 a-parida12 mentioned this issue Aug 3, 2023
Open
7 tasks
@vikashg
Copy link

vikashg commented Jan 4, 2024

@KumoLiu and @a-parida12 is it completed. ?
If not please reopen.

@vikashg vikashg closed this as completed Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@a-parida12 a-parida12

Labels
Contribution wanted Feature request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

warning to truncate affine a-parida12/MONAI
4 participants
@wyli @kretes @vikashg @a-parida12