[Fix] Fix dimension error when using slide inference with Mask2Former head #3752
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
The method
predict
in mmseg/models/decode_heads/mask2former_head.py is incompatible with the'slide'
inference mode.To elaborate, on line 280 of
slide_inference
in mmseg/models/segmentors/encoder_decoder.py, the key'img_shape'
ofbatch_img_metas[0]
is overwritten by the shape of the cropped image / sliding window.In line 353 of mmseg/models/decode_heads/decode_head.py, this is the first key that is used to get the target size for upsampling. As such,
crop_seg_logits
will have the same shape ascrop_img
.In mmseg/models/decode_heads/mask2former_head.py, the first shape that is referenced as a target for upsampling is
'pad_shape'
instead. Sinceslide_inference
does not overwrite this key, each cropped image is upsampled to the size of the full image, and then further padded with zeros byslide_inference
, leading to a dimension mismatch when trying to add up the paddedcrop_seg_logits
. This leads to the issue described in #3666.Modification
I have simply adjusted the size selection in mmseg/models/decode_heads/mask2former_head.py to match that of mmseg/models/decode_heads/decode_head.py. As a result, a dimension mismatch no longer occurs when using slide inference with a Mask2Former head.
Checklist