Add padding while evaluating on the original mask size

[blumenstiel]

Is your feature request related to a problem? Please describe.
Some model have a specific patch size which only allow to run images with multiples of these sizes. E.g., a patch size of 14 and image size of 512 is not possible.

Describe the solution you'd like
A simple approach is to add padding (e.g. nearest or mirror padding) to increase the image size to e.. 518 (which is divisible by 14).
However, doing this using albumentation.PadIfNeeded would probably also increase the mask (not tested, just an assumption). The evaluation should be performed on the original 512 mask, not the 518 padded mask, as this would upsample the edge pixels. Therefore, TerraTorch needs to handle this in some way.

Describe alternatives you've considered (optional)
Alternatively, the padding is performed only on the images, and the mask is cropped afterwards to the original size. This could happened in the TerraTorch tasks.

@daniszw @paolo-fraccaro

[singam96]

This is currently possible

backbone = timm.create_model(model_name, pretrained=False, features_only=True, num_frames=NUM_FRAMES, out_indices=out_indices, padding='constant')

We can use padding modes from https://pytorch.org/vision/main/generated/torchvision.transforms.Pad.html

[paolofraccaro]

But not all models are instantiated through timm right?

[romeokienzler]

@Joao-L-S-Almeida thinks this might be already solved by one of @blumenstiel 's PR's, let's test

[blumenstiel]

No, it's not working at the moment and IMHO it is a must have.

Some models support padding as pointed out earlier. But the important part is that the tasks only evaluate the original area (which is currently not implemented).

[Joao-L-S-Almeida]

In this way, I'll work in this task. Em seg., 23 de dez. de 2024, 06:57, Benedikt Blumenstiel < ***@***.***> escreveu:

…

No, it's not working at the moment and IMHO it is a must have. Some models support padding as pointed out earlier. But the important part is that the tasks only evaluate the original area (which is currently not implemented). — Reply to this email directly, view it on GitHub <#261 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIQOYDSZW5CJV7U5TXN6QC32G7M7RAVCNFSM6AAAAABSQNXHCCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJZGMZTONBZGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Joao-L-S-Almeida]

It's being implemented in #342.

[romeokienzler]

@blumenstiel is it ok now?

[romeokienzler]

@blumenstiel lmc