torch.ops._caffe2.BoxWithNMSLimit and tracing for rotated boxes

[alekseynp]

Around a year ago I forked detectron2 and implemented torch.jit.trace for C++ cuda rotated faster rcnn inference. That implementation has never been pretty and the differing code paths between train/eval, and trace have been a source of pain. As I re-evaluate our options, I have the following questions:

• Why not use these torch.ops._caffe2 functions in the training code path?
• What is the long term future for these ops?
• If I encounter serious bugs in these ops, will someone be interested to see them?

I see that detectron2 has come a long way in deployment over the last year. The docs show support for tracing to c++ faster rcnn presumably supporting gpu with a torchvision dependency. If you were me and wanted rotated box support, what would you do?

[ppwwyyxx]

Many detection ops in torch.ops._caffe2 don't have gradients AFAIK, so they can't be used for training.
And the idea with pytorch development is that in the long term they're going to be abandoned if more users migrate to using pytorch ops.

If there are serious bugs like correctness issues we're still interested in fixing them since our users rely on them for deployment.

Deployment needs those ops, and they today exist in caffe2 and in detectron2 (which is not a C++ library). So I guess the choices for you are either to use the caffe2 solution (which don't support GPU very well) or see if torchvision can host these ops (pytorch/vision#2761)