What is execution context memory?

[wxsms]

We found that with same model & same inference image & same configuration, model running on different devices will result in different execution context memory, and the difference could be very large. for example, on 4090, it takes 2304 MiB, and on A10 it takes 6384 MiB. Is this expected or configurable?

[nv-guomingz]

I'd like to say it's a expected but not configurable(depends on your device hw).

The value refers to a memory which mainly used for temporary storage required by layer implementations.
Different hw will has their own optimization, so the value difference is expected.