CUDA Context

[soldierofhell]

Hi, could you please share some details how you manage context under the hood? In python code we can see only memory limit for Tensorflow.
By the way, is GPU mandatory for depth processing?

Regards,

[adujardin]

Hi,
The ZED SDK requires CUDA and therefore a GPU, for any tasks, including the depth computation.

The CUDA Context is by default created by the ZED SDK during the Camera::open.
Using the C++ API you can provide your own with InitParameters::sdk_cuda_ctx.

In the past, some users work around this in python with a script (see this issue)

[soldierofhell]

Thank you @adujardin, in fact I already wanted to rewrite this python script to C++.
So for example by sharing context we can grab image and depth matrix/tensor directly on GPU without making copies through numpy CPU, saving GPU memory and processing time, am I right?

[adujardin]

Yes, the GPU buffers can be shared within the same CUDA Context without copying.

However, sharing the same context basically means there's no overlap possible and the computing of the ZED SDK and the inference won't be in parallel anymore. It might still be faster at the end but not necessarily.

NB: To get parallelism while sharing a context you need to create non-blocking streams but you can't control it in the ZED SDK (it uses the default one) and I doubt you can in TensorFlow either.

[soldierofhell]

But if ZED and TF can parallelize their tasks so that all cores are used there's no benefit from overlap?

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days