Add QNN EP HTP shared memory allocator

[edgchen1]

Description

Adds QNN EP HTP shared memory allocator.

The HTP shared memory allocator (HtpSharedMemoryAllocator) calls the rpcmem shared library (libcdsprpc.so/dll) to allocate and free memory that can be shared between HTP and CPU.

The allocator can be enabled by setting QNN EP option enable_htp_shared_memory_allocator to 1. QNNExecutionProvider::CreatePreferredAllocators() will then return an instance of HtpSharedMemoryAllocator.

For each QNN context, we also need to register and unregister memory handles in order to use the HTP shared memory. This memory handle management is added to QnnBackendManager, which also manages the QNN context handles.

For more information about using HTP shared memory with QNN, see: https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/htp_shared_buffer_tutorial.html#shared-buffer-tutorial

Limitations:

• HTP shared memory usage is only supported for graph inputs and outputs. Intermediate values are not supported.
• An allocation is assigned to a single shared memory buffer. The allocator is not smart enough to have multiple allocations share a single shared memory buffer.

Motivation and Context

Improve performance by using HTP shared memory to avoid overhead from copying data between CPU and NPU.

[yuslepukhin]

Can this be used for Lora support when the model is modified to have optional inputs, and the data can be fed to override default initializers?

[yuslepukhin]

🕐

[edgchen1]

Can this be used for Lora support when the model is modified to have optional inputs, and the data can be fed to override default initializers?

I'm not too familiar with the scenario. If that can be done using OrtValues, an input OrtValue can use this new allocator.

[yuslepukhin]

[edgchen1]

/azp run Windows GPU WebGPU CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).