Skip to content

Latest commit

 

History

History
138 lines (84 loc) · 7.11 KB

IND2MemoryRegion.md

File metadata and controls

138 lines (84 loc) · 7.11 KB

IND2MemoryRegion interface

Use to make application buffers accessible to the NetworkDirect adapter, which will use them in I/O operations.

Note that only one memory buffer can be registered to a memory region object. The user should be able to register the same memory buffer multiple times, even for different access rights. However, one can not register two different memory buffers to the same memory region.

The IND2MemoryRegion interface inherits from IND2Overlapped. In addition, IND2MemoryRegion defines the following methods.

  • Register - Registers an application-defined buffer.

  • Deregister - Deregisters a previously registered application-defined buffer.

  • GetLocalToken - Retrieves the local token for the memory region, used in ND2_SGE structures as part of I/O requests.

  • GetRemoteToken - Retrieves the remote token that allows remote access to the memory region. It is used by the connected QueuePair in Read and Write requests.

Remarks:

Memory registrations are fundamental to data transfer operations with NetworkDirect adapters. Memory registrations map application buffers to a NetworkDirect adapter and provide the NetworkDirect adapter the ability to access application buffers directly without operating system involvement.

Memory registration is typically an expensive operation and the NetworkDirect SPI allows all memory registration and deregistration operations to be processed asynchronously in order to parallelize application processing with registration. Because memory region operations can be asynchronous, there is no opportunity for post-processing by the user-mode NetworkDirect provider libraries that support NetworkDirect adapters.

IND2MemoryRegion::Register

Registers an application-defined buffer that is used for Send, Receive, Read, and Write requests.

HRESULT Register(
 [in] const void *pBuffer,
 [in] SIZE_T cbBuffer,
 [in] ULONG flags,
 [in, out] OVERLAPPED *pOverlapped
);

Parameters:

  • pBuffer [in]

    The application-defined buffer to register.

  • cbBuffer [in]

    The size, in bytes, of the application-defined buffer.

  • flags [in]

    The following flags control the behavior of the Register request. You can specify one or more of the following flags:

    • ND_MR_FLAG_ALLOW_LOCAL_WRITE

      Allow the adapter Write access to the memory region.

    • ND_MR_FLAG_ALLOW_REMOTE_READ

      Enable Read access to the memory region by any connected peer.

    • ND_MR_FLAG_ALLOW_REMOTE_WRITE

      Enable write access to the memory region by any connected peer.

    • ND_MR_FLAG_RDMA_READ_SINK

      Enable access to the memory region to be used as a sink for RDMA read operation.

    • ND_MR_FLAG_DO_NOT_SECURE_VM

      Instructs the provider to not secure the virtual memory range by calling the MmSecureVirtualMemory function. Applications that are using AWE memory management functions must specify this flag to register AWE memory. Securing memory is only required when implementing a cache of memory registrations. Memory registrations that are not going to be cached (where the lifetime of the memory region is always less than the lifetime of the buffer) do not need to be secured.

  • pOverlapped [in, out]

A pointer to an OVERLAPPED structure that must remain valid for the duration of the operation.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation completed successfully.
  • ND_PENDING - The NetworkDirect adapter is in process of registering the memory. Completion is reported asynchronously.
  • ND_INSUFFICIENT_RESOURCES - There were not enough hardware resources to register the memory.
  • ND_ACCESS_VIOLATION - The specified buffer or buffer size was not valid.
  • ND_DEVICE_REMOVED - The underlying NetworkDirect adapter was removed from the system. Only cleanup operations on the NetworkDirect adapter will succeed.
  • ND_INVALID_PARAMETER - The buffer exceeds the size supported by the adapter. For details, see the MaxRegistrationSize member of the ND2_ADAPTER_INFO structure.

Remarks:

The MaxRegistrationSize member of the ND2_ADAPTER_INFO structure contains the maximum application-defined buffer size that can be registered with the adapter. To get the adapter information, call the IND2Adapter::Query method.

You can allocate and register your memory before connecting, or you can start the connection process and then register the memory before calling IND2Connector::Accept or IND2Connector::CompleteConnect.

The registered memory's scope is the adapter object. It can be used on any objects within the adapter, for example, multiple queue pairs can both use the registered memory.

IND2MemoryRegion::Deregister

Removes the memory registration for an application-defined buffer.

HRESULT Deregister (
 [in] OVERLAPPED *pOverlapped
);

Parameters:

  • pOverlapped [in]

    A pointer to an OVERLAPPED structure that must remain valid for the duration of the operation.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation completed successfully.
  • ND_PENDING - The memory region is in the process of being freed. Completion will be reported asynchronously.
  • ND_DEVICE_BUSY - The memory region still has memory windows bound to it.

Remarks:

It is very important that Deregister is called before the memory region is released, otherwise the registered memory buffer will stay locked and count against your process.

IND2MemoryRegion::GetLocalToken

Gets the local memory token to allow the region to be referenced in ND2_SGE structures that are passed to Send, Receive, Read, and Write requests.

UINT32 GetLocalToken();

Return Value:

The local token to use in ND2_SGE structures.

IND2MemoryRegion::GetRemoteToken

Gets the remote memory token to send to remote peers to enable them to perform Read or Write requests against the region.

UINT32 GetRemoteToken();

Return Value:

The remote access token to send to connected peers. This value is opaque to the client. This value should be in network order to enable interoperability with other system architectures.