Skip to content

Latest commit

 

History

History
447 lines (295 loc) · 24.5 KB

IND2Connector.md

File metadata and controls

447 lines (295 loc) · 24.5 KB

IND2Connector interface

Use to connect to a remote peer.

The IND2Adapter:CreateConnector method returns this interface.

The IND2Connector interface inherits from IND2Overlapped. In addition, IND2Connector defines the following methods.

  • Bind - Binds the queue pair to a local address.
  • Connect - Connects the queue pair to a listening peer.
  • CompleteConnect - Completes the connection request initiated by a previous call to the Connect method.
  • Accept - Accepts a pending connection request and assigns a queue pair to use in the data exchange.
  • Reject - Rejects a pending connection request.
  • GetReadLimits- Retrieves the inbound and outbound read limits of the peer.
  • GetPrivateData- Retrieves the private data that the peer sends when it accepts or rejects the connection request.
  • GetLocalAddress - Retrieves the address of the local NetworkDirect adapter.
  • GetPeerAddress - Retrieves the address of the peer's NetworkDirect adapter.
  • NotifyDisconnect - Notifies the caller that an established connection was disconnected.
  • Disconnect - Disconnects the queue pair from the peer.

Remarks:

The remote peer gets the connecting peer's requested inbound and outbound read limits by calling the GetReadLimits method. When the remote peer calls Accept, Accept transfers the (potentially updated) limits back to the connecting peer. The connecting peer retrieves the limits by calling the GetReadLimits method.

The connecting peer can accept the limits and call the CompleteConnect method to complete the connection or call the Reject method to reject the connection. When the connecting peer calls CompleteConnect, the limit values in its queue pair are updated with the new limits.

IND2Connector::Bind

Binds the queue pair to a local address.

HRESULT Bind(
 [in] const struct sockaddr *pAddress,
 [in] ULONG cbAddress
);

Parameters:

  • pAddress [in]

    A sockaddr buffer that specifies the local address to use for the connection. Typically, this is a sockaddr_in structure for IPv4 addresses and a sockaddr_in6 structure for IPv6 addresses.

    If the sin_port or sin6_port member is specified as zero, the provider assigns a unique port number to the application.

  • cbAddress [in]

    Size, in bytes, of the pAddress buffer.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues. Unless otherwise noted, errors do not allow the connector to be used to retry the connection attempt.

  • ND_SUCCESS - The operation succeeded.
  • ND_CANCELED- The queue pair was removed.
  • ND_DEVICE_REMOVED - The underlying NetworkDirect adapter was removed from the system.
  • ND_SHARING_VIOLATION - The local address is already in use.
  • ND_TOO_MANY_ADDRESSES - The client specified a local port number of zero, and the NetworkDirect provider was unable to allocate a port from the ephemeral port space (ports 49152-65535.)
  • ND_ADDRESS_ALREADY_EXISTS - A connection with the combination of local address, local port, remote address, and remote port already exists.

IND2Connector::Connect

Connects the queue pair to a listening peer.

HRESULT Connect(
 [in] IUnknown *pQueuePair,
 [in] const struct sockaddr *pDestAddress,
 [in] ULONG cbDestAddress,
 [in] ULONG inboundReadLimit,
 [in] ULONG outboundReadLimit,
 [in, optional] const void *pPrivateData,
 [in] ULONG cbPrivateData,
 [in, out] OVERLAPPED *pOverlapped
);

Parameters:

  • pQueuePair [in]

    An interface that identifies the queue pair to use for the connection.

  • pDestAddress [in]

    A sockaddr buffer that specifies the address of the peer to connect to. Typically, this is a sockaddr_in structure for IPv4 addresses and a sockaddr_in6 structure for IPv6 addresses.

  • cbDestAddress [in]

    Size, in bytes, of the pDestAddress buffer.

  • inboundReadLimit [in]

    The maximum number of in-flight RDMA Read requests the caller will support from the peer on this connection. This value may be zero if you do not support Read requests from the peer.

  • outboundReadLimit [in]

    The maximum number of in-flight RDMA Read requests the caller would like to initiate on this connection. This value may be zero if you are not issuing Read requests to the peer.

  • pPrivateData [in, optional]

Private data that is sent with the connection request. May be nullptr if PrivateDataLength is zero. To get this data, the peer calls the IND2Connector::GetPrivateData method.

  • cbPrivateData [in]

    Length, in bytes, of the pPrivateData buffer. May be zero.

  • pOverlapped [in, out]

    A pointer to an OVERLAPPED structure that is used to indicate completion of the operation.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues. Unless otherwise noted, errors do not allow the connector to be used to retry the connection attempt.

  • ND_SUCCESS - The operation succeeded.
  • ND_PENDING - The request is pending and will be completed when the listening peer accepts the connection request.
  • ND_INVALID_BUFFER_SIZE - The specified private data length exceeded the capabilities of the underlying NetworkDirect hardware. The limits are reported in the ND2_ADAPTER_INFO structure (see IND2Adapter::Query).
  • ND_CANCELED- The queue pair was removed.
  • ND_DEVICE_REMOVED - The underlying NetworkDirect adapter was removed from the system.
  • ND_CONNECTION_ACTIVE - This queue pair is already connected to another peer.
  • ND_NETWORK_UNREACHABLE - The remote network was not reachable. The connection attempt may be retried.
  • ND_HOST_UNREACHABLE - The remote system was not reachable. The connection attempt may be retried.
  • ND_CONNECTION_REFUSED - The remote system refused the connection request. This can be due to lack of listener, backlog limits, or the peer actively rejecting the connection request. The connection attempt may be retried.
  • ND_IO_TIMEOUT - The connection request timed out. The connection attempt may be retried. Timeout values are selected by NetworkDirect providers to match their respective network characteristics.
  • ND_ACCESS_VIOLATION - The pPrivateData buffer was not valid for the size specified in PrivateDataLength.

Implementation Notes:

Providers are responsible for transmitting the inbound and outbound read limits. Callers may specify inboundReadLimit and outboundReadLimit values greater than the adapter limits, in which case providers silently limit the values to their adapter capabilities. NetworkDirect providers should avoid timing out connections because the listening peer did not call IND2Connector::Accept, there is no timeliness requirement placed on client applications.

Remarks:

The peer to that you are trying to connect must have issued an IND2Listener::GetConnectionRequest request to receive your connection request. The connection is performed asynchronously. If the connection fails, the queue pair is not affected, and it remains in the same state as when the call was made.

When the request completes, you must call the IND2Connector::CompleteConnect method to complete the connection process or the IND2Connector::Reject method to reject the connection. The application protocol defines the content and format of the private data exchange. Typically, the private data contains information for negotiating the connection with the peer.

To abandon a connection attempt, you can release the connector or call the IND2Overlapped::CancelOverlappedRequests method. The peer abandons the connection attempt by rejecting the request. Either side of a connection can terminate the connection by calling the IND2Connector::Disconnect method. When a queue pair is disconnected, all requests are flushed. The completion status is ND_CANCELED for all outstanding requests on both sides of the connection.

Providers are expected to manage a separate port space for NetworkDirect connections from the host TCP port space, unless the underlying hardware must share the host TCP port space, in which case a port mapping service must be provided by the provider to map NetworkDirect ports to host TCP ports. Local port numbers should not be shared between connections for connections established through this method.

IND2Connector::CompleteConnect

Completes the connection request initiated by a previous call to the IND2Connector::Connect method. Call this method after the peer accepts your connection request (and your connection request completes).

HRESULT CompleteConnect(
 [in, out] OVERLAPPED *pOverlapped
);

Parameters:

  • pOverlapped [in, out]

    A pointer to an OVERLAPPED structure that is used to indicate completion of the operation. The variable must remain valid for the duration of the operation.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues. If the request fails, either immediately or asynchronously, the IND2Connector object cannot be reused.

  • ND_SUCCESS - The operation succeeded.
  • ND_CONNECTION_INVALID - The queue pair is not connecting.
  • ND_PENDING - The request is pending and will be completed when the operation completes.
  • ND_DEVICE_REMOVED - The underlying local NetworkDirect adapter was removed from the system.
  • ND_IO_TIMEOUT - Connection establishment timed out. This is not an indication of a catastrophic or permanent failure, but it ends connection establishment for this particular connector.

Implementation Notes:

NetworkDirect providers should avoid timing out connections when the client does not call this method in a timely manner because there is no timeliness requirement placed on client applications.

Remarks:

You must call this method to complete the connection. If you do not call this method, the connection may time out.

IND2Connector::Accept

Accepts a pending connection request and assigns a queue pair to use in the data exchange.

HRESULT Accept(
 [in] IUnknown *pQueuePair,
 [in] ULONG inboundReadLimit,
 [in] ULONG outboundReadLimit,
 [in, optional] const VOID *pPrivateData,
 [in] ULONG cbPrivateData,
 [in, out] OVERLAPPED *pOverlapped
);

Parameters:

  • pQueuePair [in]

    An interface that defines the queue pair to connect to the peer's queue pair. The queue pair must have been created by using the same IND2Adapter instance as this instance of the Connector object.

  • inboundReadLimit [in]

    The maximum number of in-flight RDMA Read requests that the caller will support from the peer on this connection. This value may be zero if you do not support Read requests from the peer.

  • outboundReadLimit [in]

    The maximum number of in-flight RDMA Read requests that the caller would like to initiate on this connection. This value may be zero if you are not issuing Read requests to the peer.

  • pPrivateData [in, optional]

    Private data to transmit to the peer. May be nullptr if cbPrivateData is zero.

  • cbPrivateData [in]

    The size, in bytes, of the pPrivateData buffer.

  • pOverlapped [in, out]

    A pointer to an OVERLAPPED structure that is used to indicate completion of the operation.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues. If the request fails, either immediately or asynchronously, the IND2Connector object cannot be reused.

  • ND_SUCCESS - The operation succeeded.
  • ND_PENDING - The request is pending and will be completed when the connection is fully established.
  • ND_CANCELED - The queue pair was removed.
  • ND_DEVICE_REMOVED - The underlying NetworkDirect adapter was removed from the system.
  • ND_CONNECTION_ACTIVE - The specified queue pair is already connected.
  • ND_CONNECTION_ABORTED - The connecting peer abandoned the connection establishment.
  • ND_ACCESS_VIOLATION - The pPrivateData buffer is not valid for the size specified in cbPrivateData.
  • ND_INVALID_BUFFER_SIZE - The specified private data buffer length exceeds the capabilities of the underlying NetworkDirect hardware. The limits are reported in the ND2_ADAPTER_INFO structure (see IND2Adapter::Query).
  • ND_IO_TIMEOUT - The peer did not call the IND2Connector::CompleteConnect method to complete the connection.

Implementation Notes: Providers are responsible for transmitting the inbound and outbound read limits. Callers may specify inboundReadLimit and outboundReadLimit values greater than the adapter limits or greater than the values offered by the connecting peer, in which case providers silently limit the values to the lower of their adapter capabilities or the values offered by the connecting peer.

NetworkDirect providers should avoid timing out connections when the client does not call this method in a timely manner because there is no timeliness requirement placed on client applications.

Remarks:

Calling this method causes the peer's connection request (IND2Connector::Connect) to complete. A queue pair can be connected to only one other queue pair at a time.

IND2Connector::Reject

Rejects a pending connection request.

HRESULT Reject(
 [in, optional] const VOID *pPrivateData,
 [in] ULONG cbPrivateData
);

Parameters:

  • pPrivateData [in, optional]

    Private data to transmit to the peer. May be nullptr if cbPrivateData is zero.

  • cbPrivateData [in]

    The size, in bytes, of the pPrivateData buffer.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_PENDING - The request is pending and will be completed when the connection is fully established.
  • ND_DEVICE_REMOVED - The underlying NetworkDirect adapter was removed from the system.
  • ND_CONNECTION_ABORTED - The connecting peer abandoned the connection establishment.
  • ND_ACCESS_VIOLATION - The pPrivateData buffer is not valid for the size given in PrivateDataLength.
  • ND_INVALID_BUFFER_SIZE - The specified buffer length exceeds the capabilities of the underlying NetworkDirect hardware. The limits are reported in the ND2_ADAPTER_INFO structure (see IND2Adapter::Query).

Remarks:

The remote peer or connecting peer can call this method to reject the connection. The connecting peer would call this method after determining that the inbound and outbound Read limits that the remote peer specifies are not acceptable.

IND2Connector::GetReadLimits

Retrieves the read limits of the peer.

HRESULT GetReadLimits(
 [out, optional] ULONG *pInboundReadLimit,
 [out, optional] ULONG *pOutboundReadLimit,
);

Parameters:

  • pInboundReadLimit [out, optional]

    Pointer to the value of outboundReadLimit that is specified by the peer when calling IND2Connector::Connect, or IND2Connector::Accept. The value is limited by the NetworkDirect hardware capabilities on both ends of the connection, and the value can be less than or equal to the value that is specified by the remote peer. Use this value to determine whether to accept or complete a connection.

  • pOutboundReadLimit [out, optional]

    Pointer to the value of inboundReadLimit that is specified by the peer when calling IND2Connector::Connect, or IND2Connector::Accept. The value is limited by the NetworkDirect hardware capabilities on both ends of the connection, and the value may be less than or equal to the value that is specified by the remote peer. Use this value to determine whether to accept or complete a connection.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_CONNECTION_INVALID - The connector is not connected.
  • ND_UNSUCCESSFUL - The operation failed.

Remarks:

The connecting peer calls this method after the IND2Connector::Connect call completes and before calling the IND2Connector::CompleteConnect method, as well as after the IND2Listener::GetConnectionRequest call completes and before calling IND2Connector::Accept.

IND2Connector::GetPrivateData

Retrieves the private data that the peer sends when it accepts or rejects the connection request.

HRESULT GetPrivateData(
 [out, optional] VOID *pPrivateData,
 [in, out] ULONG *pcbPrivateData
);

Parameters:

  • pPrivateData [out, optional]

    The buffer to receive the private data. May be nullptr if pcbPrivateData is zero.

  • pcbPrivateData [in, out]

    The size, in bytes, of the pPrivateData buffer. If the buffer is too small, the method fails with ND_BUFFER_OVERFLOW, and this parameter is set to the required buffer size. If the buffer is too big, the method sets this parameter to the size used on output.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_BUFFER_OVERFLOW - The pPrivateData buffer is not large enough to hold all the private data. Partial data was returned up to the size of the buffer. The pcbPrivateData is updated with the required buffer size.
  • ND_CONNECTION_INVALID - The connector is not connected.
  • ND_UNSUCCESSFUL - The operation failed.

Remarks:

The connecting peer calls this method after the IND2Connector::Connect call completes and before calling the IND2Connector::CompleteConnect method, as well as after the IND2Listener::GetConnectionRequest call completes and before calling IND2Connector::Accept.

IND2Connector::GetLocalAddress

Retrieves the connector's address, which includes the connector's adapter address and port number. To succeed, the connector must be connected or in the process of connecting.

HRESULT GetLocalAddress(
 [out, optional] struct sockaddr *pAddress,
 [in, out] ULONG *pcbAddress
);

Parameters:

  • pAddress [out, optional]

    A sockaddr buffer that will receive the local address. May be nullptr if pcbAddress is zero.

  • pcbAddress [in, out]

    The size, in bytes, of the pAddress buffer. If the buffer is too small, the method fails with ND_BUFFER_OVERFLOW and sets this parameter to the required buffer size. If the buffer is too big, the method sets this parameter to the size that is used on output.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_BUFFER_OVERFLOW - The pAddress buffer is not large enough to hold the address, and the buffer was not modified. The pcbAddress parameter is updated with the required buffer size.
  • ND_CONNECTION_INVALID - The connector is not connected or not in the process of connecting.

IND2Connector::GetPeerAddress

Retrieves the address of the peer's NetworkDirect queue pair. To succeed, the queue pair must be connected.

HRESULT GetPeerAddress(
 [out, optional] struct sockaddr *pAddress,
 [in, out] ULONG *pcbAddress
);

Parameters:

  • pAddress [out, optional]

    A sockaddr buffer that will receive the remote address. May be nullptr if pcbAddress is zero.

  • pcbAddress [in, out]

    The size, in bytes, of the pAddress buffer. If the buffer is too small, the method fails with ND_BUFFER_OVERFLOW and sets this parameter to the required buffer size. If the buffer is too big, the method sets this parameter to the size that is used on output.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_BUFFER_OVERFLOW - The pAddress buffer is not large enough to hold the address—the buffer was not modified. The pcbAddress parameter is updated with the required buffer size.
  • ND_CONNECTION_INVALID - The connector is not connected.

IND2Connector::NotifyDisconnect

Notifies the caller that an established connection was disconnected.

HRESULT NotifyDisconnect(
 [in, out] OVERLAPPED *pOverlapped
);

Parameters:

  • pOverlapped [in, out]

    A pointer to an OVERLAPPED structure that is used to indicate a disconnection request.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_PENDING - The request is pending.
  • ND_CONNECTION_INVALID - The connector was never connected.

Asynchronously, the method can return ND_CANCELED (ND_CANCELED should only be returned when a caller calls IND2Overlapped::CancelOverlappedRequests).

Implementation Notes:

A queue pair should not have its outstanding requests completed with a canceled status when receiving a disconnection request. The only action a provider should take is to complete the NotifyDisconnect request, if any.

IND2Connector::Disconnect

Disconnects the connector and associated queue pair from the peer.

HRESULT Disconnect(
 [in, out] OVERLAPPED *pOverlapped
);

Parameters:

  • pOverlapped [in, out]

    A pointer to an OVERLAPPED structure that is used to indicate completion of the operation.

Return Value:

When you implement this method, you should return the following return values. If you return others, try to use well-known values to aid in debugging issues.

  • ND_SUCCESS - The operation succeeded.
  • ND_PENDING - The request is pending and will be completed when the listening peer is disconnected.
  • ND_CANCELED - The queue pair was removed.
  • ND_DEVICE_REMOVED - The underlying NetworkDirect adapter was removed from the system.
  • ND_CONNECTION_INVALID - The connector is not connected.

Remarks:

This method implicitly flushes all outstanding requests on the connector’s associated queue pair. You cannot reuse the queue pair after the request completes. Either side of the connection can terminate the connection; the disconnect handshake is protocol dependent.

Implementation Notes:

Calling the IND2QueuePair::Release method on the queue pair without first disconnecting is valid. If the holder of the last reference does not disconnect before releasing, the object should implicitly disconnect. Disconnecting should implicitly flush pending requests (the pending requests will complete in the completion queue with an ND_CANCELED status).