From 6e262c66127b6db4559e70f80f7a7df08f078ae1 Mon Sep 17 00:00:00 2001 From: Anup Patel Date: Fri, 17 Jan 2025 18:18:41 +0530 Subject: [PATCH 1/3] ext-mpxy: Improve notification related text To address the ARC review feedback, improve notification related text as follows: 1) Replace the term "notification indication" with just "indication". 2) Minimize the use of term "client" and replace it with the term "supervisor software" at appropriate places to be consistent with other SBI extensions. Signed-off-by: Anup Patel Reviewed-by: Rahul Pathak Reviewed-by: Samuel Holland --- src/ext-mpxy.adoc | 77 ++++++++++++++++++++++++----------------------- 1 file changed, 39 insertions(+), 38 deletions(-) diff --git a/src/ext-mpxy.adoc b/src/ext-mpxy.adoc index 3074593..ade0c83 100644 --- a/src/ext-mpxy.adoc +++ b/src/ext-mpxy.adoc @@ -66,11 +66,11 @@ MUST support these attributes. Apart from standard attributes, a message channel may also have message protocol attributes which are defined by the message protocol specification. -Once a client in supervisor software has verified the channel and its -associated attributes, it can use the MPXY interface to send messages over the -message channel where each message is identified by a 32 bits unsigned integer -called `message_id`. The set of `message_id` that can be sent over an MPXY -channel are defined by the message protocol specification. +Once supervisor software has verified the channel and its associated attributes, +it can use the MPXY interface to send messages over the message channel where +each message is identified by a 32 bit unsigned integer called `message_id`. +The set of `message_id` that can be sent over an MPXY channel are defined by +the message protocol specification. [#table_mpxy_channel_attributes] .MPXY Channel Attributes @@ -151,11 +151,12 @@ wait for response can use this timeout value. ``` Any defined bit as `1` means the corresponding capability is supported. -The SBI implementation only needs to support one notification indication method, -either MSI or SSE. If both are enabled, the MSI is preferred over the SSE event. +The SBI implementation only needs to support one method to indicate the +availability of notifications, either MSI or SSE. If both are enabled, the +MSI is preferred over the SSE event. -If the Get Notifications (FID #6) is not supported then the Events State -Support, SSE Event and MSI bits will be `0`. +If Get Notifications function (FID #6) is not supported then the Events State +Support, SSE Event and MSI bits must be `0`. | SSE_EVENT_ID | 0x00000006 @@ -169,7 +170,7 @@ unspecified. | 0x00000007 | RW | *MSI Control* + -MSI control for notification indication. +Control for MSI based indication. ``` 0 = Disable 1 = Enable @@ -177,9 +178,9 @@ MSI control for notification indication. This attribute can be set to `1` if `MSI_ADDR_LOW` and `MSI_ADDR_HIGH` attributes point to a valid MSI target. -If the message channel does not support MSI based notification indication -discovered via the `CHANNEL_CAPABILITY` attribute, then the `MSI_CONTROL` will -ignore writes and always reads zero. +If the message channel does not support MSI based indication as discovered via +the `CHANNEL_CAPABILITY` attribute, then `MSI_CONTROL` ignore writes and always +reads zero. The reset value of this attribute is `0`. @@ -189,8 +190,8 @@ The reset value of this attribute is `0`. | *MSI Address Low* + Low 32 bits of the MSI target physical address. -If the message channel does not support MSI based notification indication then -this attribute ignores writes and always reads `0`. +If the message channel does not support MSI based indication then this attribute +ignores writes and always reads `0`. The reset value of this attribute is `0`. @@ -200,8 +201,8 @@ The reset value of this attribute is `0`. | *MSI Address High* + High 32 bits of the MSI target physical address. -If the message channel does not support MSI based notification indication then -this attribute ignores writes and always reads `0`. +If the message channel does not support MSI based indication then this attribute +ignores writes and always reads `0`. The reset value of this attribute is `0`. @@ -211,8 +212,8 @@ The reset value of this attribute is `0`. | *MSI Data* + MSI data word written to the MSI target. -If the message channel does not support MSI based notification indication then -this attribute ignores writes and always reads `0`. +If the message channel does not support MSI based indication then this attribute +ignores writes and always reads `0`. The reset value of this attribute is `0`. @@ -225,9 +226,9 @@ attribute can be used to enable state reporting like number of events `RETURNED`, `REMAINING` or `LOST` after a call to Get Notifications (FID #6) function. -The reset value of this attribute is `0`, which means disabled. If a client -wants to enable events state reporting, it MUST write `1`. If the events -state reporting is not supported by the channel or the +The reset value of this attribute is `0`, which means disabled. If supervisor +software wants to enable events state reporting, it MUST write `1`. If the +events state reporting is not supported by the channel or the Get Notifications (FID #6) function is not implemented as indicated by the `CHANNEL_CAPABILITY` attribute, then the writes to this attribute will be ignored. @@ -715,12 +716,12 @@ struct sbiret sbi_mpxy_get_notification_events(uint32_t channel_id) Get the message protocol specific notification events on the MPXY channel specified by the `channel_id` parameter. The events are message protocol specific and MUST be defined in the respective message protocol specification. The SBI -implementation may support notification indication mechanisms like MSI or SSE -to indicate the availability of events to the supervisor software. +implementation may support indication mechanisms like MSI or SSE to inform the +supervisor software about the availability of events. -NOTE: If the message channel does not support or is not configured for a -notification indication mechanism, such as MSI or SSE, the supervisor software -can periodically invoke the poll operation `sbi_mpxy_get_notification_events`. +NOTE: If the message channel does not support or is not configured for an +indication mechanism, such as MSI or SSE, the supervisor software can periodically +invoke the poll operation `sbi_mpxy_get_notification_events`. NOTE: Notifications are asynchronous from the perspective of the supervisor software. Any caching or buffering mechanism is specific to the SBI @@ -731,16 +732,16 @@ implementation. Depending on the message protocol implementation, a channel may support events state which includes data like number of events `RETURNED`, `REMAINING` and -`LOST`. Events state data is optional and if the message protocol -implementation supports then the channel will have a corresponding bit set in -`CHANNEL_CAPABILITY` attribute. + -By default the events state is disabled and clients can explicitly enable it -through the `EVENTS_STATE_CONTROL` attribute. +`LOST`. Events state data is optional, and if the message protocol implementation +supports it, then the channel will have the corresponding bit set in the +`CHANNEL_CAPABILITY` attribute. By default the events state is disabled and +supervisor software can explicitly enable it through the `EVENTS_STATE_CONTROL` +attribute. NOTE: Only after enabling the events state reporting through `EVENTS_STATE_CONTROL` attribute, the events state data will start getting accumulated by the SBI -implementation. A client may enable the `EVENTS_STATE_CONTROL` attribute in the -initialization phase if it is supported. +implementation. The supervisor software may enable the `EVENTS_STATE_CONTROL` +attribute in the initialization phase if it is supported. In the shared memory, 16 bytes starting from offset `0x0` are used to return this state data. @@ -755,10 +756,10 @@ Offset 0x10: Start of message protocol specific notification events data ``` The `RETURNED` field represents the number of events which are returned in the -shared memory when this function is called. The `REMAINING` field -represents the number of events still remaining with SBI implementation and the -client may need to call this function again until the `REMAINING` field becomes -`0`. +shared memory when this function is called. The `REMAINING` field represents +the number of events still remaining with SBI implementation. The supervisor +software may need to call this function again until the `REMAINING` field +becomes `0`. The `LOST` field represents the number of events which are lost due to limited buffer size managed by the message protocol implementation. Details of From 95ef0092866fd9fac859aed2995480c33febb54e Mon Sep 17 00:00:00 2001 From: Anup Patel Date: Fri, 17 Jan 2025 22:15:23 +0530 Subject: [PATCH 2/3] ext-mpxy: Improve description of sbi_mpxy_send_message_with(out)_response() Based on ARC feedback, improve/clarify description of the SBI MPXY functions to send message with/without response. Signed-off-by: Anup Patel Reviewed-by: Rahul Pathak --- src/ext-mpxy.adoc | 43 +++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/src/ext-mpxy.adoc b/src/ext-mpxy.adoc index ade0c83..0272f0f 100644 --- a/src/ext-mpxy.adoc +++ b/src/ext-mpxy.adoc @@ -583,19 +583,17 @@ sbi_mpxy_send_message_with_response(uint32_t channel_id, uint32_t message_id, unsigned long message_data_len) ``` +Send a message to the MPXY channel specified by the `channel_id` parameter and +wait until a message response is received from the MPXY channel. The `message_id` +parameter specifies the message protocol specific identification of the message +to be sent whereas the `message_data_len` parameter represents the length of +message data in bytes which is located at the offset `0x0` in the shared memory +setup by the calling hart. -Send a message to the MPXY channel specified by the `channel_id` -parameter. The `message_id` parameter specifies a message specific to a message -protocol to be sent whereas the `message_data_len` parameter represents the -length of message data in bytes which is located at the offset `0x0` in the -shared memory setup by the calling hart. - -After sending the message, this function waits for the message response from -the SBI implementation. This function only succeeds upon receipt of the response. -Some messages may require sending multiple times for complete data transfer so -the supervisor software is responsible for doing multiple requests in such cases. -Details of such cases can be found in respective message protocol -specifications. +This function only succeeds upon receipt of a message response from the MPXY +channel. In cases where complete data transfer requires multiple transmissions, +the supervisor software shall send multiple messages as necessary. Details of +such cases can be found in respective message protocol specifications. Upon calling this function the SBI implementation MUST write the response message data at the offset `0x0` in the shared memory setup by the calling hart @@ -655,19 +653,24 @@ sbi_mpxy_send_message_without_response(uint32_t channel_id, uint32_t message_id, unsigned long message_data_len) ``` - -Send a message to the MPXY channel specified by the `channel_id` parameter. -The `message_id` parameter specifies a message specific to a message protocol +Send a message to the MPXY channel specified by the `channel_id` parameter +without waiting for a message response from the MPXY channel. The `message_id` +parameter specifies the message protocol specific identification of the message to be sent whereas the `message_data_len` parameter represents the length of message data in bytes which is located at the offset `0x0` in the shared memory setup by the calling hart. -This function does not wait for response and returns after successful message -transmission. +This function does not wait for message response from the channel and returns +after successful message transmission. In cases where complete data transfer +requires multiple transmissions, the supervisor software shall send multiple +messages as necessary. Details of such cases can be found in the respective +message protocol specification. -In cases where complete data transfer requires multiple transmissions, the -supervisor software shall issue multiple requests as necessary. Details of such -cases can be found in the respective message protocol specification. +NOTE: The messages which do not have an expected response as per the underlying +message protocol specification are also referred to as posted messages. This +function should be only used for such posted messages. The respective message +protocol specification should define a mechanism to track the status of posted +messages using notification events or some other message with response if required. This function is optional. If this function is implemented, the corresponding bit in the `CHANNEL_CAPABILITY` attribute is set to `1`. From e57e11ef7c99ccfc65e8d2109c30bad03e75839c Mon Sep 17 00:00:00 2001 From: Anup Patel Date: Fri, 17 Jan 2025 23:13:18 +0530 Subject: [PATCH 3/3] ext-mpxy: Add new SBI function to get shared memory size The per-HART shared memory size must be greater than the biggest MSG_DATA_MAX_LEN attribute value across all MPXY channels so add a new separate SBI function to let supervisor software discover it. Signed-off-by: Anup Patel Reviewed-by: Rahul Pathak Reviewed-by: Samuel Holland --- src/ext-mpxy.adoc | 95 +++++++++++++++++++++++++++++------------------ 1 file changed, 59 insertions(+), 36 deletions(-) diff --git a/src/ext-mpxy.adoc b/src/ext-mpxy.adoc index 0272f0f..f9ccf0e 100644 --- a/src/ext-mpxy.adoc +++ b/src/ext-mpxy.adoc @@ -108,7 +108,8 @@ message protocol specification. | 0x00000002 | RO | *Maximum Message Data Length* + -Maximum message data size in bytes supported by the message channel. +Maximum message data size in bytes supported by the message channel to send +or receive message. | MSG_SEND_TIMEOUT | 0x00000003 @@ -134,14 +135,14 @@ wait for response can use this timeout value. ``` [31:6]: Reserved and `0` - [5]: Get Notifications (FID #6) + [5]: Get Notifications (FID #7) Support [4]: Send Message without - Response (FID #5) Support + Response (FID #6) Support [3]: Send Message with - Response (FID #4) Support + Response (FID #5) Support [2]: Events State Support @@ -155,7 +156,7 @@ The SBI implementation only needs to support one method to indicate the availability of notifications, either MSI or SSE. If both are enabled, the MSI is preferred over the SSE event. -If Get Notifications function (FID #6) is not supported then the Events State +If Get Notifications function (FID #7) is not supported then the Events State Support, SSE Event and MSI bits must be `0`. | SSE_EVENT_ID @@ -224,17 +225,17 @@ The reset value of this attribute is `0`. If the message channel supports notification events state data then this attribute can be used to enable state reporting like number of events `RETURNED`, `REMAINING` or `LOST` after a call to -Get Notifications (FID #6) function. +Get Notifications (FID #7) function. The reset value of this attribute is `0`, which means disabled. If supervisor software wants to enable events state reporting, it MUST write `1`. If the events state reporting is not supported by the channel or the -Get Notifications (FID #6) function is not implemented as indicated by +Get Notifications (FID #7) function is not implemented as indicated by the `CHANNEL_CAPABILITY` attribute, then the writes to this attribute will be ignored. More details on events state data are mentioned in the function -Get Notifications (FID #6) description. +Get Notifications (FID #7) description. | RESERVED | 0x0000000C - 0x7fffffff @@ -275,11 +276,30 @@ assigned ID. | Custom vendor specific message protocol |=== -=== Function: Set shared memory (FID #0) +=== Function: Get shared memory size (FID #0) + +```c +struct sbiret sbi_mpxy_get_shmem_size(void) +``` + +Get the shared memory size in number of bytes for sending and receiving messages. + +The shared memory size returned by the SBI implementation MUST satisfy the +following requirements: + +. The shared memory size MUST be same for all HARTs +. The shared memory size MUST be at least 4096 bytes +. The shared memory size MUST be multiple of 4096 bytes +. The shared memory size MUST not be less than the biggest MSG_DATA_MAX_LEN + attribute value across all MPXY channels + +This function always returns SBI_SUCCESS in `sbiret.error` and it will return +the shared memory size in `sbiret.uvalue`. + +=== Function: Set shared memory (FID #1) ```c -struct sbiret sbi_mpxy_set_shmem(unsigned long shmem_size, - unsigned long shmem_phys_lo, +struct sbiret sbi_mpxy_set_shmem(unsigned long shmem_phys_lo, unsigned long shmem_phys_hi, unsigned long flags) ``` @@ -287,13 +307,14 @@ struct sbiret sbi_mpxy_set_shmem(unsigned long shmem_size, Set the shared memory for sending and receiving messages on the calling hart. If both `shmem_phys_lo` and `shmem_phys_hi` parameters are not all-ones -bit-wise then the `shmem_phys_lo` specifies the lower XLEN bits and -`shmem_phys_hi` specifies the upper XLEN bits of the shared memory physical base -address. The `shmem_phys_lo` MUST be 4096 bytes aligned and the `shmem_size` -MUST be multiples of 4096 bytes. +bit-wise then the `shmem_phys_lo` parameter specifies the lower XLEN bits +and the `shmem_phys_hi` parameter specifies the upper XLEN bits of the +shared memory physical base address. The `shmem_phys_lo` parameter MUST +be 4096 bytes aligned and the shared memory size is assumed to be same as +returned by the Get shared memory size function (FID #0). If both `shmem_phys_lo` and `shmem_phys_hi` parameters are all-ones bit-wise -then shared memory is disabled and `shmem_size` parameter is ignored. +then shared memory is disabled. The `flags` parameter specifies configuration for shared memory setup and it is encoded as follows: @@ -318,10 +339,9 @@ based on the passed parameters. | OVERWRITE-RETURN | 0b01 -|Same as `OVERWRITE` mode and additionally after the new shared memory state is -enabled, the old shared memory `shmem_size`, `shmem_phys_lo` and -`shmem_phys_hi` are written in the same order to the new shared memory at -offset `0x0`. + +| Same as `OVERWRITE` mode and additionally after the new shared memory state is +enabled, the old shared memory `shmem_phys_lo` and `shmem_phys_hi` are written +in the same order to the new shared memory at offset `0x0`. + This flag provide provision to software layers in the supervisor software that want to send messages using the shared memory but do not know the shared @@ -360,9 +380,7 @@ The possible error codes returned in `sbiret.error` are below. | SBI_ERR_INVALID_PARAM | The `flags` parameter has invalid value or the bits set are within the reserved -range. + -Or the `shmem_phys_lo` parameter is not 4096 bytes aligned or `shmem_size` -is not multiple of 4096 bytes. +range. Or the `shmem_phys_lo` parameter is not 4096 bytes aligned. | SBI_ERR_INVALID_ADDRESS | The shared memory pointed to by the `shmem_phys_lo` and `shmem_phys_hi` @@ -376,7 +394,7 @@ parameters does not satisfy the requirements described in NOTE: The supervisor software MUST call this function to setup the shared memory first before calling any other function in this extension. -=== Function: Get Channel IDs (FID #1) +=== Function: Get Channel IDs (FID #2) ```c struct sbiret sbi_mpxy_get_channel_ids(uint32_t start_index) @@ -453,7 +471,7 @@ returned in `sbiret.error` are below. | Failed due to other unspecified errors. |=== -=== Function: Read Channel Attributes (FID #2) +=== Function: Read Channel Attributes (FID #3) ```c struct sbiret sbi_mpxy_read_attributes(uint32_t channel_id, @@ -510,7 +528,7 @@ Or the `base_attribute_id` is not valid. | Failed due to other unspecified errors. |=== -=== Function: Write Channel Attributes (FID #3) +=== Function: Write Channel Attributes (FID #4) ```c struct sbiret sbi_mpxy_write_attributes(uint32_t channel_id, @@ -575,7 +593,7 @@ overlaps with standard and message protocol specific attributes. | Failed due to other unspecified errors. |=== -=== Function: Send Message with Response (FID #4) +=== Function: Send Message with Response (FID #5) ```c struct sbiret @@ -645,7 +663,7 @@ Or this function is not supported. | Failed due to other unspecified errors. |=== -=== Function: Send Message without Response (FID #5) +=== Function: Send Message without Response (FID #6) ```c struct sbiret @@ -710,7 +728,7 @@ Or this function is not supported. | Failed due to other unspecified errors. |=== -=== Function: Get Notifications (FID #6) +=== Function: Get Notifications (FID #7) ```c struct sbiret sbi_mpxy_get_notification_events(uint32_t channel_id) @@ -818,38 +836,43 @@ Or this function is not supported. | FID | EID -| sbi_mpxy_set_shmem +| sbi_mpxy_get_shmem_size | 3.0 | 0 | 0x4D505859 -| sbi_mpxy_get_channel_ids +| sbi_mpxy_set_shmem | 3.0 | 1 | 0x4D505859 -| sbi_mpxy_read_attributes +| sbi_mpxy_get_channel_ids | 3.0 | 2 | 0x4D505859 -| sbi_mpxy_write_attributes +| sbi_mpxy_read_attributes | 3.0 | 3 | 0x4D505859 -| sbi_mpxy_send_message_with_response +| sbi_mpxy_write_attributes | 3.0 | 4 | 0x4D505859 -| sbi_mpxy_send_message_without_response +| sbi_mpxy_send_message_with_response | 3.0 | 5 | 0x4D505859 -| sbi_mpxy_get_notification_events +| sbi_mpxy_send_message_without_response | 3.0 | 6 | 0x4D505859 + +| sbi_mpxy_get_notification_events +| 3.0 +| 7 +| 0x4D505859 |===