diff --git a/docs/log-data-in-reportportal/HowToReportAttributesToReportPortal.mdx b/docs/log-data-in-reportportal/HowToReportAttributesToReportPortal.mdx new file mode 100644 index 0000000000..ff4c11165b --- /dev/null +++ b/docs/log-data-in-reportportal/HowToReportAttributesToReportPortal.mdx @@ -0,0 +1,139 @@ +# How to report attributes to ReportPortal + +Attributes are essential for categorizing, filtering, and analyzing test data. This article will guide you through the process of reporting attributes to our test results reporting service. + +Be sure to check out our dedicated article for [best practices on using attributes](/work-with-reports/LaunchesTestItemsAttributes) effectively. + +## Parameters of attributes + +* Format – Key : Value +* Key – any string value, optional field +* Value – any string value, mandatory field +* The maximum limit is 512 symbols each for both key and value + +Multiple values can be associated with a single key. For example, if we have a `jira_id` key, the value can differ based on the Jira ticket. + + + +:::important +The only valid use of the colon is to split the key and value.
+When a colon is present in the key or in the value, it breaks the string into two parts. +::: + +### Colon scenarios + +**1.** Key and value reported, colon used in key only. + +**Example:**
+```{"key": "la:bel", "value": "smoke"}``` + +**Behavior on UI:**
+Key: la
+Value: bel + +The original value "smoke" is not transmitted at all. + +**2.** Key and value reported, colon used in both key and value. + +**Example:**
+```{"key": "la:bel", "value": "smo:ke"}``` + +**Behavior on UI:**
+Key: la
+Value: bel + +The original value "smoke" is not transmitted at all. + +**3.** Key and value reported, colon used in value only. + +**Example:**
+```{"key": "label", "value": "smo:ke"}``` + +**Behavior on UI:**
+Key: label
+Value: smo + +The value is truncated at the colon, and only the part before the colon is transmitted. + +**4.** Value reported, colon used in value only. + +**Example:**
+```{"value": "smo:ke"}``` + +**Behavior on UI:**
+Value: smo + +The value is truncated at the colon, and only the part before the colon is transmitted. + +## Adding attributes via test framework integration + +### JavaScript + +Here is an example of adding attributes for a Playwright agent. + +You can add attributes for the Launch in ReportPortal [agent config](https://github.com/reportportal/examples-js/blob/main/example-playwright/playwright.config.ts#L8): + + + +You can add attributes for the test item in your [test file](https://github.com/reportportal/examples-js/blob/main/example-playwright/tests/rp-features/attributes-and-description.spec.ts#L33): + + + +### .NET + +Attributes in .NET agents can be added declaratively or at runtime. To add launch attributes via standard [ReportPortal.json config file](https://github.com/reportportal/example-net-nunit/blob/68f596712e153fd67e45cbd479a9aa40a7d5423a/src/Example/ReportPortal.config.json#L15): + +```json +{ + "launch": { + "attributes": ["platform:unix", "arch:x64" ] + } +} +``` + +And to add test attributes ([sample for NUnit](https://github.com/reportportal/example-net-nunit/blob/68f596712e153fd67e45cbd479a9aa40a7d5423a/src/Example/Tests/Class1.cs#L142)): + +```csharp +[Test, Category("browser:chrome")] +public void Test1() +{ + ReportPortal.Shared.Context.Current.Metadata.Attributes.Add("browser", "chrome"); +} +``` + +In Gherkin: + +``` +@browser:chrome +Scenario: Scenario1 +``` + +## Adding attributes via API + +You can initiate (launch, suite, test, or step) with one set of attributes and finish with a different set. + +Importantly, the new attributes do not overwrite the old ones; instead, they are added. + + + +## System attributes + +In the ReportPortal, there is an opportunity to influence specific processes and data processing rules. To this end, ReportPortal supports system attributes. These attributes can be applied to the selected test items and trigger ReportPortal to take certain actions. System attributes are not visible on the UI (for that, the attribute should be sent with `system: true`). + +### immediateAutoAnalysis + +If `immediateAutoAnalysis=true`, then each test item will be analyzed right after it is completed, without waiting for the whole launch to finish. + + + +### immediatePatternAnalysis + +When `immediatePatternAnalysis=true`, then each test item will be analyzed right after it is completed, without waiting for the whole launch to finish. + + + +### skippedIsNotIssue + +If `skippedIsNotIssue=true` during launch import, then all test items with the SKIPPED status will be processed without applying a `To Investigate` defect type. + + diff --git a/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes1.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes1.png new file mode 100644 index 0000000000..a4604b2cd0 Binary files /dev/null and b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes1.png differ diff --git a/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes2.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes2.png new file mode 100644 index 0000000000..e041b08837 Binary files /dev/null and b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes2.png differ diff --git a/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes3.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes3.png new file mode 100644 index 0000000000..e5a41e2586 Binary files /dev/null and b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes3.png differ diff --git a/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes4.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes4.png new file mode 100644 index 0000000000..c4b9ba47ba Binary files /dev/null and b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes4.png differ diff --git a/docs/work-with-reports/img/system-attributes/SystemAttributes1.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes5.png similarity index 100% rename from docs/work-with-reports/img/system-attributes/SystemAttributes1.png rename to docs/log-data-in-reportportal/img/report-attributes/ReportAttributes5.png diff --git a/docs/work-with-reports/img/system-attributes/SystemAttributes2.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes6.png similarity index 100% rename from docs/work-with-reports/img/system-attributes/SystemAttributes2.png rename to docs/log-data-in-reportportal/img/report-attributes/ReportAttributes6.png diff --git a/docs/work-with-reports/img/system-attributes/SystemAttributes3.png b/docs/log-data-in-reportportal/img/report-attributes/ReportAttributes7.png similarity index 100% rename from docs/work-with-reports/img/system-attributes/SystemAttributes3.png rename to docs/log-data-in-reportportal/img/report-attributes/ReportAttributes7.png diff --git a/docs/work-with-reports/LaunchesTestItemsAttributes.mdx b/docs/work-with-reports/LaunchesTestItemsAttributes.mdx index 8028637f4c..de3c027fd2 100644 --- a/docs/work-with-reports/LaunchesTestItemsAttributes.mdx +++ b/docs/work-with-reports/LaunchesTestItemsAttributes.mdx @@ -7,6 +7,8 @@ sidebar_label: Launches/test items attributes For more convenient and effective use of our test report dashboard, we recommend paying attention to attributes. +## What are attributes? + Attributes are specific data points that can be included at all levels in ReportPortal: **Launch** @@ -27,88 +29,13 @@ Attributes are specific data points that can be included at all levels in Report 2. Attributes are used for filtering. 3. You can create a filter based on the attribute and then build widgets using this filter. -## Parameters of attributes - -* Format – Key : Value -* Key – any string value, optional field -* Value – any string value, mandatory field -* The maximum limit is 512 symbols each for both key and value. - -Multiple values can be associated with a single key. For example, if we have a `jira_id` key, the value can differ based on the Jira ticket. - - - -:::important -The only valid use of the colon is to split the key and value.
-When a colon is present in the key or in the value, it breaks the string into two parts. -::: - -### Colon scenarios - -**1.** Key and value reported, colon used in key only. - -**Example:**
-```{"key": "la:bel", "value": "smoke"}``` - -**Behavior on UI:**
-Key: la
-Value: bel - -The original value "smoke" is not transmitted at all. - -**2.** Key and value reported, colon used in both key and value. - -**Example:**
-```{"key": "la:bel", "value": "smo:ke"}``` - -**Behavior on UI:**
-Key: la
-Value: bel - -The original value "smoke" is not transmitted at all. - -**3.** Key and value reported, colon used in value only. - -**Example:**
-```{"key": "label", "value": "smo:ke"}``` - -**Behavior on UI:**
-Key: label
-Value: smo - -The value is truncated at the colon, and only the part before the colon is transmitted. - -**4.** Value reported, colon used in value only. - -**Example:**
-```{"value": "smo:ke"}``` - -**Behavior on UI:**
-Value: smo +You can [report attributes to ReportPortal](/log-data-in-reportportal/HowToReportAttributesToReportPortal) via test framework integrations or via API and add or edit them via the UI. For that, click on the pencil icon next to the desired launch/test item. Then click on the existing attribute to edit the key and value. -The value is truncated at the colon, and only the part before the colon is transmitted. + -## Adding attributes + -Attributes can be added in the following ways: - -1. Via test reporting. - -You can initiate (launch, suite, test, or step) with one set of attributes and finish with a different set. - -Importantly, the new attributes do not overwrite the old ones; instead, they are added. - - - -2. Via UI (manually). - -To add or edit attributes, click on the `pencil` icon next to the desired launch/test item. Then click on the `Add new` link or click on the existing attribute to edit the key and value. - - - - - - + ## Best practices for using attributes @@ -155,7 +82,9 @@ Below you can find some examples of how attributes can describe launch/test item -## Use Case: using attributes for filtering +## Use Cases + +### Using attributes for filtering Let's consider the following example. In the launch below, there are 31 Failed tests. @@ -175,9 +104,9 @@ As a result of the investigation, we found that the least number of failures occ Thus, we understand that we need to focus on tests with controller type `contr_launch`. -## Use Case: using attributes for creating widgets +### Using attributes for creating widgets -Widgets allow you to visually track metrics for launches. We can define these metrics using filters that can be created based on attributes. For example, to determine the growth trend of summary statistics of launches with the same attribute key, let's construct a [Cumulative trend chart widget](/dashboards-and-widgets/CumulativeTrendChart). +Widgets allow you to visually track [qa and test automation metrics metrics](/dashboards-and-widgets/ReportingAndMetricsInReportPortal/#qa-and-test-automation-metrics-in-reportportal) for launches. We can define these metrics using filters that can be created based on attributes. For example, to determine the growth trend of summary statistics of launches with the same attribute key, let's construct a [Cumulative trend chart widget](/dashboards-and-widgets/CumulativeTrendChart). There are some launches with the `smoke` attribute. @@ -195,6 +124,26 @@ On the screenshot below, we can observe a growth in statistics on the platforms +Based on attributes specified for test items, you can also build a [Component Health Check widget](/dashboards-and-widgets/ComponentHealthCheck/) to visually represent the passing rate of different application components. + +For example, if there are some launches with the `demo` attribute, you can create a filter using this attribute. + + + +Next, construct a Component Health Check widget using this filter. Specify the components, functionality, platforms, or other criteria you want to monitor by defining the relevant attributes across different levels. + + + +As a result: + +- On the first level, you can view the passing rate for each build. + + + +- On the second level, you can see which platform has the highest number of failed test cases, helping you identify areas that may need more attention. + + + Therefore, properly using attributes in ReportPortal can greatly optimize test management and assist in providing clearer visualizations of trends through widgets. -There are also [system attributes](/work-with-reports/SystemAttributes) that allow to extend the functionality of ReportPortal. +There are also [system attributes](/log-data-in-reportportal/HowToReportAttributesToReportPortal#system-attributes) that allow to extend the functionality of ReportPortal. diff --git a/docs/work-with-reports/SystemAttributes.mdx b/docs/work-with-reports/SystemAttributes.mdx deleted file mode 100644 index 5c76cb69a1..0000000000 --- a/docs/work-with-reports/SystemAttributes.mdx +++ /dev/null @@ -1,26 +0,0 @@ ---- -sidebar_position: 5 -sidebar_label: System attributes ---- - -# System attributes - -In the ReportPortal, there is an opportunity to influence specific processes and data processing rules. To this end, ReportPortal supports system attributes. These attributes can be applied to the selected test items and trigger ReportPortal to take certain actions. System attributes are not visible on the UI (for that, the attribute should be sent with `system: true`). - -## immediateAutoAnalysis - -If `immediateAutoAnalysis=true`, then each test item will be analyzed right after it is completed, without waiting for the whole launch to finish. - - - -## immediatePatternAnalysis - -When `immediatePatternAnalysis=true`, then each test item will be analyzed right after it is completed, without waiting for the whole launch to finish. - - - -## skippedIsNotIssue - -If `skippedIsNotIssue=true` during launch import, then all test items with the SKIPPED status will be processed without applying a `To Investigate` defect type. - - diff --git a/docs/work-with-reports/img/launches-test-items-attributes/Attributes5.png b/docs/work-with-reports/img/launches-test-items-attributes/Attributes5.png index a4604b2cd0..6e64cc7099 100644 Binary files a/docs/work-with-reports/img/launches-test-items-attributes/Attributes5.png and b/docs/work-with-reports/img/launches-test-items-attributes/Attributes5.png differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/Attributes6.png b/docs/work-with-reports/img/launches-test-items-attributes/Attributes6.png index c4b9ba47ba..9cdddae6f7 100644 Binary files a/docs/work-with-reports/img/launches-test-items-attributes/Attributes6.png and b/docs/work-with-reports/img/launches-test-items-attributes/Attributes6.png differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/Attributes7.png b/docs/work-with-reports/img/launches-test-items-attributes/Attributes7.png index 58d0385234..ac3d4a32f1 100644 Binary files a/docs/work-with-reports/img/launches-test-items-attributes/Attributes7.png and b/docs/work-with-reports/img/launches-test-items-attributes/Attributes7.png differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/Attributes8.png b/docs/work-with-reports/img/launches-test-items-attributes/Attributes8.png deleted file mode 100644 index 90c589cd69..0000000000 Binary files a/docs/work-with-reports/img/launches-test-items-attributes/Attributes8.png and /dev/null differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/Attributes9.png b/docs/work-with-reports/img/launches-test-items-attributes/Attributes9.png deleted file mode 100644 index 59237c92fd..0000000000 Binary files a/docs/work-with-reports/img/launches-test-items-attributes/Attributes9.png and /dev/null differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck1.png b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck1.png new file mode 100644 index 0000000000..8252c1c028 Binary files /dev/null and b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck1.png differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck2.png b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck2.png new file mode 100644 index 0000000000..553731874a Binary files /dev/null and b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck2.png differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck3.png b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck3.png new file mode 100644 index 0000000000..955421c92b Binary files /dev/null and b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck3.png differ diff --git a/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck4.png b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck4.png new file mode 100644 index 0000000000..3d8e49b3f0 Binary files /dev/null and b/docs/work-with-reports/img/launches-test-items-attributes/ComponentHealthCheck4.png differ