Move results format specification out of L2 metadata level (#3421)

This should make the result format documentation more visible, more
prominent. It is important part of tmt integration, and deserves its own
place in specification, similar to what HW requirements have.

docs/guide.rst

@@ -971,7 +971,7 @@ to their services, synchronization, etc.
 
 tmt fully supports one test being executed multiple times. This is
 especially visible in the format of results, see
-:ref:`/spec/plans/results`. Every test is assigned a "serial
+:ref:`/spec/results`. Every test is assigned a "serial
 number", if the same test appears in multiple discover phases, each
 instance would be given a different serial number. The serial number
 and the guest from which a result comes from are then saved for each

docs/releases.rst

@@ -29,7 +29,7 @@ support for submitting jobs on behalf of a group through the
 ``beaker-job-group`` key. The submitting user must be a member of
 the given job group.
 
-The ``note`` field of tmt :ref:`/spec/plans/results` changes from
+The ``note`` field of tmt :ref:`/spec/results` changes from
 a string to a list of strings, to better accommodate multiple notes.
 
 The ``Node`` alias for the ``Core`` class has been dropped as it
@@ -256,7 +256,7 @@ committish reference, either branch, tag, git-describe, or if all
 fails the commit hash.  You may encounter this in the verbose log
 of ``tmt tests show`` or plan/test imports.
 
-:ref:`Result specification</spec/plans/results>` now defines
+:ref:`Result specification</spec/results>` now defines
 ``original-result`` key holding the original outcome of a test,
 subtest or test checks. The effective outcome, stored in
 ``result`` key, is computed from the original outcome, and it is
@@ -286,7 +286,7 @@ tmt-1.36.0
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 tmt will now emit a warning when :ref:`custom test results</spec/tests/result>`
-file does not follow the :ref:`result specification</spec/plans/results>`.
+file does not follow the :ref:`result specification</spec/results>`.
 
 We have started to use ``warnings.deprecated`` to advertise upcoming
 API deprecations.
@@ -621,7 +621,7 @@ The **avc** :ref:`/spec/tests/check` allows to detect avc denials
 which appear during the test execution.
 
 A new ``skip`` custom result outcome has been added to the
-:ref:`/spec/plans/results` specification.
+:ref:`/spec/results` specification.
 
 All context :ref:`/spec/context/dimension` values are now handled
 in a case insensitive way.

docs/scripts/generate-stories.py

@@ -38,6 +38,7 @@ def __getattr__(cls, name: str) -> 'Mock':
     '/spec/stories': 'Stories',
     '/spec/context': 'Context',
     '/spec/hardware': 'Hardware',
+    '/spec/results': 'Results'
     }
 
 

docs/spec.rst

@@ -54,4 +54,5 @@ Level 3: Stories
     spec/stories
     spec/context
     spec/hardware
+    spec/results
     spec/lint

spec/plans/execute.fmf

@@ -20,7 +20,7 @@ description: |
 
     In each plan, the execute step must produce a ``results.yaml`` file
     with results for executed tests. The format of the file is described
-    at :ref:`/spec/plans/results`.
+    at :ref:`/spec/results`.
 
 /upgrade:
     summary: Perform system upgrades during testing

spec/plans/results.fmf → spec/results.fmf

@@ -1,6 +1,6 @@
-summary: Define format of on-disk storage of results
-title: Results Format
-order: 90
+story:
+    To integrate tmt with other tools, tmt should standardize and
+    document the format in which test results are saved on storage.
 
 description: |
     The following text defines a YAML file structure tmt uses for storing
@@ -153,7 +153,7 @@ description: |
           check:
             ...
 
-    .. _/spec/plans/results/outcomes:
+    .. _/spec/results/outcomes:
 
     The ``result`` key can have the following values:
 

spec/tests/result.fmf

@@ -23,7 +23,7 @@ description: |
         test needs to create its own ``results.yaml`` or
         ``results.json`` file in the ``${TMT_TEST_DATA}``
         directory. The format of the file, notes and detailed examples
-        are documented at :ref:`/spec/plans/results`.
+        are documented at :ref:`/spec/results`.
     restraint
         handle ``tmt-report-result``, ``rstrnt-report-result`` and
         ``rhts-report-result`` commands as a separate test and

tmt/checks/avc.py

@@ -277,9 +277,9 @@ class AvcDenials(CheckPlugin[Check]):
         To work correctly, the check requires SELinux to be enabled on the
         guest, and ``auditd`` must be running. Without SELinux, the
         check will turn into no-op, reporting
-        :ref:`skip</spec/plans/results/outcomes>` result, and
+        :ref:`skip</spec/results/outcomes>` result, and
         without ``auditd``, the check will discover no AVC denials,
-        reporting :ref:`pass</spec/plans/results/outcomes>`.
+        reporting :ref:`pass</spec/results/outcomes>`.
 
         If the test manipulates ``auditd`` or SELinux in general, the
         check may report unexpected results.