Implement check `result` key, checks affect test result by default#3239

Merged

psss merged 35 commits intoteemtee:mainfrom martinhoyer:check-results

Oct 25, 2024

+550-67

Commits on Oct 25, 2024

Implement check result key feature

martinhoyer
authored and
psss
committed
Add check-result spec and release note

martinhoyer
authored and
psss
committed
Move and modify check schema

martinhoyer
authored and
psss
committed
Fix doc generation

martinhoyer
authored and
psss
committed
Add is_enum to plugins doc generate script

happz
authored and
psss
committed
Fix result interpretation, notes, tests

martinhoyer
authored and
psss
committed
Do not interpret any result with CUSTOM interpret

martinhoyer
authored and
psss
committed
Addressing comments

martinhoyer
authored and
psss
committed
Add note for running one check multiple times

martinhoyer
authored and
psss
committed
Add description to check-result tests

martinhoyer
authored and
psss
committed
fixup! Addressing comments

martinhoyer
authored and
psss
committed
Move the result definition list to spec

martinhoyer
authored and
psss
committed
Addressing comments for tests, spec

martinhoyer
authored and
psss
committed
[PoC] use metadata.choices for the enums

martinhoyer
authored and
psss
committed
Improve xfail check result handling

martinhoyer
authored and
psss
committed
A minor spec wording adjustment
psss
committed
Small release notes adjustment, fix a typo
psss
committed
Add missing coversions to and from spec
psss
committed
Define the to_minimal_spec() method as well
psss
committed
Improve tests

martinhoyer
authored and
psss
committed
Add missing parameter in docstring

martinhoyer
authored and
psss
committed
Fix avc, dmesg tests expecting exit code 0

martinhoyer
authored and
psss
committed
Addressing comments

martinhoyer
authored and
psss
committed
fixup! Fix avc, dmesg tests expecting exit code 0

martinhoyer
authored and
psss
committed
Addressing yet more valuable comments

martinhoyer
authored and
psss
committed
Aggregating check results, attempt 1

martinhoyer
authored and
psss
committed
Unit tests attempt 1

martinhoyer
authored and
psss
committed
Aggregating check results first

martinhoyer
authored and
psss
committed
Aggregate check results attempt 2

martinhoyer
authored and
psss
committed
Aggregate check results attempt 3
psss
committed
Adjust tests accordingly
psss
committed
Adjust tests accordingly
psss
committed
Drop aggregate_check_results and related test
psss
committed
Add warning, fix typos, extend release note a bit
psss
committed
A couple of necessary test output check adjustments
psss
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement check `result` key, checks affect test result by default#3239

Commits on Oct 25, 2024

Implement check result key feature

Add check-result spec and release note

Move and modify check schema

Fix doc generation

Add is_enum to plugins doc generate script

Fix result interpretation, notes, tests

Do not interpret any result with CUSTOM interpret

Addressing comments

Add note for running one check multiple times

Add description to check-result tests

fixup! Addressing comments

Move the result definition list to spec

Addressing comments for tests, spec

[PoC] use metadata.choices for the enums

Improve xfail check result handling

A minor spec wording adjustment

Small release notes adjustment, fix a typo

Add missing coversions to and from spec

Define the `to_minimal_spec()` method as well

Improve tests

Add missing parameter in docstring

Fix avc, dmesg tests expecting exit code 0

Addressing comments

fixup! Fix avc, dmesg tests expecting exit code 0

Addressing yet more valuable comments

Aggregating check results, attempt 1

Unit tests attempt 1

Aggregating check results first

Aggregate check results attempt 2

Aggregate check results attempt 3

Adjust tests accordingly

Adjust tests accordingly

Drop `aggregate_check_results` and related test

Add warning, fix typos, extend release note a bit

A couple of necessary test output check adjustments