Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve installer tests compatibility with pit #15308

Merged
merged 1 commit into from
Sep 3, 2024

Conversation

rmynar
Copy link
Contributor

@rmynar rmynar commented Jun 4, 2024

This change improves compatibility with PIT testing.

  • added RHEL_SOURCE variable to robottelo.conf to choose which repositories to use. ga - use released RHEL, internal - use custom RHEL (e.g. release candidate)
  • removed 'beta' from available sources of satellite and capsule as we don't use beta releases any more
  • updated validators

We can choose source repositories for RHEL and Satellite. Following combinations of sat@rhel are possible:
internal@internal - pre-release testing
ga@internal - PIT testing
internal@ga - common testing
ga@ga - invalid combination (we don't care about retesting already published products)
The same applies for Capsule. It's also possible to use different version of Capsule and Satellite (e.g. n-1 testing)

  • reworked satellite installation - use RHEL and Satellite sources as defined by robottelo.rhel_source and server.version
  • reworked capsule installation - setup RHEL and Capsule sources on Satellite, register capsule as host and perform capsule installation

See SAT-26896

@rmynar rmynar self-assigned this Jun 4, 2024
@ogajduse
Copy link
Member

trigger: test-robottelo
pytest: tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation

@rmynar rmynar added 6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 labels Jun 17, 2024
@rmynar rmynar marked this pull request as ready for review June 17, 2024 12:39
@rmynar rmynar requested review from a team as code owners June 17, 2024 12:39
@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 7403
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation --external-logging
Test Result : ============ 1 failed, 256 warnings, 1 error in 2759.16s (0:45:59) =============

@Satellite-QE Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label Jun 17, 2024
@rmynar
Copy link
Contributor Author

rmynar commented Jun 18, 2024

trigger: test-robottelo
pytest: tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 7419
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation --external-logging
Test Result : ================= 2 passed, 404 warnings in 4529.02s (1:15:29) =================

@Satellite-QE Satellite-QE added PRT-Passed Indicates that latest PRT run is passed for the PR and removed PRT-Failed Indicates that latest PRT run is failed for the PR labels Jun 18, 2024
@Satellite-QE Satellite-QE removed the PRT-Passed Indicates that latest PRT run is passed for the PR label Jun 25, 2024
@rmynar rmynar added the CherryPick PR needs CherryPick to previous branches label Jun 26, 2024
@rmynar rmynar marked this pull request as draft June 26, 2024 13:50
@rmynar rmynar force-pushed the pit-installer branch 2 times, most recently from 49844fa to 2233a40 Compare June 26, 2024 16:34
Copy link
Contributor

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structure wise I it feels like it does a lot. Would it make sense to split it up into a helper? Like a fixture that creates the product and repositories. Then create an activation key for those.

tests/foreman/installer/test_installer.py Show resolved Hide resolved
Comment on lines 335 to 261
for task in sync_tasks:
logger.info(f'Waiting for task {task['id']}')
sat_non_default_install.wait_for_tasks(
search_query=(f'id = {task["id"]}'),
poll_timeout=1800,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API wise I'd expect that wait_for_tasks doesn't need to be used in a loop. Would it be better to have a more complex search query that waits for all tasks? This probably works:

Suggested change
for task in sync_tasks:
logger.info(f'Waiting for task {task['id']}')
sat_non_default_install.wait_for_tasks(
search_query=(f'id = {task["id"]}'),
poll_timeout=1800,
)
sat_non_default_install.wait_for_tasks(
search_query=(f'id ^ {",".join(task["id"] for task in sync_tasks}'),
poll_timeout=1800,
)

But performance wise it may actually be worse. Perhaps you'd also need to pass in a status filter, but IMHO wait_for_tasks(tasks=sync_tasks) should just do that without needing to think about it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, the function has its own loop so it can accept multiple tasks in search query.

2. Configure capsule repos
3. Install and enable fapolicyd
4. Enable capsule module
1. Use Satellite with fapolicyd enabled (non-default)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this uses sat_non_default_install but I really disagree that we should be testing with --foreman-rails-cache-store type:file. If we want proper integration testing we should be testing a basic default Satellite instead of the edge case.

If anything, the test that actually wants to test Satellite with non-Redis should explicitly configure Satellite that way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to create another fixture with default setup but including fapolicyd. Let's consider this as a workaround I hope it can be done better (e.g. parametrized), but it's out of scope of this PR.

)

# setup source repositories
if settings.server.version.source == "ga":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"ga" ?!??
Wouldn't be "cdn" better ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a remainder of historical state when there were two versions available on cdn - ga and beta. Beta is no more used. However it still seems logical to me - we have internal source or generally available source. In case of renaming I would suggest different naminginternal/public.

@rmynar rmynar force-pushed the pit-installer branch 3 times, most recently from f28d411 to e435c17 Compare July 24, 2024 14:41
@rmynar rmynar marked this pull request as ready for review July 24, 2024 14:58
@rmynar rmynar requested a review from a team as a code owner July 24, 2024 14:58
Copy link
Contributor

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With my limited robottelo knowledge, it looks good to me.

@rmynar
Copy link
Contributor Author

rmynar commented Jul 25, 2024

trigger: test-robottelo
pytest: tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 8067
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation --external-logging
Test Result : ================== 7 warnings, 2 errors in 208.09s (0:03:28) ===================

@Satellite-QE Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label Aug 9, 2024
@rmynar
Copy link
Contributor Author

rmynar commented Aug 12, 2024

trigger: test-robottelo
pytest: tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 8079
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation --external-logging
Test Result : ================= 2 passed, 522 warnings in 4548.05s (1:15:48) =================

@Satellite-QE Satellite-QE added PRT-Passed Indicates that latest PRT run is passed for the PR and removed PRT-Failed Indicates that latest PRT run is failed for the PR labels Aug 12, 2024
@SatelliteQE SatelliteQE deleted a comment from Satellite-QE Aug 12, 2024
@SatelliteQE SatelliteQE deleted a comment from Satellite-QE Aug 12, 2024
@jameerpathan111 jameerpathan111 marked this pull request as draft August 20, 2024 14:25
@jameerpathan111
Copy link
Contributor

jameerpathan111 commented Aug 20, 2024

@rmynar I converted pr to draft for now, please mark it ready for review once you're done with changes.

@jyejare jyejare added the 6.16.z Introduced in or relating directly to Satellite 6.16 label Aug 22, 2024
@ogajduse
Copy link
Member

trigger: test-robottelo
pytest: tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 8310
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/installer/test_installer.py::test_satellite_installation tests/foreman/installer/test_installer.py::test_capsule_installation --external-logging
Test Result : ================= 2 passed, 519 warnings in 4686.80s (1:18:06) =================

@rmynar rmynar marked this pull request as ready for review September 2, 2024 11:44
@ogajduse ogajduse merged commit 3d020bc into SatelliteQE:master Sep 3, 2024
15 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 3, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Sep 19, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Sep 20, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Sep 20, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Sep 20, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Sep 23, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Sep 23, 2024
rmynar added a commit to rmynar/robottelo that referenced this pull request Oct 8, 2024
jyejare pushed a commit to jyejare/robottelo that referenced this pull request Oct 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 6.16.z Introduced in or relating directly to Satellite 6.16 CherryPick PR needs CherryPick to previous branches PRT-Passed Indicates that latest PRT run is passed for the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants