Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playwright tests and CI integration #647

Merged
merged 82 commits into from
Jun 26, 2024
Merged

Playwright tests and CI integration #647

merged 82 commits into from
Jun 26, 2024

Conversation

dwhitestratiform
Copy link
Contributor

@dwhitestratiform dwhitestratiform commented Jun 11, 2024

Description

First cut at getting initial playwright tests running in github actions.

This PR does the following:

  1. adds 2 new jobs to the deploy workflow
    a) playwright tests - this is self explanatory. A separate job for playwright tests while leaving the cypress and a11y jobs still there
    b) upload reports - this is a custom job that uses various different actions to publish the playwright report to github pages. This report also now includes video files to be displayed from github pages but also can be downloaded.
  2. Creates a nightly cron that runs at midnight UTC to delete playwright reports currently older than 1 day. This needs to be merged to main so that we can test, this is why I kept the threshold so low. Once we verify this works we will want to up the retention to 30 days to match that of how long we keep artifacts in the github workflow. This workflow also has slack notification to the products specific channel if it fails.
  3. Adds an initial set of playwright tests that largely include logging in and doing a simple action as a state and admin user.

Things worth noting -

  • The github pages have no reference to the branch that created it only the timestamp as a unique identifier. I considered how this would work in a destroy action and I think that gets messy pretty quickly.

  • The github action that publishes the report to github pages does so by pushing the report in a folder named with a timestamp to the gh-pages branch which is a protected branch in github.

  • The delete workflow runs at midnight, it pulls the gh-pages branch and deletes folders older than "n" days ... in this pr 1.

  • The delete workflow has to live in both the main branch as well as the gh-pages branch so that was checked in directly to the branch ... also important. the "rm_old_folders.py" file needs to live on the gh-pages branch. This whole workflow is a weird corner of github. the gh-pages branch is a reserved github branch that doesnt contain a copy of our source code like any other branch off of the integration branch. So when we put files there I typically do it through the github UI.

  • I kept cypress and a11y living side by side with Playwright. Playwright takes less time to run currently and I think having this in our main pipeline will give more visibility to positives and negatives of playwright than any POC branch could. I would propose merging this and working through any current pain points for a true evaluation. If we decide we do not like playwright, we have not touched the other tests and can simply rip Playwright out.

Related ticket(s)

https://jiraent.cms.gov/browse/CMDCT-3722
https://jiraent.cms.gov/browse/CMDCT-3728

How to test

merge and watch the pipeline, also the delete pages workflow is untested because we cant test it until it gets to main.

here is an example of a github deploy workflow that ran tests and published the report to github pages
https://github.com/Enterprise-CMCS/macpro-mdct-mfp/actions/runs/9569595722

Notes

learning - https://blog.martioli.com/publish-your-playwright-reports-to-github-pages/
I loosely followed this guide for the github pages integration


Pre-review checklist

  • I have added thorough tests, if necessary
  • I have updated relevant documentation, if necessary
  • I have performed a self-review of my code
  • I have manually tested this PR in the deployed cloud environment

Pre-merge checklist

Review

  • Design: This work has been reviewed and approved by design, if necessary
  • Product: This work has been reviewed and approved by product owner, if necessary

Security

If either of the following are true, notify the team's ISSO (Information System Security Officer).

  • These changes are significant enough to require an update to the SIA.
  • These changes are significant enough to require a penetration test.

convert to a different template: test → val | val → prod

@dwhitestratiform dwhitestratiform changed the title DRAFT -- Playwright CI DRAFT -- Playwright tests and CI integration Jun 18, 2024
@dwhitestratiform dwhitestratiform marked this pull request as ready for review June 25, 2024 16:56
Copy link

codeclimate bot commented Jun 25, 2024

Code Climate has analyzed commit 2f5efed and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (90% is the threshold).

This pull request will bring the total coverage in the repository to 92.4% (0.0% change).

View more on Code Climate.

Copy link
Contributor

@jessabean jessabean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am clearly biased, but I approve this message

@dwhitestratiform dwhitestratiform added the ready for review Ready for all the reviews! label Jun 25, 2024
@jessabean jessabean merged commit 92dd2ae into main Jun 26, 2024
21 checks passed
@jessabean jessabean deleted the doug-playwright-ci branch June 26, 2024 15:57
jessabean added a commit that referenced this pull request Jun 26, 2024
@dwhitestratiform dwhitestratiform removed the ready for review Ready for all the reviews! label Jul 1, 2024
@dwhitestratiform dwhitestratiform changed the title DRAFT -- Playwright tests and CI integration Playwright tests and CI integration Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants