actions: keep scripts out of the workflow scope

[gustavovalverde]

Motivation

Some workflows are script intensive, making the workflows unreadable.

Specifications

A good approach to handle this would be moving all the scripts to multiple files outside the workflow, or create a GitHub Action to execute them in a reusable manner.

Complex Code or Requirements

• Define a place to hold all the script files
• Keep linting working on these files
• Use a name convention for script files

[teor2345]

The other advantage of this is that it's easier to test the scripts locally outside the workflows.

[mpguerra]

Hey team! Please add your planning poker estimate with Zenhub @arya2 @gustavovalverde @oxarbitrage @teor2345 @upbqdn

[teor2345]

This is potentially a very large task, depending on how much work we put into de-duplicating or parameterising our scripts.

What's the intended scope of this ticket? Is it just moving the existing scripts into files? Or will there be a lot of refactoring?

(CI refactoring is risky and often breaks things, because it's difficult to test before merging.)

[gustavovalverde]

Is it just moving the existing scripts into files? Or will there be a lot of refactoring?

It's basically moving scripts into .sh files, but just doing that can be considered a lot of refactoring.

[teor2345]

Is it just moving the existing scripts into files? Or will there be a lot of refactoring?

It's basically moving scripts into .sh files, but just doing that can be considered a lot of refactoring.

I guess what I meant was: are we just moving the script text into files and calling those files? Or are we modifying the functionality of the scripts at the same time?

I think it would reduce the risk and scope of this ticket to restrict it to copy-pasting scripts, and also do it in stages. Maybe we could try it with a small workflow or a single script first, to check if there are any issues?

[gustavovalverde]

Yeah, it should basically be copy-pasting, and we can surely test with the less criticals first. I've anyways remove this from the Sprint backlog as we have these dependencies:

• ci: use self-hosted runners for CI pipelines #5572
• actions: segregate workflows based on their purpose #6167

[teor2345]

In PR #8132 I reverted this PR because it has been causing internal GitHub errors. Unfortunately no-one picked up on this during the review process or after it merged.

[teor2345]

When we re-do this PR, we need to manually go to the ci-integration-tests-gcp.yml workflow run, scroll down, and check it for errors. These errors don't reliably show up in the CI summary or auto-opened tickets.

For example, here is a successful run:
https://github.com/ZcashFoundation/zebra/actions/runs/7188577552?pr=8080

And here is a failed run:
https://github.com/ZcashFoundation/zebra/actions/runs/7381289342