Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 1288: Adding CodeQL and Ruff as additional workflow items #1307

Draft
wants to merge 10 commits into
base: develop
Choose a base branch
from
100 changes: 100 additions & 0 deletions .github/workflows/codeql.yml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is missing the codeql directory, which is the configuration, from here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make sure to add them in.

Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# For more information see:
# https://nasa.github.io/scrub/
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [main, develop]
pull_request:
# The branches below must be a subset of the branches above
branches: [develop]
schedule:
# default branch on sundays at 5a
- cron: '0 5 * * 0'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: write
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
# CodeQL supports ['cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby']
# Learn more about CodeQL language support at https://git.io/codeql-language-support
language: ['python']

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
#config-file: ./.github/workflows/codeql/codeql-config.yml
languages: ${{ matrix.language }}
queries: security-and-quality, security-extended
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3

- name: Post-Process Output
run: |
python3 -m pip install nasa-scrub

results_dir=`realpath ${{ github.workspace }}/../results`
sarif_files=`find $results_dir -name '*.sarif'`

for sarif_file in $sarif_files
do
output_file="$results_dir/$(basename $sarif_file .sarif).scrub"

python3 -m scrub.tools.parsers.translate_results $sarif_file $output_file ${{ github.workspace }} scrub
done

python3 -m scrub.tools.parsers.csv_parser $results_dir

echo "RESULTS_DIR=$results_dir" >> $GITHUB_ENV


- name: Upload CodeQL Artifacts
uses: actions/upload-artifact@v4
with:
name: codeql-artifacts
path: ${{ env.RESULTS_DIR }}
if-no-files-found: error
overwrite: true
retention-days: 15
69 changes: 69 additions & 0 deletions .github/workflows/ruff.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to disable certain Ruff checks in the
# "Analyze" configuration block below.
#
# For more information see:
# https://nasa-ammos.github.io/slim/docs/guides/software-lifecycle/application-starter-kits/python-starter-kit/
#
# ******** NOTE ********
# Ruff is a Python-based linter that works to evaluate Python code.
#
name: "Ruff"

on:
push:
branches: [main, develop]
pull_request:
# The branches below must be a subset of the branches above
branches: [develop]

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: write
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it need write access? That seems like we're handing the VM a lot of power.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not as it's merely analyzing the code. I'm assuming I should take away its permission to write during security-events as well?

contents: read
security-events: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Upgrade tooling
run: |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to use the build command to make this happen? Maybe through a virtual environment so there's not so much installation stuff (that seems a little friable for future upgrades)? Please let me know if I'm wrong.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make what happen exactly? Upgrading tools? I'm a little confused what installing tools through a virtual environment would do in this situation and how it would mitigate the need to install extra stuff.

Also, I don't think we need pip3 install --upgrade build importlib_metadata setuptools setuptools_scm wheel as we're running a lint check on the code.

python3 -m pip install --upgrade pip
pip3 install --upgrade build importlib_metadata setuptools setuptools_scm wheel
pip3 install ruff
- name: Install dependencies
run: |
pip3 install -r requirements.txt
pip3 install -e .
- name: Prepare PYTHONPATH
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All that path stuff is required, I think. Well maybe it can be condensed into the above: 'Prepare Test Environment' or something...

run: |
src_paths=`find ${PWD} -type f -maxdepth 3 -mindepth 2 -name "*.py" -exec dirname {} + | uniq`
pythonpathplus=""
for p in $src_paths
do
pythonpathplus="${pythonpathplus:+:${pythonpathplus}}:$p"
done
echo "PYTHONPATH=${PYTHONPATH:+:${PYTHONPATH}}${pythonpathplus}:." >> $GITHUB_ENV
- name: Analyze
# Refer to https://docs.astral.sh/ruff/configuration/#command-line-interface
# to add extra rules or a configuration file.
run: ruff check --output-file ruff_report.txt . || true
continue-on-error: true
- name: Upload Artifact
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. But what if we don't compress it? Then maybe it can be viewable in the browser instead of requiring a bunch of clicks and unpacking, etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can run ruff check --output-format=github . and have it output in the browser if you'd like.

uses: actions/upload-artifact@v4
with:
name: ruff_report
path: ruff_report.txt
if-no-files-found: error
overwrite: true
retention-days: 15