Introduce Megalinter (#40)

---------

Co-authored-by: Mebin Abraham <[email protected]>
Co-authored-by: kacperpONS <[email protected]>
Co-authored-by: Dan Braghiș <[email protected]>
Co-authored-by: Jake Howard <[email protected]>
Co-authored-by: Helen Chapman <[email protected]>
Co-authored-by: Mebin Abraham <[email protected]>

.copier-answers.yml

@@ -1,3 +1,4 @@
+---
 # Changes here will be overwritten by Copier
 # Do not modify this file directly
 # To update the project using the original template, run make copier-update

.development.env

@@ -1,4 +1,4 @@
-DJANGO_SETTINGS_MODULE=cms.settings.dev 
 DATABASE_URL=postgres://ons:pass@localhost:5432/postgres
-REDIS_URL=redis://localhost:6379
+DJANGO_SETTINGS_MODULE=cms.settings.dev
 ENABLE_DJANGO_DEFENDER=false
+REDIS_URL=redis://localhost:6379

.docker/bashrc.sh

@@ -1,6 +1,6 @@
+#!/usr/bin/env bash
 # Note: This file is loaded on all environments, even production.
 
-if [ -n "$DEVCONTAINER" ]
-then
+if [ -n "$DEVCONTAINER" ]; then
     alias honcho="honcho -f .docker/Procfile"
 fi

.editorconfig

@@ -35,7 +35,7 @@ insert_final_newline = true
 [*.{js,json,yaml,yml}]
 indent_size = 2
 
-[*.{json,yaml,yml}]
+[*.{json,yaml,yml,md}]
 max_line_length = 160
 
 [Makefile]

.github/ISSUE_TEMPLATE.md

@@ -22,9 +22,9 @@ Describe what actually happened. Include details like error messages or unexpect
 
 Any relevant technical information. Include as much information as possible, including:
 
--   **Operating System**:
--   **Browser/Application Version**:
--   **Additional relevant software versions**:
+- **Operating System**:
+- **Browser/Application Version**:
+- **Additional relevant software versions**:
 
 ### Proposed Solutions (Optional)
 

.github/linters/.checkov.yml

@@ -7,5 +7,12 @@ output: cli
 quiet: true
 soft-fail: false
 skip-check:
-  - CKV2_GHA_1 # Not needed
+  # Ensure top-level permissions are not set to write-all
+  - CKV2_GHA_1
+  # Ensure that sudo isn't used
+  - CKV2_DOCKER_1
+  # Ensure that APT isn’t used
+  - CKV_DOCKER_9
+  # Ensure that HEALTHCHECK instructions have been added to container images
+  - CKV_DOCKER_2
 summary-position: bottom

.github/linters/.gitleaks.toml

@@ -0,0 +1,15 @@
+title = "gitleaks config"
+
+[extend]
+# useDefault will extend the base configuration with the default gitleaks config:
+# https://github.com/zricethezav/gitleaks/blob/master/config/gitleaks.toml
+useDefault = true
+
+[allowlist]
+description = "Global allowlists (skip conditions)"
+paths = [
+    '''(?i)^.*\.env$''',  # Matches any file ending in .env (case-insensitive)
+    '''(?i)^.*\.env\..*$''',  # Matches files like .env.test, .env.local, .env.production, etc.
+    "cms/settings/local.py",  # Ignore local.py in the cms/settings directory
+    "megalinter-reports"  # Ignore megalinter-reports directory
+]

.github/linters/.hadolint.yaml

@@ -0,0 +1,12 @@
+---
+#######################################################
+##               Hadolint config file                ##
+## Rules: https://github.com/hadolint/hadolint#rules ##
+#######################################################
+ignored:
+  # DL3018: Pin versions in apk add. Instead of apk add <package>
+  # use apk add <package>=<version>.
+  - DL3018
+  # DL3027: Do not use apt as it is meant to be an end-user tool,
+  # use apt-get or apt-cache instead
+  - DL3027

.github/linters/.markdown-lint.yml

@@ -82,11 +82,11 @@ MD012:
 # https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md013.md
 MD013:
   # Number of characters
-  line_length: 120
+  line_length: 160
   # Number of characters for headings
   heading_line_length: 80
   # Number of characters for code blocks
-  code_block_line_length: 120
+  code_block_line_length: 160
   # Include code blocks
   code_blocks: true
   # Include tables

.github/linters/.secretlintrc.json

@@ -0,0 +1,7 @@
+{
+  "rules": [
+    {
+      "id": "@secretlint/secretlint-rule-preset-recommend"
+    }
+  ]
+}

.github/linters/.yaml-lint.yml

@@ -33,7 +33,7 @@ rules:
     require-starting-space: true
     ignore-shebangs: true
     min-spaces-from-content: 1 # Used to follow prettier standard: https://github.com/prettier/prettier/pull/10926
-  comments-indentation: true
+  comments-indentation: enable
   document-end: disable
   document-start:
     level: error

.github/linters/kics.config

@@ -2,4 +2,17 @@
 
 # https://docs.kics.io/latest/
 exclude-queries:
+  - "451d79dc-0588-476a-ad03-3c7f0320abb3"  # Container Traffic Not Bound To Host Interface Severity: Medium
   - "555ab8f9-2001-455e-a077-f2d0f41e2fb9"  # Allow not pinning to commit SHA in GitHub CI
+  - "610e266e-6c12-4bca-9925-1ed0cd29742b"  # Security Opt Not Set
+  # Docker Compose Healthcheck Not Set Check containers periodically to see if they are running properly.
+  # Run as local not relevant
+  - "698ed579-b239-4f8f-a388-baa4bcb13ef8"
+  - "8c978947-0ff6-485c-b0c2-0bfca6026466"  # DockerCompose Shared Volumes Between Containers
+  - "a88baa34-e2ad-44ea-ad6f-8cac87bc7c71"  # Passwords And Secrets - Generic Password, Severity High
+  - "aa93e17f-b6db-4162-9334-c70334e7ac28"  # Chown Flag Exists Dockerfile
+  - "b03a748a-542d-44f4-bb86-9199ab4fd2d5"
+  - "b84a0b47-2e99-4c9f-8933-98bcabe2b94d"
+  - "ce76b7d0-9e77-464d-b86f-c5c48e03e22d"
+  - "f2f903fb-b977-461e-98d7-b3e2185c6118"
+

.github/release.yml

@@ -1,3 +1,4 @@
+---
 changelog:
   categories:
     - title: ⚠️ Breaking Changes

.github/workflows/ci.yml

@@ -133,8 +133,8 @@ jobs:
       - name: Add coverage data to GitHub Action summary
         run: |
           poetry run coverage html --skip-covered --skip-empty
-          echo "## Coverage summary" >> $GITHUB_STEP_SUMMARY
-          poetry run coverage report --format=markdown >> $GITHUB_STEP_SUMMARY
+          echo "## Coverage summary" >> "$GITHUB_STEP_SUMMARY"
+          poetry run coverage report --format=markdown >> "$GITHUB_STEP_SUMMARY"
       - name: Upload coverage HTML report
         uses: actions/upload-artifact@v4
         with:

.github/workflows/mega-linter.yml_todo → .github/workflows/mega-linter.yml

@@ -79,7 +79,7 @@ jobs:
 
         # You can override MegaLinter flavor used to have faster performances
         # More info at https://megalinter.io/latest/flavors/
-        uses: oxsecurity/megalinter@v7
+        uses: oxsecurity/megalinter/flavors/cupcake@v8
 
         id: ml
 

.gitignore

@@ -190,3 +190,6 @@ static/
 
 # Media files
 /media
+
+# bak files
+*.bak

.mega-linter.yml

@@ -20,21 +20,36 @@ FORMATTERS_DISABLE_ERRORS: false
 DISABLE:
   - COPYPASTE
   - PYTHON
+  - JAVASCRIPT
+  - CSS
   - EDITORCONFIG
   - SPELL
 
+# REPOSITORY_TRIVY, REPOSITORY_DUSTILOCK, HTML_HTMLHINT Have been disabled as part of the initial set up
+# CMS_
 DISABLE_LINTERS:
   - REPOSITORY_DEVSKIM
   - REPOSITORY_TRUFFLEHOG
   - REPOSITORY_GIT_DIFF
+  - HTML_HTMLHINT
+  - REPOSITORY_TRIVY
+  - REPOSITORY_DUSTILOCK
 
 SHOW_ELAPSED_TIME: true
 
 FILEIO_REPORTER: false
 # Uncomment if you want MegaLinter to detect errors but not block CI to pass
 # DISABLE_ERRORS: true
 
-# Use yml file to allow the use of comments. Megalinter's default is .markdown-lint.json
+# Use yml file to allow the use of comments.
+# Megalinter's default is .markdown-lint.json
 MARKDOWN_MARKDOWNLINT_CONFIG_FILE: .markdown-lint.yml
+# Dependabot yaml in SchemaStore is currently not up to date.
+# Dependabot is validated on push by GitHub by default.
+YAML_V8R_FILTER_REGEX_EXCLUDE: 'dependabot.yml'
 
-YAML_V8R_FILTER_REGEX_EXCLUDE: 'dependabot.yml' # Dependabot yaml in SchemaStore is currently not up to date. Dependabot is validated on push by GitHub by default.
+# GitLeaks: only scan the files in this commit, not the entire history
+REPOSITORY_GITLEAKS_ARGUMENTS: '--no-git'
+
+# .gitignore files are excluded by default
+ADDITIONAL_EXCLUDED_DIRECTORIES: ['.git']

.pre-commit-config.yaml

@@ -1,9 +1,9 @@
+---
 default_language_version:
   node: system
   python: python3.12
 
-repos:
-  # Python linting and formatting
+repos: # Python linting and formatting
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: 'v0.7.4' # keep version in sync with pyproject.toml
     hooks:
@@ -18,13 +18,12 @@ repos:
         entry: poetry run pylint
         language: system
         types: [python]
-        args: [
-            '--reports=n', # only display messages
-            '--output-format=colorized',
-            '--rcfile=.pylintrc',
-            '--django-settings-module=cms.settings.production',
-            '-j 0', # detect number of processors available to use
-          ]
+        args:
+          - '--reports=n' # only display messages
+          - '--output-format=colorized'
+          - '--rcfile=.pylintrc'
+          - '--django-settings-module=cms.settings.production'
+          - '-j 0' # detect number of processors available to use
 
   - repo: https://github.com/python-poetry/poetry
     rev: '1.8.4' # keep version in sync with version installed in the Dockerfile
@@ -70,5 +69,7 @@ repos:
           (?x)^(
               .*/tests/.*|
               package-lock.json|
-              poetry.lock
+              poetry.lock|
+              .*\.env.*|
+              .*local\.py
           )$

CODE_OF_CONDUCT.md

@@ -13,9 +13,9 @@ Read the [Civil Service Code](https://www.gov.uk/government/publications/civil-s
 
 > Note:
 >
-> -   where the code of conduct says "project" we mean ONS, `ONSdigital` and all repositories hosted within it.
-> -   where the code of conduct says "maintainer" we mean `ONSdigital` organisation owners
-> -   where the code of conduct says "leadership" we mean both `ONSdigital` organisation owners, line managers, and other
+> - where the code of conduct says "project" we mean ONS, `ONSdigital` and all repositories hosted within it.
+> - where the code of conduct says "maintainer" we mean `ONSdigital` organisation owners
+> - where the code of conduct says "leadership" we mean both `ONSdigital` organisation owners, line managers, and other
 
     leadership within ONS
 
@@ -36,24 +36,24 @@ diverse, inclusive, and healthy community.
 Examples of behavior that contributes to a positive environment for our
 community include:
 
--   Demonstrating empathy and kindness toward other people
--   Being respectful of differing opinions, viewpoints, and experiences
--   Giving and gracefully accepting constructive feedback
--   Accepting responsibility and apologising to those affected by our mistakes,
-    and learning from the experience
--   Focusing on what is best not just for us as individuals, but for the overall
-    community
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologising to those affected by our mistakes,
+  and learning from the experience
+- Focusing on what is best not just for us as individuals, but for the overall
+  community
 
 Examples of unacceptable behavior include:
 
--   The use of sexualised language or imagery, and sexual attention or advances of
-    any kind
--   Trolling, insulting or derogatory comments, and personal or political attacks
--   Public or private harassment
--   Publishing others' private information, such as a physical or email address,
-    without their explicit permission
--   Other conduct which could reasonably be considered inappropriate in a
-    professional setting
+- The use of sexualised language or imagery, and sexual attention or advances of
+  any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
 
 ## Enforcement Responsibilities
 

Dockerfile

@@ -220,6 +220,10 @@ ARG POSTGRES_VERSION=16
 # RUN --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
 #     --mount=type=cache,target=/var/cache/apt,sharing=locked \
 #     <<EOF
+
+# Set default shell with pipefail option
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
 RUN <<EOF
     apt --quiet --yes update
     apt --quiet --yes install \

Makefile

@@ -74,7 +74,7 @@ megalint:  ## Run the mega-linter.
 	docker run --platform linux/amd64 --rm \
 		-v /var/run/docker.sock:/var/run/docker.sock:rw \
 		-v $(shell pwd):/tmp/lint:rw \
-		oxsecurity/megalinter:v7
+		oxsecurity/megalinter-cupcake:v8
 
 .PHONY: load-design-system-templates
 load-design-system-templates:  ## Load the design system templates