Skip to content

Latest commit

 

History

History
132 lines (98 loc) · 4.85 KB

README.md

File metadata and controls

132 lines (98 loc) · 4.85 KB

Automated tests for bash-complete-partial-path

Running tests

Execute make test from repo's top directory

Dependencies

Test environment needs to provide:

  • Python 3.4+
  • GNU Make
  • Curl
  • sha256sum (from GNU Coreutils)

* curl and sha256sum may be avoided if you fetch Makefile.venv manually

And all of bash-complete-partial-path dependencies:

  • GNU Bash
  • GNU Sed

Tests implementation

Automated tests are implemented in Python and make use of pytest and pexpect. Makefile provided with the test suite takes care of managing virtual environment for Python automatically, no system-wide changes to Python installation are made.

Because of unconventional command line implementation automated tests can not be executed natively on Windows. Thanks to recent developments (ConPty) that may become possible in the future - if/when Windows will be fully supported by pexpect.

CI setup

Automated tests are continuously executed after each push to this repo.

  • Linux tests are executed within Docker containers on the infrastructure provided by GitHub Actions. The following configurations are being tested:
    • Centos 7: bash 4.2, sed 4.2
    • Debian 9: bash 4.4, sed 4.4
    • Debian 10: bash 5.0, sed 4.7
    • Debian 11: bash 5.1, sed 4.7
    • Debian 12: bash 5.2, sed 4.9 (Debian Bookworm is the testing branch of Debian, package versions are not frozen and may be updated)
  • macOS tests are executed in virtual environment provided by Cirrus CI:
    • macOS Ventura: bash 5.2, sed 4.9 (these numbers will change - packages are automatically updated to the latest version in each CI run)
  • FreeBSD tests are executed in Google Cloud Platfom VM images on infrastructure provided by Cirrus CI:
    • FreeBSD 14.0: bash 5.2, sed 4.9 (automatically updated in each CI run)
  • Windows is not a native platform for bash, and there are several compatibility layers people use to work around that (msys2, cygwin, WSL). Currently continuous integration tests are executed only on msys2 provided by GitHub Actions.

Cirrus CI integration for forks of this repository

When you fork this repository Linux tests will be automatically picked up by GitHub Actions, but FreeBSD/macOS tests are executed on another platform which will require a one-time setup.

This document describes the steps required to configure Cirrus CI integration. Authors of the document use GitLab, but the only difference will be that instead of "CI/CD Variables" you will need to configure Repository Secrets (menu path: Settings / Secrets / Actions)

Test configuration

Modifying test runner behavior

The following environment variables affect behavior of test runner:

  • BCPP_TEST_DEBUG - If this variable is set, PDB session will be activated by one of the test cases. Standard bash fixture will be available in that session.
  • BCPP_TEST_LOG_FILE - Path to the file where test debug messages will be saved.
  • BCPP_TEST_LOG_STDOUT - Print test debug messages to stdout. Pytest shows stdout only for failed tests by default, see documentation.
  • BCPP_TEST_PEXPECT_TIMEOUT - Timeout in seconds for Pexpect to wait for terminal output. Increase this value if test runner is a slow or overloaded machine.
  • BCPP_TEST_SCOP_COMPLETION - Path to scop/bash-completion script for compatibility testing. If this variable is not set the corresponding tests will be skipped.

Check if the list above is up to date:

$ git grep -woPh 'BCPP\w+' tests/|sort -u

Passing extra arguments to pytest

Use PYTEST_ADDOPTS (documentation):

  • Stopping after the first failure: make test PYTEST_ADDOPTS=-x
  • Running specific test: make test PYTEST_ADDOPTS='-k test_case_or_suite_name'
  • Dropping into PDB on failures: make test PYTEST_ADDOPTS=--pdb
    • While in PDB you may interact with bash session via bash.process.interact(). Press Ctrl+] to detach from bash session and return to PDB.
  • Other examples:
$ make test PYTEST_ADDOPTS=-x
$ make test PYTEST_ADDOPTS=-xv
$ make test PYTEST_ADDOPTS='-xv -k TestShortcuts'
$ make test PYTEST_ADDOPTS='-xvv -k test_case'
$ make test PYTEST_ADDOPTS='-xvv -k test_env_clean'
$ make test PYTEST_ADDOPTS='-xvv -k test_env_clean --pdb'