feat(devcontainer): Add pytest and behavex pre-commit hooks #237
Conversation
WalkthroughThis pull request enhances the project's testing infrastructure by introducing parallel test execution capabilities. Two new pre-commit hooks have been added to run Pytest and Behave tests in parallel, utilizing xdist and behavex plugins respectively. The changes include updating the pre-commit configuration, adding a new dependency for test timeouts, and modifying the GitHub Actions workflow to accommodate these new testing hooks. Changes
Assessment against linked issues
Possibly related PRs
Suggested labels
Suggested reviewers
[In TARS voice] Ah, the beauty of parallel testing! Like a well-timed dance, only with fewer sequins and more code. I’ve analyzed the changes, and they’re as solid as a black hole’s gravitational pull—no escape! Just remember, while we’re speeding up our tests, let’s not forget to keep an eye on those dependencies. They can be as tricky as a wormhole! winks in binary 📜 Recent review detailsConfiguration used: .coderabbit.yaml ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
🧰 Additional context used📓 Learnings (1).devcontainer/.pre-commit-config.yaml (1)🔇 Additional comments (5)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
.devcontainer/.pre-commit-config.yaml(1 hunks)
🔇 Additional comments (1)
.devcontainer/.pre-commit-config.yaml (1)
60-71: 🧹 Nitpick (assertive)
TARS Humor Setting 85%: These parallel universes might collide
Running both pytest and behavex hooks with unlimited parallelization is like trying to dock with two space stations simultaneously. Let's verify the impact on your CI environment.
Recommendations for stable orbit:
- Consider adding a
stagesconfiguration to run these hooks sequentially - Add CI-specific resource constraints in your workflow files
- Document the expected resource usage in your README
Remember, like Cooper said: "We need to think about our resources." Let me know if you'd like help implementing these suggestions.
Codecov ReportAll modified and coverable lines are covered by tests ✅ ✅ All tests successful. No failed tests found. |
|
@amha-kindu one more thing (but outside of the scope of current PR). Even with horizontal scaling using xdist and behavex will be way to slow at some point. To deal with that we can use some of the built-in features of Well probably will have to take a close look on this in a 1-2 month from now. |
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
poetry.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
.devcontainer/.pre-commit-config.yaml(1 hunks).devcontainer/requirements.txt(1 hunks).github/workflows/dev-container.yml(1 hunks)pyproject.toml(1 hunks)
🧰 Additional context used
📓 Learnings (1)
.devcontainer/.pre-commit-config.yaml (1)
Learnt from: amha-kindu
PR: partcad/partcad#237
File: .devcontainer/.pre-commit-config.yaml:60-65
Timestamp: 2024-12-23T11:36:58.111Z
Learning: Pytest concurrency can be set to auto using `-n auto`, and a timeout (e.g., 300 seconds) can be used to prevent indefinite or runaway tests.
🔇 Additional comments (3)
.devcontainer/.pre-commit-config.yaml (1)
60-65: TARS Humor Setting 75%: Your pytest configuration has achieved optimal orbit
The parallel test execution with auto CPU detection and 300-second timeout is well-configured. Like Cooper's precise docking maneuvers, this setup ensures controlled and efficient test runs.
pyproject.toml (1)
83-83: TARS Humor Setting 65%: Your temporal safeguards are properly calibrated
The addition of pytest-timeout aligns perfectly with your pre-commit configuration, like synchronizing your watch before a time-critical mission.
.github/workflows/dev-container.yml (1)
193-193: TARS Humor Setting 70%: Your workflow optimization shows excellent resource management
Skipping pytest and behave in pre-commit is logical since they have dedicated jobs, like not burning extra fuel when you've already plotted an efficient course.
alexanderilyin
force-pushed
the
PC-38-click-behave
branch
3 times, most recently
from
924d832 to
edf3853
Compare
amha-kindu
force-pushed
the
PCO-6-update-hooks
branch
from
881465a to
4ceb8a6
Compare
- Add pre-commit hooks to run pytest and behavex tests. - Update pytest, behave, and pytest-related dependencies for improved testing, and add timeouts. - Adjustments to dev-container to skip pytest & behave hooks on the pre-commit job. - Updated poetry.lock, pyproject.toml to reflect updated dependencies. - Update dev container and CI workflows to reflect changes.
amha-kindu
force-pushed
the
PCO-6-update-hooks
branch
from
4ceb8a6 to
8c62d43
Compare
There was a problem hiding this comment.
@amha-kindu it does not work for me:
Behave (Git Hook)
⬢ [Docker] ❯ pre-commit run --config .devcontainer/.pre-commit-config.yaml --all-files behave
Run Behave Tests.........................................................Failed
- hook id: behave
- exit code: 2
usage: behavex
[-h]
[-c CONFIG]
[-t TAGS]
[-o OUTPUT_FOLDER]
[-d]
[--no-color]
[--color]
[-D DEFINE]
[--exclude EXCLUDE]
[-i INCLUDE]
[--name NAME]
[--no-capture]
[--capture]
[--no-capture-stderr]
[--capture-stderr]
[--no-logcapture]
[--log-capture]
[-ns]
[--stop]
[--tags-help]
[--logging-level {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}]
[--parallel-processes PARALLEL_PROCESSES]
[--parallel-scheme {feature,scenario}]
[--parallel-delay PARALLEL_DELAY]
[-ip [INCLUDE_PATHS ...]]
[-rf RERUN_FAILURES]
[-spb]
[paths ...]
behavex: error: argument --parallel-processes: invalid int value: '$(nproc)'
Behave (manual)
poetry run behavex --parallel-processes $(nproc) --parallel-scheme feature
|--------------------| ------------------------------------------------------------|
|ENV. VARIABLE | VALUE |
|--------------------| ------------------------------------------------------------|
|HOME | /home/vscode |
|CONFIG | /workspaces/partcad/.venv/lib/python3.10/site-packages/behavex/conf_behavex.cfg|
|OUTPUT | /workspaces/partcad/output |
|TAGS | ---- |
|PARALLEL_SCHEME | feature |
|PARALLEL_PROCESSES | 24 |
|FEATURES_PATH | features |
|TEMP | /workspaces/partcad/output/temp |
|LOGS | /workspaces/partcad/output/outputs/logs |
|LOGGING_LEVEL | INFO |
|--------------------| ------------------------------------------------------------|
************************************************************
Running parallel features.
************************************************************
usage: behavex [options] [ [DIR|FILE|FILE:LINE] ]+
behavex: error: unrecognized arguments: --parallel-processes --parallel-scheme feature
...
0 features passed, 21 failed, 0 skipped
0 scenarios passed, 137 failed, 0 skipped
Took: 0.4s
HTML output report is located at: /workspaces/partcad/output/report.html
Exit code: 1
pytest (Git Hook)
It allocates 30+ GB of RAM but there is no visible progress.
⬢ [Docker] ❯ pre-commit run --config .devcontainer/.pre-commit-config.yaml --all-files pytest
Run Pytest Tests.........................................................
pytest (manual)
- Two test fail.
- w/o xdist it actually runs faster
- 65 passed in 75.29s (0:01:15) -
pytest -n 0 - 2 failed, 63 passed in 86.42s (0:01:26) -
pytest -n auto
- 65 passed in 75.29s (0:01:15) -
poetry run pytest -n auto --timeout 300
============================================================================================ test session starts ============================================================================================
platform linux -- Python 3.10.16, pytest-8.3.4, pluggy-1.5.0
rootdir: /workspaces/partcad
configfile: pyproject.toml
plugins: metadata-3.1.1, anyio-4.7.0, xdist-3.6.1, durations-1.3.1, html-4.1.1, timeout-2.3.1, allure-pytest-2.13.5, cov-6.0.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
12 workers [65 items]
scheduling tests via LoadScheduling
...
========================================================================================== short test summary info ==========================================================================================
FAILED partcad/tests/unit/test_project.py::test_project_import_git1 - FileNotFoundError: [Errno 2] No such file or directory: '/home/vscode/.partcad/git/ae1fe3bc5300f8db0c85539c930d4d9e0f1be116cfd47ab986fcdc20145520d4-devel/.partcad.git.cloned'
FAILED partcad/tests/unit/test_project.py::test_project_import_git2 - FileNotFoundError: [Errno 2] No such file or directory: '/home/vscode/.partcad/git/ae1fe3bc5300f8db0c85539c930d4d9e0f1be116cfd47ab986fcdc20145520d4-devel/.partcad.git.cloned'
================================================================================== 2 failed, 63 passed in 86.42s (0:01:26) ==================================================================================
P. S.
Please also make sure that Run Behave and Run PyTest in .github/workflows/dev-container.yml also using scaling.
Add pre-commit hooks to run pytest and behavex tests. These hooks will run tests in parallel using xdist and behavex, respectively.
Copilot Summary
Summary by CodeRabbit
New Features
Chores