Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add Taskfile tasks to lint YAML files. #3

Merged
merged 4 commits into from
Oct 23, 2024

Conversation

kirkrodrigues
Copy link
Member

@kirkrodrigues kirkrodrigues commented Oct 23, 2024

Description

  • Adds taskfiles and tasks to lint YAML files using yamllint and our configs from yscope-dev-utils
    • As a result, we also add yscope-dev-utils as a submodule.

Validation performed

  • task lint:check validated the existing YAML files.
  • task lint:fix did the same.
  • task lint:yml did the same.
  • task lint:yml-check did the same.
  • task lint:yml-fix did the same.
  • Unquoted a string in Taskfile.yml and re-ran the steps above to ensure they detected the violation.
  • task clean cleaned any generated output.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added a new submodule for enhanced development utilities.
    • Introduced a structured task management system for linting and build processes.
  • Documentation

    • Updated the README file with a "Contributing" section, including setup instructions and linting guidelines.
  • Bug Fixes

    • Specified a minimum version for the yamllint dependency to ensure consistent linting standards.

These updates improve project organization and support for contributors, ensuring better code quality and maintainability.

Copy link

coderabbitai bot commented Oct 23, 2024

Walkthrough

The changes introduced in this pull request include updates to the .gitignore file to exclude specific build and IDE-related files, the addition of a new submodule in the project configuration, enhancements to the README.md file with a new "Contributing" section, and the introduction of a Taskfile.yml for task management. Additionally, a new dependency for linting YAML files has been specified in lint-requirements.txt, while a new lint-tasks.yml file has been created to manage linting tasks. A new subproject commit has also been added to the tools/yscope-dev-utils directory.

Changes

File Change Summary
.gitignore Added entries to exclude .task, build, and .idea directories and files from version control.
.gitmodules Added submodule tools/yscope-dev-utils linked to https://github.com/y-scope/yscope-dev-utils.git.
README.md Updated to include a "Contributing" section with project requirements, setup instructions, and linting guidelines.
Taskfile.yml Introduced task management with tasks clean and init, and defined variable G_BUILD_DIR.
lint-requirements.txt Added dependency yamllint>=1.35.1 for YAML linting.
lint-tasks.yml Created file with tasks for YAML linting: check, fix, yml, and an internal task venv for virtual environment management.
tools/yscope-dev-utils Added a new subproject commit identified by hash 159768c7d171595ed2cba17b758c10043a2efe96.

Sequence Diagram(s)

sequenceDiagram
    participant Contributor
    participant Git
    participant LintingTool
    participant TaskManager

    Contributor->>Git: Clone repository
    Contributor->>TaskManager: Initialize project
    TaskManager->>Git: Update submodules
    Contributor->>LintingTool: Run linting checks
    LintingTool->>Contributor: Provide linting results
    Contributor->>LintingTool: Fix linting issues
    LintingTool->>Contributor: Confirm fixes
    Contributor->>Git: Submit pull request
Loading

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Outside diff range and nitpick comments (6)
Taskfile.yml (2)

7-8: Consider adding a comment to document the build directory variable.

The variable declaration is correct, but adding a brief comment would help explain its purpose.

 vars:
+  # Directory for build artifacts and temporary files
   G_BUILD_DIR: "{{.ROOT_DIR}}/build"

10-19: Consider adding a safeguard to the clean task.

The tasks are well-structured, but the clean task could benefit from a safety check to prevent accidental deletion if G_BUILD_DIR is empty.

   clean:
     cmds:
-      - "rm -rf '{{.G_BUILD_DIR}}'"
+      - |
+        if [ -n "{{.G_BUILD_DIR}}" ]; then
+          rm -rf '{{.G_BUILD_DIR}}'
+        else
+          echo "Error: G_BUILD_DIR is not set"
+          exit 1
+        fi
README.md (4)

5-6: Specify the minimum Python version requirement

Consider specifying the minimum Python version required (e.g., "Python 3.8 or higher") to ensure contributors use a compatible version.


9-12: Add context about the submodules being initialized

Consider adding a brief explanation of what submodules are being initialized (e.g., "Initialize the yscope-dev-utils and other required submodules:") to help contributors understand what they're setting up.


23-26: Improve wording in the linting description

Replace "AND" with "and" for consistent and professional documentation style.

-To run all linting checks AND automatically fix any fixable issues:
+To run all linting checks and automatically fix any fixable issues:
🧰 Tools
🪛 LanguageTool

[style] ~23-~23: Consider using a different verb for a more formal wording.
Context: ...un all linting checks AND automatically fix any fixable issues: ```shell task lint:...

(FIX_RESOLVE)


4-7: Add installation instructions for requirements

Consider adding instructions or links for installing the required tools (Python 3 and Task). This would make the setup process more straightforward for new contributors.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 72a9b1e and ecf16a8.

📒 Files selected for processing (7)
  • .gitignore (1 hunks)
  • .gitmodules (1 hunks)
  • README.md (1 hunks)
  • Taskfile.yml (1 hunks)
  • lint-requirements.txt (1 hunks)
  • lint-tasks.yml (1 hunks)
  • tools/yscope-dev-utils (1 hunks)
✅ Files skipped from review due to trivial changes (4)
  • .gitignore
  • .gitmodules
  • lint-requirements.txt
  • tools/yscope-dev-utils
🧰 Additional context used
🪛 LanguageTool
README.md

[style] ~23-~23: Consider using a different verb for a more formal wording.
Context: ...un all linting checks AND automatically fix any fixable issues: ```shell task lint:...

(FIX_RESOLVE)

🔇 Additional comments (5)
Taskfile.yml (1)

1-2: LGTM! Version declaration is correct.

The version declaration follows the proper format and uses a stable version.

README.md (2)

33-36: LGTM! Well-structured task documentation

The table clearly presents the available linting tasks with appropriate descriptions.


38-38: LGTM! Proper reference link

The Task reference link is correctly formatted and matches the tool mentioned in requirements.

lint-tasks.yml (2)

30-56: Well-structured virtual environment management

The venv task implementation follows best practices:

  • Uses checksums for efficient cache invalidation
  • Clear dependency management
  • Proper virtual environment setup

3-4: Verify G_BUILD_DIR variable definition

The task file references {{.G_BUILD_DIR}} which isn't defined in this file.

Comment on lines +3 to +5
includes:
lint: "lint-tasks.yml"
utils: "tools/yscope-dev-utils/taskfiles/utils.yml"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Initialize and update the yscope-dev-utils submodule

The verification shows that yscope-dev-utils is a Git submodule that hasn't been initialized yet (indicated by the leading - in the submodule status). This explains why the utils.yml file is missing. The submodule needs to be initialized and updated before the taskfile can be used.

  • The local lint-tasks.yml exists and is properly referenced
  • The tools/yscope-dev-utils submodule is present but needs initialization
  • Run git submodule update --init tools/yscope-dev-utils to make utils.yml available
🔗 Analysis chain

Verify the existence of included task files.

The includes section references two external files. Let's verify their existence.


Let me verify if the utils file might be in a different location or if there are any similar YAML files that could help us understand the structure better.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the referenced task files exist

echo "Checking for included task files..."
for file in "lint-tasks.yml" "tools/yscope-dev-utils/taskfiles/utils.yml"; do
  if [ -f "$file" ]; then
    echo "✓ Found: $file"
  else
    echo "✗ Missing: $file"
  fi
done

Length of output: 490


Script:

#!/bin/bash
# Search for utils.yml files and taskfiles directories
echo "Searching for utils.yml files:"
fd utils.yml

echo -e "\nSearching for taskfiles directories:"
fd taskfiles -t d

echo -e "\nChecking if yscope-dev-utils is a submodule:"
git submodule status | grep yscope-dev-utils

Length of output: 411

lint-tasks.yml Outdated
Comment on lines 26 to 28
.github \
lint-tasks.yml \
Taskfile.yml
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider expanding YAML file coverage

The current configuration only lints specific files and directories. Consider using a more comprehensive glob pattern to catch all YAML files in the repository.

-          .github \
-          lint-tasks.yml \
-          Taskfile.yml
+          "$(fd -e yml -e yaml)"

Committable suggestion was skipped due to low confidence.

lint-tasks.yml Outdated
Comment on lines 23 to 24
yamllint \
--config-file "{{.ROOT_DIR}}/tools/yscope-dev-utils/lint-configs/.yamllint.yml" \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Missing yamllint configuration file in referenced submodule

The yamllint configuration file is missing from the expected location tools/yscope-dev-utils/lint-configs/.yamllint.yml. The git submodule yscope-dev-utils exists but appears to be not properly initialized or is missing the required configuration file. This will cause the YAML linting task to fail.

  • The submodule is present at commit 159768c7d171595ed2cba17b758c10043a2efe96 but not initialized
  • The referenced configuration path tools/yscope-dev-utils/lint-configs/.yamllint.yml does not exist
  • Only two YAML files are present in the repository: Taskfile.yml and lint-tasks.yml
🔗 Analysis chain

Verify yamllint configuration file existence

The task references a config file in the yscope-dev-utils submodule.


Let me verify if the yamllint configuration file exists in other locations or if there are any similar YAML configuration files in the repository.


Let me try one more search with different commands to ensure we haven't missed any YAML configuration files.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if the yamllint config exists
test -f tools/yscope-dev-utils/lint-configs/.yamllint.yml && echo "Config exists" || echo "Config missing"

Length of output: 98


Script:

#!/bin/bash
# Search for yamllint configuration files
fd -t f ".yamllint" -t f ".yamllint.yml" -t f ".yamllint.yaml"

# Search for any YAML files that might contain yamllint configuration
rg -l "extends:|rules:" --type yaml

Length of output: 258


Script:

#!/bin/bash
# List all files in tools directory to check submodule structure
ls -la tools/

# Search for any yaml/yml files in the repository
fd -e yml -e yaml

# Check if yscope-dev-utils is a git submodule
git submodule status

Length of output: 325

Comment on lines +7 to +13
check:
cmds:
- task: "yml-check"

fix:
cmds:
- task: "yml-fix"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Tasks yml-check and yml-fix are aliases for the same YAML linting operation

Looking at the lint-tasks.yml file, I can now verify that:

  • Both yml-check and yml-fix are defined as aliases under the yml task
  • They both execute the same yamllint command with strict configuration
  • The tasks are properly configured with virtual environment setup and dependencies

Note: The tasks appear to perform the same operation without any distinction between check and fix modes, which might be worth reviewing as the README suggests different behaviours:

  • lint:yml-check is documented to run linters
  • lint:yml-fix is documented to run linters and fix violations
🔗 Analysis chain

Verify yml-check and yml-fix task definitions

The tasks reference yml-check and yml-fix which aren't defined in this file.


Let me try a different search strategy to locate these task definitions.


Let's examine the Taskfile.yml and README.md to find the task definitions.


Let's check the content of lint-tasks.yml since it's included in Taskfile.yml under the 'lint' namespace.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for yml-check and yml-fix task definitions
rg -l 'yml-(check|fix):' .

Length of output: 98


Script:

#!/bin/bash
# Search for yml-check and yml-fix task definitions using broader patterns
rg -l "yml.*check|yml.*fix" .

# Also search for Taskfile or similar task definition files
fd -e yml -e yaml "Taskfile" .

Length of output: 106


Script:

#!/bin/bash
# Check the content of Taskfile.yml and README.md
echo "=== Taskfile.yml ==="
cat Taskfile.yml

echo -e "\n=== README.md sections mentioning yml tasks ==="
rg -A 5 -B 5 "yml.*(check|fix)" README.md

Length of output: 1037


Script:

#!/bin/bash
# Check the content of lint-tasks.yml
echo "=== lint-tasks.yml ==="
cat lint-tasks.yml

# Also check if there are any other task files in the tools directory
echo -e "\n=== Looking for additional task files ==="
fd -e yml -e yaml . tools/

Length of output: 1523

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (6)
README.md (6)

1-2: Consider enhancing the introduction with more context.

The introduction could be more welcoming and informative by including:

  • Types of contributions welcomed (e.g., bug fixes, features)
  • Link to issue tracker
  • Reference to coding standards
 # Contributing
-Follow the steps below to develop and contribute to the project.
+Welcome! We appreciate your interest in contributing to the project. Before you begin,
+please review our contribution guidelines below. We welcome various types of contributions
+including bug fixes, feature enhancements, and documentation improvements.
+
+For bug reports and feature requests, please check our issue tracker first.

4-7: Add installation instructions for requirements.

Consider adding brief installation instructions or links for each requirement to help new contributors get started quickly.

 ## Requirements
 * Python 3.8 or higher
+  * Installation: Visit [python.org](https://www.python.org/downloads/)
 * [Task] 3.38.0 or higher
+  * Installation: Follow the [Task installation guide](https://taskfile.dev/installation/)

8-12: Enhance setup instructions with additional steps.

Consider adding:

  • Cloning instructions for new contributors
  • Virtual environment setup
  • Dependencies installation
 ## Set up
+First, clone the repository:
+```shell
+git clone https://github.com/y-scope/spider.git
+cd spider
+```
+
 Initialize and update submodules:
 ```shell
 git submodule update --init --recursive

+Create and activate a virtual environment:
+shell +python -m venv venv +source venv/bin/activate # On Windows, use `venv\Scripts\activate` +
+
+Install development dependencies:
+shell +pip install -r requirements.txt +pip install -r lint-requirements.txt +


---

`23-26`: **Improve wording and formatting of linting instructions.**

The use of "AND" in all caps is informal. Consider rewording for a more professional tone.

```diff
-To run all linting checks AND automatically fix any fixable issues:
+To run all linting checks and automatically apply available fixes:
 ```shell
 task lint:fix

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary><blockquote>

[style] ~23-~23: Consider using a different verb for a more formal wording.
Context: ...un all linting checks AND automatically fix any fixable issues: ```shell task lint:...

(FIX_RESOLVE)

</blockquote></details>

</details>

---

`33-36`: **Add details about YAML linting configuration.**

Consider adding information about:
- The location of YAML linting configuration
- Common YAML linting rules in use
- How to customize or suppress specific rules

```diff
 | Task                    | Description                                              |
 |-------------------------|----------------------------------------------------------|
 | `lint:yml-check`        | Runs the YAML linters.                                   |
 | `lint:yml-fix`          | Runs the YAML linters and fixes some violations.         |
+
+The YAML linting configuration is inherited from the `yscope-dev-utils` submodule. To suppress
+specific warnings, you can add inline directives:
+```yaml
+# yamllint disable-line rule:key-name
+```

38-38: Consider moving reference links to a dedicated section.

For better visibility and maintenance, consider creating a "References" section at the end of the document.

-[Task]: https://taskfile.dev
+## References
+
+* [Task] - Task automation tool
+
+[Task]: https://taskfile.dev
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between ecf16a8 and cb36ca2.

📒 Files selected for processing (1)
  • README.md (1 hunks)
🧰 Additional context used
🪛 LanguageTool
README.md

[style] ~23-~23: Consider using a different verb for a more formal wording.
Context: ...un all linting checks AND automatically fix any fixable issues: ```shell task lint:...

(FIX_RESOLVE)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant