Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

noseyparker 0.19.0 #179041

Merged
merged 2 commits into from
Aug 1, 2024
Merged

noseyparker 0.19.0 #179041

merged 2 commits into from
Aug 1, 2024

Conversation

BrewTestBot
Copy link
Member

Created by brew bump


Created with brew bump-formula-pr.

release notes
### Additions
  • The scan and github repos list commands offer a new --github-repo-type={all,source,fork} option to select a subset of repositories (#204).

  • A category mechanism is now provided for rules (#208). Each rule can have zero or more freeform text categories assigned to it. The existing rules have been updated with category information with the following meanings:

    • secret: the rule detects things that are in fact secrets
    • identifier: the rule detects things that are not secrets but could be used to enumerate additional resources (e.g., S3 bucket names)
    • hashed: the rule detects hashed payloads (e.g., bcrypt hashes)
    • test: the rule detects test deployment-specific payloads (e.g., stripe test keys)
    • api: the rule detects payloads used for API access
    • generic: the rule is a "generic" one rather than one that detects a specific type of payload (e.g., username/password pairs)
    • fuzzy: the rule pattern requires matching of non-payload surrounding context

    The category information is included in output in the rules list command.

Changes

  • The scan and github repos list commands now only consider non-forked repositories by default (#204). This behavior can be reverted to the previous behavior using the --github-repo-type=all option.

  • The Alpine-based Docker image has been updated to use the alpine:latest base image instead of alpine:3.18 (#201).

  • The "Blynk Organization" rules have been refined (#208). The two "Blynk Organization Client ID" and two "Blynk Organization Client Secret" variations have been subsumed by two new Blynk Organization Client Credential rules. These new rules combine the client ID and client secret into single findings instead of reporting them as two separate findings as previous.

  • Several rules have been renamed (#208):

    • AWS S3 Bucket (subdomain style) -> AWS S3 Bucket
    • AWS S3 Bucket (path style) -> AWS S3 Bucket
    • Blynk Organization Access Token (URL first) -> Blynk Organization Access Token.
    • Blynk Organization Access Token (URL last) -> Blynk Organization Access Token.
    • Generic Password (double quoted) -> Generic Password
    • Generic Password (single quoted) -> Generic Password
    • Generic Username and Password (quoted) -> Generic Username and Password
    • Generic Username and Password (unquoted) -> Generic Username and Password
    • Google Cloud Storage Bucket (path style) -> Google Cloud Storage Bucket
    • Google Cloud Storage Bucket (subdomain style) -> Google Cloud Storage Bucket
    • Google OAuth Client Secret (prefixed) -> Google OAuth Client Secret
    • New Relic License Key (non-suffixed) -> New Relic License Key
    • particle.io Access Token (URL first) -> particle.io Access Token
    • particle.io Access Token (URL last) -> particle.io Access Token

    Note that although several rules share the same name now, they all still have distinct IDs.

  • The default set of patterns for the existing gitignore-style path-based exclusion mechanism (scan --ignore=GITIGNORE_FILE) has been expanded (#209). The new patterns cover test files from things like vendored Python, Node.js, and Go packages.

  • The gitignore-style path-based exclusion patterns (scan --ignore=GITIGNORE_FILE) now also apply to content found within Git history, and not just paths on the filesystem (#209). When a blob is found in Git history with at least 1 associated pathname, if all of the associated pathnames match the ignore rules, the blob is not scanned.

  • The Rust version required to build has been bumped from 1.76 to 1.77. This is necessary to support C-string literals in the rusqlite crate.

@github-actions github-actions bot added rust Rust use is a significant feature of the PR or issue bump-formula-pr PR was created using `brew bump-formula-pr` boost Boost use is a significant feature of the PR or issue labels Jul 31, 2024
@chenrui333
Copy link
Member

==> /home/linuxbrew/.linuxbrew/Cellar/noseyparker/0.19.0/bin/noseyparker scan --git-url https://github.com/Homebrew/brew
  Error: noseyparker: failed
  Error: noseyparker: failed
  An exception occurred within a child process:
    Minitest::Assertion: Expected /0\/0\ new\ matches/ to match "Scanned 1.06 GiB from 72,880 blobs in 2 seconds ([54](https://github.com/Homebrew/homebrew-core/actions/runs/10172018209/job/28134033744?pr=179041#step:4:55)1.37 MiB/s); 4/4 new matches\n\n Rule                Findings   Matches   Accepted   Rejected   Mixed   Unlabeled \n──────────────────────────────────────────────────────────────────────────────────\n GitHub Secret Key          1         4          0          0       0           1 \n\nRun the `report` command next to show finding details.\n".

@chenrui333 chenrui333 added test failure CI fails while running the test-do block CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. labels Jul 31, 2024
@chenrui333
Copy link
Member

seen the test failure in rust 1.80.0 as well, #178436

@chenrui333 chenrui333 force-pushed the bump-noseyparker-0.19.0 branch from 9659026 to 3171d9e Compare August 1, 2024 16:38
@bradlarsen
Copy link
Contributor

The test failure here is not related to the rust version used to build, but because there are some items picked up in the brew Git repository now. When I run locally, I see this:

% noseyparker scan --git-url https://github.com/Homebrew/brew
Fetching Git repos  ████████████████████ 100%  1/1  [00:00:07]
Found 1.06 GiB from 17 plain files and 72,884 blobs from 1 Git repo [00:00:02]
Scanning content  ████████████████████ 100%  1.06 GiB/1.06 GiB  [00:00:00]                                                                                                                                                                                                                                                                                                  Scanned 1.06 GiB from 72,901 blobs in 0 seconds (2.15 GiB/s); 4/4 new matches

 Rule                Findings   Matches   Accepted   Rejected   Mixed   Unlabeled
──────────────────────────────────────────────────────────────────────────────────
 GitHub Secret Key          1         4          0          0       0           1

Run the `report` command next to show finding details.

The details of the findings:

% ./release/bin/noseyparker report
Finding 1/1
Rule: GitHub Secret Key
Group: 31c86eb3b33c9b601a1f60f98dcbfd1d70f379b4
Showing 3/4 occurrences:

    Occurrence 1/4
    Git repo: /Users/blarsen/projects/noseyparker/datastore.np/clones/https/github.com/Homebrew/brew
    Commit: first seen in e2e956598439d4f65886da0cbf7796a4711ed703

        Author:     Thierry Moisan <[email protected]>
        Date:       2024-07-13
        Summary:    workflows: pin actions
        Path:       .github/workflows/vendor-gems.yml

    Blob: 472704d576a457adc7bdcc781eebf5976bfa3933 (4209 bytes, text/x-yaml, unknown charset)
    Lines: 101:30-101:86

        I files for ${GEM_NAME}." \
                               -m "Autogenerated by the [vendor-gems](https://github.com/Homebrew/brew/blob/HEAD/.github/workflows/vendor-gems.yml) workflow."
                  fi

              - name: Generate push token
                uses: actions/create-github-app-token@31c86eb3b33c9b601a1f60f98dcbfd1d70f379b4 # v1
                id: app-token
                if: github.event_name == 'pull_request_target' || github.event_name == 'workflow_dispatch'
                with:
                  app-id: ${{ vars.BREW_COMMIT_APP_ID }}
                  private-key: ${{ secrets.BREW_COMMIT_APP_KEY }}

              -

    Occurrence 2/4
    Git repo: /Users/blarsen/projects/noseyparker/datastore.np/clones/https/github.com/Homebrew/brew
    Commit: first seen in 5d6be127320a72561c80b33e038f343103f07868

        Author:     Ruoyu Zhong <[email protected]>
        Date:       2024-07-13
        Summary:    workflows/vendor-gems: handle input string more robustly
        Path:       .github/workflows/vendor-gems.yml

    Blob: 621ee9a5738ef6adfa5c2d1600463639267b4446 (4229 bytes, text/x-yaml, unknown charset)
    Lines: 102:30-102:86

        I files for ${GEM_NAME}." \
                               -m "Autogenerated by the [vendor-gems](https://github.com/Homebrew/brew/blob/HEAD/.github/workflows/vendor-gems.yml) workflow."
                  fi

              - name: Generate push token
                uses: actions/create-github-app-token@31c86eb3b33c9b601a1f60f98dcbfd1d70f379b4 # v1
                id: app-token
                if: github.event_name == 'pull_request_target' || github.event_name == 'workflow_dispatch'
                with:
                  app-id: ${{ vars.BREW_COMMIT_APP_ID }}
                  private-key: ${{ secrets.BREW_COMMIT_APP_KEY }}

              -

    Occurrence 3/4
    Git repo: /Users/blarsen/projects/noseyparker/datastore.np/clones/https/github.com/Homebrew/brew
    Commit: first seen in 41c43dcd9fb46bd42c8505b771a9f9e3dad57a25

        Author:     Ruoyu Zhong <[email protected]>
        Date:       2024-07-14
        Summary:    workflows/vendor-gem: simplify condition
        Path:       .github/workflows/vendor-gems.yml

    Blob: 8fda5885aed3dae39f8ccbdeae6e0fc5dbd301e2 (3564 bytes, text/x-yaml, unknown charset)
    Lines: 93:30-93:86

        I files for ${GEM_NAME}." \
                               -m "Autogenerated by the [vendor-gems](https://github.com/Homebrew/brew/blob/HEAD/.github/workflows/vendor-gems.yml) workflow."
                  fi

              - name: Generate push token
                uses: actions/create-github-app-token@31c86eb3b33c9b601a1f60f98dcbfd1d70f379b4 # v1
                id: app-token
                if: github.event_name == 'workflow_dispatch'
                with:
                  app-id: ${{ vars.BREW_COMMIT_APP_ID }}
                  private-key: ${{ secrets.BREW_COMMIT_APP_KEY }}

              - name: Push to pull request
                if: github

In this case, these are all false positives from a rule that is somewhat imprecise.

The test assertion in the formula probably needs to be rephrased.

@bradlarsen
Copy link
Contributor

P.S. I'm the author and maintainer of Nosey Parker. Is there by chance some what that I can be automatically notified of new noseyparker PRs in Homebrew?

@chenrui333
Copy link
Member

P.S. I'm the author and maintainer of Nosey Parker. Is there by chance some what that I can be automatically notified of new noseyparker PRs in Homebrew?

👋 I dont think you can do that with the notification part :(

noseyparker: update test

Signed-off-by: Rui Chen <[email protected]>
@chenrui333 chenrui333 force-pushed the bump-noseyparker-0.19.0 branch from 3171d9e to 951a923 Compare August 1, 2024 17:33
@chenrui333 chenrui333 added ready to merge PR can be merged once CI is green and removed test failure CI fails while running the test-do block CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. labels Aug 1, 2024
@chenrui333
Copy link
Member

chenrui333 commented Aug 1, 2024

I just updated the test to use a much smaller repo.

@p-linnane
Copy link
Member

@bradlarsen Our autobump workflow checks for new versions and opens PR's every 3 hours. Once you publish a release, you can expect there will be a PR opened within that timeframe. As @chenrui333 said, unfortunately there's no way for us to have you pinged when a PR is opened.

Copy link
Contributor

github-actions bot commented Aug 1, 2024

🤖 An automated task has requested bottles to be published to this PR.

@github-actions github-actions bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label Aug 1, 2024
@BrewTestBot BrewTestBot enabled auto-merge August 1, 2024 21:36
@BrewTestBot BrewTestBot added this pull request to the merge queue Aug 1, 2024
Merged via the queue into master with commit 8ff165a Aug 1, 2024
15 checks passed
@BrewTestBot BrewTestBot deleted the bump-noseyparker-0.19.0 branch August 1, 2024 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
boost Boost use is a significant feature of the PR or issue bump-formula-pr PR was created using `brew bump-formula-pr` CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. ready to merge PR can be merged once CI is green rust Rust use is a significant feature of the PR or issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants