Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix docker extract bugs #1513

Merged
merged 3 commits into from
Dec 26, 2024
Merged

Fix docker extract bugs #1513

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e1dcfc2
Test docker extract bugs
prabhu Dec 24, 2024
56fc686
Support for fail-on-error for container sbom generation. Env variable…
prabhu Dec 26, 2024
1651977
Support for fail-on-error for container sbom generation. Env variable…
prabhu Dec 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 26 additions & 16 deletions .github/workflows/dockertests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ concurrency:
jobs:
linux-tests:
strategy:
fail-fast: true
matrix:
os: [ubuntu-latest]
node-version: ['23.x']
Expand Down Expand Up @@ -50,32 +51,36 @@ jobs:
path: 'repotests/grafana-operator'
- name: dockertests
run: |
bin/cdxgen.js ubuntu:latest -t docker -o bomresults/bom-ubuntu.json
bin/cdxgen.js ubuntu:latest -t docker -o bomresults/bom-ubuntu.json --fail-on-error
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running with --fail-on-error would improve confidence in the tests.

docker rmi ubuntu:latest
bin/cdxgen.js almalinux:9.4-minimal -t docker -o bomresults/bom-almalinux.json
bin/cdxgen.js alpine:latest -t docker -o bomresults/bom-alpine.json --fail-on-error
docker rmi alpine:latest
bin/cdxgen.js almalinux:9.4-minimal -t docker -o bomresults/bom-almalinux.json --fail-on-error
docker rmi almalinux:9.4-minimal
bin/cdxgen.js centos:latest -t oci -o bomresults/bom-centos.json
bin/cdxgen.js centos:latest -t oci -o bomresults/bom-centos.json --fail-on-error
docker rmi centos:latest
bin/cdxgen.js phpmyadmin@sha256:1092481630056189e43cc0fe66fd01defcc9334d78ab4611b22f65e9a39869bd -o bomresults/bom-phpmyadmin.json --validate
bin/cdxgen.js phpmyadmin@sha256:1092481630056189e43cc0fe66fd01defcc9334d78ab4611b22f65e9a39869bd -o bomresults/bom-phpmyadmin.json --fail-on-error
docker rmi phpmyadmin@sha256:1092481630056189e43cc0fe66fd01defcc9334d78ab4611b22f65e9a39869bd
bin/cdxgen.js shiftleft/scan-slim -o bomresults/bom-scanslim.json -t container --validate
bin/cdxgen.js shiftleft/scan-slim -o bomresults/bom-scanslim.json -t container --fail-on-error
docker rmi shiftleft/scan-slim
bin/cdxgen.js redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e -o bomresults/bom-redmine.json --validate
bin/cdxgen.js redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e -o bomresults/bom-redmine.json --fail-on-error
docker rmi redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e
bin/cdxgen.js rocket.chat@sha256:379f7afa0e67497c363ac9a9b3e7e6a6d31deee228233307c987e4a0c68b28e6 -o bomresults/bom-rocket.json --validate
bin/cdxgen.js rocket.chat@sha256:379f7afa0e67497c363ac9a9b3e7e6a6d31deee228233307c987e4a0c68b28e6 -o bomresults/bom-rocket.json --fail-on-error
docker rmi rocket.chat@sha256:379f7afa0e67497c363ac9a9b3e7e6a6d31deee228233307c987e4a0c68b28e6
bin/cdxgen.js sonarqube@sha256:7c0edcb99c964984db6d24330db33bb12de1e8ae0d5974d77640b1efea1483d1 -o bomresults/bom-sonar.json --validate
bin/cdxgen.js sonarqube@sha256:7c0edcb99c964984db6d24330db33bb12de1e8ae0d5974d77640b1efea1483d1 -o bomresults/bom-sonar.json --fail-on-error
docker rmi sonarqube@sha256:7c0edcb99c964984db6d24330db33bb12de1e8ae0d5974d77640b1efea1483d1
bin/cdxgen.js zookeeper@sha256:5bf00616677db5ef57d8a2da7c5dadf67f1a6be54b0c33a79be3332c9c80aeb6 -o bomresults/bom-zoo.json --validate
bin/cdxgen.js zookeeper@sha256:5bf00616677db5ef57d8a2da7c5dadf67f1a6be54b0c33a79be3332c9c80aeb6 -o bomresults/bom-zoo.json --fail-on-error
docker rmi zookeeper@sha256:5bf00616677db5ef57d8a2da7c5dadf67f1a6be54b0c33a79be3332c9c80aeb6
docker pull shiftleft/scan-slim:latest
docker save shiftleft/scan-slim:latest -o /tmp/scanslim.tar
docker rmi shiftleft/scan-slim:latest
bin/cdxgen.js /tmp/scanslim.tar -o bomresults/bom-scanarch.json --validate
bin/cdxgen.js -t docker-compose test/data -o bomresults/bom-dc.json --validate
bin/cdxgen.js -t operator repotests/grafana-operator -o bomresults/bom-op.json --validate
bin/cdxgen.js /tmp/scanslim.tar -o bomresults/bom-scanarch.json --fail-on-error
bin/cdxgen.js -t docker-compose test/data -o bomresults/bom-dc.json --fail-on-error
bin/cdxgen.js -t operator repotests/grafana-operator -o bomresults/bom-op.json --fail-on-error
rm /tmp/scanslim.tar
ls -ltr bomresults
env:
CDXGEN_DEBUG_MODE: debug
linux-dockertar-tests:
strategy:
matrix:
Expand Down Expand Up @@ -119,9 +124,14 @@ jobs:
docker pull elasticsearch@sha256:3686a5757ed46c9dbcf00f6f71fce48ffc5413b193a80d1c46a21e7aad4c53ad
docker save -o /tmp/elastic.tar elasticsearch@sha256:3686a5757ed46c9dbcf00f6f71fce48ffc5413b193a80d1c46a21e7aad4c53ad
docker rmi elasticsearch@sha256:3686a5757ed46c9dbcf00f6f71fce48ffc5413b193a80d1c46a21e7aad4c53ad
bin/cdxgen.js /tmp/elastic.tar -t docker -o bomresults/bom-elastic.tar.json --validate
bin/cdxgen.js /tmp/elastic.tar -t docker -o bomresults/bom-elastic.tar.json --fail-on-error
docker pull alpine:latest
docker save -o /tmp/alpine.tar alpine:latest
docker rmi alpine:latest
bin/cdxgen.js /tmp/alpine.tar -t docker -o bomresults/bom-alpine.tar.json --fail-on-error
ls -ltr bomresults

env:
CDXGEN_DEBUG_MODE: debug
os-tests:
runs-on: ubuntu-latest

Expand Down Expand Up @@ -157,7 +167,7 @@ jobs:
CI: true
- name: ostests
run: |
bin/cdxgen.js -t os -o bomresults/bom-os.json --validate
bin/cdxgen.js -t os -o bomresults/bom-os.json --fail-on-error
env:
CDXGEN_DEBUG_MODE: debug
- uses: actions/upload-artifact@v4
Expand Down Expand Up @@ -196,7 +206,7 @@ jobs:
CI: true
- name: wintests
run: |
node bin/cdxgen.js -t os -o bomresults/bom-win.json --validate
node bin/cdxgen.js -t os -o bomresults/bom-win.json --fail-on-error
dir bomresults
env:
CDXGEN_DEBUG_MODE: debug
Expand Down
4 changes: 2 additions & 2 deletions lib/cli/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6627,7 +6627,7 @@ export async function createBom(path, options) {
// Docker and image archive support
// TODO: Support any source archive
if (path.endsWith(".tar") || path.endsWith(".tar.gz")) {
exportData = await exportArchive(path);
exportData = await exportArchive(path, options);
if (!exportData) {
console.log(
`OS BOM generation has failed due to problems with exporting the image ${path}`,
Expand All @@ -6646,7 +6646,7 @@ export async function createBom(path, options) {
path.includes("@sha256") ||
path.includes(":latest")
) {
exportData = await exportImage(path);
exportData = await exportImage(path, options);
if (exportData) {
isContainerMode = true;
} else {
Expand Down
6 changes: 6 additions & 0 deletions lib/helpers/utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1336,6 +1336,9 @@ export async function parsePkgLock(pkgLockFile, options = {}) {
console.log(
`Unable to parse ${pkgLockFile} without legacy peer dependencies. Retrying ...`,
);
if (DEBUG_MODE) {
console.log(e);
}
try {
arb = new Arborist({
path: path.dirname(pkgLockFile),
Expand All @@ -1346,6 +1349,9 @@ export async function parsePkgLock(pkgLockFile, options = {}) {
console.log(
`Unable to parse ${pkgLockFile} in legacy and non-legacy mode. The resulting SBOM would be incomplete.`,
);
if (DEBUG_MODE) {
console.log(e);
}
return { pkgList, dependenciesList };
}
}
Expand Down
3 changes: 3 additions & 0 deletions lib/managers/binary.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,9 @@ export function getOSPackages(src) {
}
if (osReleaseData["VERSION_ID"]) {
distro_id = `${distro_id}-${osReleaseData["VERSION_ID"]}`;
if (OS_DISTRO_ALIAS[distro_id]) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes some warning for dependency tree in debian images.

distro_codename = OS_DISTRO_ALIAS[distro_id];
}
}
const tmpDependencies = {};
(tmpBom.dependencies || []).forEach((d) => {
Expand Down
Loading
Loading