Feat/new implementation cyclonedx bom #532

gordonrousselle · 2024-10-27T18:08:16Z

This is a proposal implementation for the next major version of the cyclonedx gradle plugin. The reason for this is twofold. Firstly, this implementation will support Gradle's configuration cache which will be enforced by default in the near future. Secondly, it aims to resolve some of the open issues reported against the current implementation.

Most of the existing features are still supported, but a couple of points are worth noting:

child projects, if exists, are now also included as component in the resulting BOM
component that represent gradle projects have purls without a type field (defaulted to JAR in previous version)
Version 1.0 of the CycloneDX BOM is not supported anymore

Signed-off-by: Gordon <[email protected]>

…for non existing file extensions Signed-off-by: Gordon <[email protected]>

Signed-off-by: Gordon <[email protected]>

skhokhlov · 2024-10-28T16:57:47Z

I think this PR should resolve #528, @gordonrousselle can you please add test for it?

Signed-off-by: Gordon <[email protected]>

src/main/java/org/cyclonedx/gradle/SbomGraphProvider.java

src/main/java/org/cyclonedx/gradle/DependencyGraphTraverser.java

src/main/java/org/cyclonedx/gradle/model/SbomComponent.java

Signed-off-by: Gordon <[email protected]>

src/main/java/org/cyclonedx/gradle/CycloneDxTask.java

src/main/java/org/cyclonedx/gradle/DependencyGraphTraverser.java

src/main/java/org/cyclonedx/gradle/SbomGraphProvider.java

src/main/java/org/cyclonedx/gradle/SbomBuilder.java

src/main/java/org/cyclonedx/gradle/model/SbomComponent.java

src/main/java/org/cyclonedx/gradle/utils/CycloneDxUtils.java

…date sbom, missing nullables and remove duplicated code Signed-off-by: Gordon <[email protected]>

…ackage:projectsAndScopes properties (adding back in later PR) Signed-off-by: Gordon <[email protected]>

…ovider package private Signed-off-by: Gordon <[email protected]>

Signed-off-by: Gordon <[email protected]>

…for non existing file extensions Signed-off-by: Gordon <[email protected]>

Signed-off-by: Gordon <[email protected]>

…date sbom, missing nullables and remove duplicated code Signed-off-by: Gordon <[email protected]>

…ackage:projectsAndScopes properties (adding back in later PR) Signed-off-by: Gordon <[email protected]>

…ovider package private Signed-off-by: Gordon <[email protected]>

….com/gordonrousselle/cyclonedx-gradle-plugin into feat/new-implementation-cyclonedxBom

Signed-off-by: Gordon <[email protected]>

skhokhlov · 2024-11-21T11:26:16Z

@stevespringett @DarthHater @nscuro @mr-zepol Do you have any concerns here? If not, I'll merge it tomorrow

jeremylong · 2024-11-21T15:58:58Z

Somehow I missed this PR - I was about to submit a PR that migrates from Configuration.getResolvedConfiguration() to Configuration.getIncoming() because I have an Android project that is facing issues with variant selection. My PR is not as substantial - but does solve the issue with variant selection (in my case). However, when I use this implementation I still face:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:cyclonedxBom'.
> Could not resolve all artifacts for configuration ...

The (trimmed) exception is:

...
Cause 22: org.gradle.internal.component.AmbiguousArtifactVariantsException: The consumer was configured to find a library for use during compile-time, preferably optimized for Android, as well as attribute 
...
 at org.cyclonedx.gradle.SbomGraphProvider.lambda$getArtifacts$6(SbomGraphProvider.java:131)

I'm looking into this more to see if I can provide a suggestion/workaround.

jeremylong · 2024-11-21T16:08:30Z

src/main/java/org/cyclonedx/gradle/SbomGraphProvider.java

+                .filter(configuration -> shouldIncludeConfiguration(configuration)
+                        && !shouldSkipConfiguration(configuration)
+                        && configuration.isCanBeResolved())
+                .flatMap(config -> config.getIncoming().getArtifacts().getArtifacts().stream())


Add lenient artifactView to avoid errors when selecting variants in Android projects:

Suggested change

.flatMap(config -> config.getIncoming().getArtifacts().getArtifacts().stream())

.flatMap(config -> config.getIncoming().artifactView(view -> {view.lenient(true);}).getArtifacts().getArtifacts().stream())

Note that this suggestion will likely fail spotless and this should be run through ./gradlew :spotlessApply.

skhokhlov · 2024-11-21T16:18:12Z

Hi @jeremylong, I'm very excited that such small change can fix Android issues. Can you please open a separate PR with this fix? Then we can properly test it and close related issues

jeremylong · 2024-11-21T16:24:24Z

The "small" change - requires moving from Configuration.getResolvedConfiguration() to Configuration.getIncoming() And using a lenient artifactView. As such, this full PR would be needed - plus my small addition.

I might have a minor fix I can propose to the current implementation. Let me test something.

jeremylong · 2024-11-21T16:35:32Z

It turns out even using a lenient configuration using the legacy getResolvedConfiguration() still fails for Android:

        final LenientConfiguration resolvedConfiguration =
                configuration.getResolvedConfiguration().getLenientConfiguration();
        final Set<ResolvedDependency> directModuleDependencies =
                resolvedConfiguration.getFirstLevelModuleDependencies();

As such, this PR - with the addition of the lenient artifact view appears to solve the android issues.

jeremylong · 2024-11-21T17:02:47Z

src/main/java/org/cyclonedx/gradle/DependencyGraphTraverser.java

+                        dependencyNode.inScopeConfiguration(projectName, configName);
+                        graph.get(graphNode).add(dependencyNode);
+                        queue.add(dependencyNode);
+                    }


Should we log any UnresolvedDependencyResult?

Suggested change

}

} else if (dep instanceof UnresolvedDependencyResult) {

UnresolvedDependencyResult unresolved = (UnresolvedDependencyResult) dep;

logger.debug(

"Unable to resolve artifact `{}` because {}",

unresolved.getAttempted().getDisplayName(),

unresolved.getFailure().toString());

}

Good suggestion. I have added it in.

jeremylong · 2024-11-21T17:05:08Z

I could submit another PR that isn't as large, follows more of the code paths that were used previously, and solves the Android issues. But honestly, the code in this PR is really good and you should just go with this one.

Signed-off-by: Gordon <[email protected]>

jeremylong · 2024-11-22T17:05:30Z

I'll submit another PR to add the lenient artifactView.

gordonrousselle requested a review from a team as a code owner October 27, 2024 18:08

This was linked to issues Oct 28, 2024

Mark components from test configurations #440

Closed

Gradle configurations not being merged, resulting bom is made by a single random configuration #367

Closed

gordonrousselle added 5 commits October 28, 2024 11:49

feat: initial new task implementation

357abd4

Signed-off-by: Gordon <[email protected]>

feat: use single graph by merging all graphs for every configuration

09e73ce

Signed-off-by: Gordon <[email protected]>

fix: enable cache for multi node project and return empty classifier …

f82f1ce

…for non existing file extensions Signed-off-by: Gordon <[email protected]>

fix: provide artifacts lazily

010b8d1

Signed-off-by: Gordon <[email protected]>

feat: support build configuration cache

e016f01

Signed-off-by: Gordon <[email protected]>

gordonrousselle force-pushed the feat/new-implementation-cyclonedxBom branch from 487c48f to e016f01 Compare October 28, 2024 11:51

skhokhlov mentioned this pull request Oct 28, 2024

NPE fails build during cyclonedxBom task #500

Closed

skhokhlov added this to the 2.0.0 milestone Oct 28, 2024

skhokhlov marked this pull request as draft November 1, 2024 22:15

gordonrousselle added 3 commits November 4, 2024 08:39

feat: implement custom configuration

ff26c9f

Signed-off-by: Gordon <[email protected]>

fix: add entry into cache

ac1c3c0

Signed-off-by: Gordon <[email protected]>

fix: getGroup and getVersion do not return String objects

4f0b179

Signed-off-by: Gordon <[email protected]>

skhokhlov reviewed Nov 4, 2024

View reviewed changes

feat: add small fixes, javadoc, logging and nullables.

e2a2290

Signed-off-by: Gordon <[email protected]>

gordonrousselle mentioned this pull request Nov 5, 2024

cyclonedxBom task does not finish - endless loop on creating bom #528

Closed

skhokhlov requested changes Nov 5, 2024

View reviewed changes

skhokhlov removed a link to an issue Nov 7, 2024

Mark components from test configurations #440

Closed

feat: log configuration parameters, use empty list for licenses, vali…

be08c09

…date sbom, missing nullables and remove duplicated code Signed-off-by: Gordon <[email protected]>

gordonrousselle force-pushed the feat/new-implementation-cyclonedxBom branch from f235111 to be08c09 Compare November 11, 2024 19:39

gordonrousselle added 2 commits November 11, 2024 20:08

feat: add additional logging for purl failures and remove cdx:maven:p…

5a5596b

…ackage:projectsAndScopes properties (adding back in later PR) Signed-off-by: Gordon <[email protected]>

feat: parse plugin.properties for tools metadata and make SbomGraphPr…

a2c603f

…ovider package private Signed-off-by: Gordon <[email protected]>

skhokhlov marked this pull request as ready for review November 13, 2024 15:16

gordonrousselle added 4 commits November 13, 2024 15:20

feat: initial new task implementation

9ea6d0e

Signed-off-by: Gordon <[email protected]>

feat: use single graph by merging all graphs for every configuration

b013821

Signed-off-by: Gordon <[email protected]>

fix: enable cache for multi node project and return empty classifier …

069d316

…for non existing file extensions Signed-off-by: Gordon <[email protected]>

fix: provide artifacts lazily

3a3ec89

Signed-off-by: Gordon <[email protected]>

gordonrousselle added 8 commits November 13, 2024 15:20

feat: support build configuration cache

78e6584

Signed-off-by: Gordon <[email protected]>

feat: implement custom configuration

6f0cf7b

Signed-off-by: Gordon <[email protected]>

fix: add entry into cache

babadc8

Signed-off-by: Gordon <[email protected]>

fix: getGroup and getVersion do not return String objects

686bf45

Signed-off-by: Gordon <[email protected]>

feat: add small fixes, javadoc, logging and nullables.

8fb80ab

Signed-off-by: Gordon <[email protected]>

feat: log configuration parameters, use empty list for licenses, vali…

15564d5

…date sbom, missing nullables and remove duplicated code Signed-off-by: Gordon <[email protected]>

feat: add additional logging for purl failures and remove cdx:maven:p…

8f204a8

…ackage:projectsAndScopes properties (adding back in later PR) Signed-off-by: Gordon <[email protected]>

feat: parse plugin.properties for tools metadata and make SbomGraphPr…

7a81820

…ovider package private Signed-off-by: Gordon <[email protected]>

skhokhlov force-pushed the feat/new-implementation-cyclonedxBom branch from a2c603f to 7a81820 Compare November 13, 2024 15:20

gordonrousselle added 2 commits November 13, 2024 15:36

Merge branch 'feat/new-implementation-cyclonedxBom' of https://github…

9927718

….com/gordonrousselle/cyclonedx-gradle-plugin into feat/new-implementation-cyclonedxBom

fix: ignore directories when checking for generated bom files

2b8aebe

Signed-off-by: Gordon <[email protected]>

skhokhlov approved these changes Nov 13, 2024

View reviewed changes

jkowalleck requested a review from a team November 13, 2024 16:53

jeremylong reviewed Nov 21, 2024

View reviewed changes

feat: info logging for unresolved dependencies

be924d6

Signed-off-by: Gordon <[email protected]>

skhokhlov merged commit e24493f into CycloneDX:master Nov 22, 2024
7 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Dec 7, 2024

This was linked to issues Dec 19, 2024

NPE fails build during cyclonedxBom task #500

Closed

Generate reproducible outcome #292

Open

Can aar types be supported ？ #234

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/new implementation cyclonedx bom #532

Feat/new implementation cyclonedx bom #532

gordonrousselle commented Oct 27, 2024 •

edited

Loading

skhokhlov commented Oct 28, 2024

skhokhlov commented Nov 21, 2024

jeremylong commented Nov 21, 2024

jeremylong Nov 21, 2024

skhokhlov commented Nov 21, 2024

jeremylong commented Nov 21, 2024

jeremylong commented Nov 21, 2024

jeremylong Nov 21, 2024

gordonrousselle Nov 22, 2024

jeremylong commented Nov 21, 2024

jeremylong commented Nov 22, 2024

	.flatMap(config -> config.getIncoming().getArtifacts().getArtifacts().stream())
	.flatMap(config -> config.getIncoming().artifactView(view -> {view.lenient(true);}).getArtifacts().getArtifacts().stream())

-                    }
+                    } else if (dep instanceof UnresolvedDependencyResult) {
+                        UnresolvedDependencyResult unresolved = (UnresolvedDependencyResult) dep;
+                        logger.debug(
+                                "Unable to resolve artifact `{}` because {}",
+                                unresolved.getAttempted().getDisplayName(),
+                                unresolved.getFailure().toString());
+                    }

Feat/new implementation cyclonedx bom #532

Feat/new implementation cyclonedx bom #532

Conversation

gordonrousselle commented Oct 27, 2024 • edited Loading

skhokhlov commented Oct 28, 2024

skhokhlov commented Nov 21, 2024

jeremylong commented Nov 21, 2024

jeremylong Nov 21, 2024

Choose a reason for hiding this comment

skhokhlov commented Nov 21, 2024

jeremylong commented Nov 21, 2024

jeremylong commented Nov 21, 2024

jeremylong Nov 21, 2024

Choose a reason for hiding this comment

gordonrousselle Nov 22, 2024

Choose a reason for hiding this comment

jeremylong commented Nov 21, 2024

jeremylong commented Nov 22, 2024

gordonrousselle commented Oct 27, 2024 •

edited

Loading