(Cluster)BuildStrategy CR about buildpacks are outdated

[cmoulliard]

Issue

The (Cluster)BuildStrategy CR about buildpacks is outdated

According to @BartWeller and as per https://github.com/buildpacks/spec/blob/main/platform.md#build
A platform MUST execute these phases either by invoking the following phase-specific lifecycle binaries in order:

/cnb/lifecycle/analyzer
/cnb/lifecycle/detector
/cnb/lifecycle/restorer
/cnb/lifecycle/builder
/cnb/lifecycle/exporter

The version of the platform should also bumped to 0.11

Important: We should certainly find a way to align what Shipwright uses vs Tekton vs Openshift Pipelines as different tasks already exist:

• Creator: https://github.com/tektoncd/catalog/blob/main/task/buildpacks/0.6/buildpacks.yaml
• Individual steps: https://github.com/tektoncd/catalog/blob/main/task/buildpacks-phases/0.2/buildpacks-phases.yaml
• Openshift Pipelines: https://github.com/openshift-pipelines/task-buildpacks/

Questions:

• Should we develop a Shipwright Buildpack strategy from scratch ?
• Should we convert one of the existing task ? How and using which one ?

@otaviof

[BarDweller]

Adding comments here for future reference...

Anything that performs a build using buildpacks, counts as an implementation of the Buildpack platform spec, and should follow that spec. Only the most recent spec releases (0.10 onwards) include support for extensions.. Extensions are required for the upcoming Paketo UBI stacks, as the Extensions manage the dependency installs via dnf.

The current yaml implements the platform spec as per release 0.4, which is why the order of the phases disagrees with the current spec (the order of analyzer & detector inverted at spec 0.7).

Here's 10k ft high view of what that means from a platform implementation perspective.

Performing a build with buildpacks essentially boils down to 'spinning up a container based on the builder image, and invoking the lifecycle binaries inside with the appropriate volume mounts and arguments'. The lifecycle binaries then handle the coordination of detecting buildpacks & performing the build with them.

This still holds true for specs at 0.10 and beyond, although there are some differences to note.

In the past, you've had the choice of using the creator lifecycle binary, or invoking each of the phases 'analyze', 'detect', 'restore', 'build', 'export' sequentially in new containers, each based from the builder image. The creator binary does not support extensions, and so platforms implementing 0.10 onwards must use separate phase executions for builder images that include extensions. Additionally, when performing a phased build with extensions, you do not drive the build lifecycle binary, but instead drive the extender binary.

If you want to see the sequence of the phases being invoked, with mounts, and arguments, you can have a peek at https://github.com/BarDweller/buildpack-platform-bash/blob/main/bash_build.sh which assumes you are building with an image using extensions, and does the separate phase invocations. Afaik, no tekton pipeline exists today that demonstrates drives the extender path.

It's probably also worth saying that as a platform, you can check if the builder image contains extensions by seeing if io.buildpacks.extension.layers and io.buildpacks.buildpack.order-extensions are populated as labels.

I think tekton pipelines support setting variables and making tasks dependent on variable content, so it should be possible to determine if the image has those labels set, and then use multi-phase execution where the builder is using extensions.

Also, from the bash platform, pay attention to the userid/groupid the containers are invoked as, with most of them being CNB_USER_ID and CNB_GROUP_ID from the builder image environment, except the extender phase that runs as 0:0 with CNB_USER_ID and CNB_GROUP_ID passed via -uid and -gid

[cmoulliard]

I tested successfully this updated strategy https://github.com/redhat-buildpacks/testing/blob/a8cfe9664ac96652bd0e136ee4b708706c08e08a/k8s/shipwright/unsecured/clusterbuildstrategy.yml with the UBI Builder image (java, nodejs) top of a quarkus project on a kind cluster + tekton (v0.56) and shipwright 0.11

Job result: https://github.com/redhat-buildpacks/testing/actions/runs/8658401429/job/23742216328?pr=51

@adambkaplan @BarDweller

[cmoulliard]

Some todos:

• Review the steps definition (env vars, params, volumes/workspaces) to align what we ship with tekton buildpacks tasks
• Ideally the buildStrategy (kind of Pipeline) should be generated OOTB from a Pipeline to avoid to have to maintain twice the same code
• Set UID/GID parameters which are currently hard coded with the values calculated from a step able to read the content of the UBI builder image to get the proper values 1001 or 1002 or etc to be used. See ticket 1354
• Support conditional steps to use lifecycle/extender or lifecycle/build as we do here with Tekton using buildpack task or buildpack-extension task. See ticket 1577

@adambkaplan