-
Notifications
You must be signed in to change notification settings - Fork 597
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert to OCI builder #125
base: master
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
**/*.txt eol=lf | ||
Makefile eol=lf | ||
hello.c eol=lf | ||
/oci/*/** linguist-generated |
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,205 @@ | ||
#!/usr/bin/env bash | ||
set -Eeuo pipefail | ||
|
||
image="${1:-hello-world}" | ||
|
||
hash="sha256" | ||
|
||
self="$(basename "$BASH_SOURCE")" | ||
cd "$(dirname "$(readlink -f "$BASH_SOURCE")")" | ||
|
||
ociroot="./oci" | ||
blobs="$ociroot/blobs/$hash" | ||
|
||
rm -rf $ociroot | ||
|
||
mkdir -p $blobs | ||
|
||
toBlob() { | ||
local content="$1"; shift | ||
|
||
digest="$(echo -n $content | sha256sum | cut -d " " -f1)" | ||
echo -n $content > "$blobs/$digest" | ||
echo $digest | ||
} | ||
|
||
annotations() { | ||
local os="$1"; shift | ||
local arch="$1"; shift | ||
local timestamp="$1"; shift | ||
local revision="$1"; shift | ||
|
||
jq -r -n --arg os $os --arg arch $arch --arg timestamp $timestamp --arg revision $revision '{ | ||
"com.docker.official-images.bashbrew.arch": $arch, | ||
"org.opencontainers.image.base.name": "scratch", | ||
"org.opencontainers.image.created": $timestamp, | ||
"org.opencontainers.image.revision": $revision, | ||
"org.opencontainers.image.source": "https://github.com/docker-library/hello-world.git#\($revision):\($arch)/hello-world", | ||
"org.opencontainers.image.url": "https://hub.docker.com/_/hello-world", | ||
"org.opencontainers.image.version": $os | ||
LaurentGoderre marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | tojson' | ||
} | ||
|
||
rootfs() { | ||
local os="$1"; shift | ||
local arch="$1"; shift | ||
|
||
path="./$arch/hello-world" | ||
tar -C $path -cf - hello > tmp.tar | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we're going to commit these tarballs (and even if we're not, really), we should go out of our way to make sure they are generated reproducibly like we do in BusyBox, which is probably going to involve/require we build them inside a container or image (such as the |
||
digest="$(cat tmp.tar | sha256sum | cut -d " " -f1)" | ||
mv tmp.tar "$blobs/$digest" | ||
echo $digest | ||
} | ||
|
||
config() { | ||
local digest="$1"; shift | ||
local os="$1"; shift | ||
local arch="$1"; shift | ||
local timestamp="$1"; shift | ||
|
||
configArch="" | ||
|
||
case "$arch" in | ||
arm64v8) | ||
configArch="arm64" | ||
;; | ||
arm32v*) | ||
configArch="arm" | ||
;; | ||
*) | ||
configArch="$arch" | ||
;; | ||
esac | ||
|
||
content="$(jq -r -n --arg digest "$hash:$digest" --arg os $os --arg arch $configArch --arg timestamp $timestamp '{ | ||
config: { | ||
Cmd: [ | ||
"/hello" | ||
], | ||
Env: [ | ||
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" | ||
], | ||
WorkingDir: "/" | ||
}, | ||
os: $os, | ||
architecture: $arch, | ||
rootfs: { | ||
diff_ids: [ | ||
$digest | ||
], | ||
type: "layers" | ||
}, | ||
"created": $timestamp, | ||
} | tojson')" | ||
|
||
toBlob $content | ||
LaurentGoderre marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
manifest() { | ||
local configdigest="$1"; shift | ||
local rootfsdigest="$1"; shift | ||
local annotations="$1"; shift | ||
|
||
configsize="$(wc -c "$blobs/$configdigest" | awk '{print $1}')" | ||
rootfssize="$(wc -c "$blobs/$rootfsdigest" | awk '{print $1}')" | ||
|
||
content="$(jq -r -n \ | ||
--arg configdigest "$hash:$configdigest" \ | ||
--arg configsize $configsize \ | ||
--arg rootfsdigest "$hash:$rootfsdigest" \ | ||
--arg rootfssize $rootfssize \ | ||
--argjson annotations $annotations \ | ||
'{ | ||
schemaVersion: 2, | ||
mediaType: "application/vnd.oci.image.manifest.v1+json", | ||
config: { | ||
mediaType: "application/vnd.oci.image.config.v1+json", | ||
digest: $configdigest, | ||
size: $configsize | tonumber | ||
}, | ||
layers: [ | ||
{ | ||
mediaType: "application/vnd.oci.image.layer.v1.tar", | ||
digest: $rootfsdigest, | ||
size: $rootfssize | tonumber | ||
} | ||
], | ||
annotations: $annotations | ||
} | tojson')" | ||
|
||
toBlob $content | ||
} | ||
|
||
indexdescriptor() { | ||
local manifestdigest="$1"; shift | ||
local os="$1"; shift | ||
local arch="$1"; shift | ||
local annotations="$1"; shift | ||
|
||
manifestsize="$(wc -c "$blobs/$manifestdigest" | awk '{print $1}')" | ||
|
||
platform="" | ||
|
||
case "$arch" in | ||
arm64v8) | ||
platform=$(jq -r -n --arg os $os --arg arch "arm64" --arg variant "v8" '{ | ||
architecture: $arch, | ||
os: $os, | ||
variant: $variant | ||
} | tojson') | ||
;; | ||
arm32v*) | ||
platform=$(jq -r -n --arg os $os --arg arch "arm" --arg variant "${arch/arm32/}" '{ | ||
architecture: $arch, | ||
os: $os, | ||
variant: $variant | ||
} | tojson') | ||
;; | ||
*) | ||
platform=$(jq -r -n --arg os $os --arg arch $arch '{ | ||
architecture: $arch, | ||
os: $os | ||
} | tojson') | ||
;; | ||
esac | ||
Comment on lines
+121
to
+144
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should let (however, we don't actually need this for |
||
|
||
jq -r -n \ | ||
--arg digest "$hash:$manifestdigest" \ | ||
--arg manifestsize $manifestsize \ | ||
--argjson platform $platform \ | ||
--argjson annotations $annotations \ | ||
'{ | ||
mediaType: "application/vnd.oci.image.manifest.v1+json", | ||
digest: $digest, | ||
size: $manifestsize | tonumber, | ||
platform: $platform, | ||
annotations: $annotations | ||
} | tojson' | ||
} | ||
|
||
created="2025-01-21T23:32:32Z" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any particular reason for this specific value? 😅 (long-term we probably need to scrape an appropriate value per-architecture from the latest Git commit to touch the relevant There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hahaha, that was s just the value from the current one in registry and forgot to change it |
||
|
||
arches=( *"/$image/hello" ) | ||
LaurentGoderre marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
manifests="[]" | ||
|
||
for path in "${arches[@]}"; do | ||
os="linux" | ||
arch="${path/\/hello\-world\/hello/}" | ||
|
||
annotations=$(annotations $os $arch $created "foo") | ||
rootfs=$(rootfs $os $arch) | ||
config=$(config $rootfs $os $arch $created) | ||
manifest=$(manifest $config $rootfs $annotations) | ||
indexdescriptor=$(indexdescriptor $manifest $os $arch $annotations) | ||
|
||
manifests=$(jq -r -n --argjson manifests $manifests --argjson manifest $indexdescriptor '$manifests | . += [$manifest] | tojson') | ||
done | ||
|
||
# TODO: Windows | ||
LaurentGoderre marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
jq -r -n --argjson manifests $manifests '{ | ||
schemaVersion: 2, | ||
mediaType: "application/vnd.oci.image.index.v1+json", | ||
manifests: $manifests | ||
}' > "$ociroot/index.json" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lumping all the architectures together in one big As much as I'd like to lump all these into a single OCI layout here (it's very cute!), I think we'll actually have a much better time downstream if we instead do a separate OCI layout per architecture ( |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might be interested in https://gist.github.com/tianon/35528d9851ebf0b0767c04cec5e01bcf, particularly https://gist.github.com/tianon/35528d9851ebf0b0767c04cec5e01bcf#file-oci-put 😅