Skip to content

Commit

Permalink
build: adds tooling for uploading to object storage
Browse files Browse the repository at this point in the history
  • Loading branch information
tdstein committed Dec 20, 2023
1 parent 5fda99f commit 340856a
Show file tree
Hide file tree
Showing 11 changed files with 179 additions and 67 deletions.
6 changes: 6 additions & 0 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ jobs:
archive:
needs: build
uses: ./.github/workflows/archive.yaml
upload:
needs:
- archive
- package
uses: ./.github/workflows/upload.yaml
secrets: inherit

# Integration Tests
bats:
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/pull-request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ jobs:
archive:
needs: build
uses: ./.github/workflows/archive.yaml
upload:
needs:
- archive
- package
uses: ./.github/workflows/upload.yaml
secrets: inherit

# Integration Tests
bats:
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ jobs:
needs:
- build
uses: ./.github/workflows/archive.yaml
upload:
needs:
- archive
- package
uses: ./.github/workflows/upload.yaml
secrets: inherit
release:
runs-on: ubuntu-latest
needs:
Expand All @@ -22,10 +28,6 @@ jobs:
with:
fetch-depth: 0
- uses: extractions/setup-just@v1
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-east-1
- uses: actions/download-artifact@v3
with:
name: archives
Expand All @@ -36,12 +38,10 @@ jobs:
path: packages
- id: get-prerelease
run: echo "prerelease=$(just pre-release)" >> "$GITHUB_OUTPUT"
- name: Release
uses: softprops/action-gh-release@v1
- uses: softprops/action-gh-release@v1
with:
draft: false
prerelease: ${{ steps.get-prerelease.outputs.prerelease == 'true' }}
files: |
archives/**/*
packages/**/*
- run: just release
23 changes: 23 additions & 0 deletions .github/workflows/upload.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Upload
on:
workflow_call:
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
jobs:
upload:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: ./.github/actions/setup
- uses: actions/download-artifact@v3
with:
name: archives
path: archives
- uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: us-east-1
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
- run: just upload
32 changes: 28 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,30 @@

## Quick Start

- [Contributing](#contributing)
- [Quick Start](#quick-start)
- [Getting Started](#getting-started)
- [Prerequisites](#prerequisites)
- [Option 1 - Native](#option-1---native)
- [Option 2 - Docker](#option-2---docker)
- [Installing](#installing)
- [Execution](#execution)
- [Testing](#testing)
- [Unit Tests](#unit-tests)
- [Coverage Reporting](#coverage-reporting)
- [Integration Tests](#integration-tests)
- [UI Tests](#ui-tests)
- [Development](#development)
- [Build Tools](#build-tools)
- [Environment Variables](#environment-variables)
- [Behavior in GitHub Actions](#behavior-in-github-actions)
- [Extension Development](#extension-development)
- [Release](#release)
- [Instructions](#instructions)
- [Pre-Releases](#pre-releases)
- [Release Lifecycle](#release-lifecycle)


The get this project up and running on your local machine, execute the following Just commands:

```console
Expand Down Expand Up @@ -106,7 +130,7 @@ When executing commands the following variables are accepted to change behavior.
| MODE | dev | enum | When set to `dev`, development is enabled. All other values disable development mode. |


#### Continuous Integration in GitHub Actions
#### Behavior in GitHub Actions

When running in GitHub Actions, the env variable `CI` is set to `true` by GitHub. When `CI=true`, the defaults for the following values are adjusted.

Expand All @@ -125,6 +149,8 @@ Execute `eval "$(just configure)"` to configure the executable on your current `

## Release

### Instructions

To start a release create a semver compatible tag.

_For this example, we will use the tag `v0.0.dev0`. This tag already exists, so you will not be able run the following commands verbatim._
Expand Down Expand Up @@ -158,9 +184,7 @@ For additional definitions see https://en.wikipedia.org/wiki/Software_release_li

Currently, the following suffix lineage is in use:

## Release Lifecycle

### Key
### Release Lifecycle

- `X`: The major version.
- `Y`: The minor version.
Expand Down
10 changes: 9 additions & 1 deletion build/ci/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM --platform=$BUILDPLATFORM ubuntu:jammy
FROM ubuntu:jammy

ARG NODEVERSION=18
RUN \
Expand Down Expand Up @@ -55,3 +55,11 @@ RUN\
&& go env -w GOPATH=/usr/local/go\
# staticcheck is required to for linting Go
&& go install honnef.co/go/tools/cmd/staticcheck@latest

# AWS is required by this project. The build tooling uses it!
ARG AWSARCHIVE=awscli-exe-linux-x86_64.zip
RUN curl -L -O "https://awscli.amazonaws.com/${AWSARCHIVE}" \
&& unzip -q "${AWSARCHIVE}" \
&& rm "${AWSARCHIVE}"\
&& ./aws/install\
&& rm -r ./aws
10 changes: 6 additions & 4 deletions install.bash
Original file line number Diff line number Diff line change
Expand Up @@ -168,10 +168,12 @@ MKTMP=("mktemp" "-d")
TAR=("tar")

# Variables
POSIT_PUBLISHER_NAME="publisher"
POSIT_PUBLISHER_VERSION="0.0.dev6"
POSIT_PUBLISHER_URL="https://cdn.posit.co/publisher/releases/tags/v${VERSION}"
POSIT_PUBLISHER_TMPDIR=$(execute "${MKTMP[@]}")
NAME="publisher"
PREFIX="/usr/local/bin"
VERSION="1.0.dev0"
URL="https://cdn.posit.co/publisher/releases/tags/v${VERSION}"
TMPDIR=$(execute "${MKTMP[@]}")


# OS specific settings
OS="$(uname)"
Expand Down
23 changes: 17 additions & 6 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ default:
just clean
just web
just build
just package
just archive

# Executes command against every justfile where avaiable. WARNING your mileage may very.
all +args='default':
Expand Down Expand Up @@ -208,21 +210,20 @@ name:
basename {{ _cmd }}

# Prints the pre-release status based on the version (see `just version`).
pre-release:
package:
#!/usr/bin/env bash
set -eou pipefail
{{ _with_debug }}
./scripts/is-pre-release.bash
just vscode package

# Releses the application. Releases are written to AWS S3. If invoked with `env CI=true` then releases are created for all architectures supported by the Go toolchain.
release:
# Prints the pre-release status based on the version (see `just version`).
pre-release:
#!/usr/bin/env bash
set -eou pipefail
{{ _with_debug }}
just _with_docker ./scripts/release.bash {{ _cmd }}
./scripts/is-pre-release.bash

# Runs the CLI via `go run`.
run *args:
Expand Down Expand Up @@ -263,6 +264,14 @@ test *args=("./..."): stub
just _with_docker go test {{ args }} -covermode set -coverprofile=cover.out

# Uploads distributions to object storage. If invoked with `env CI=true` then all architectures supported by the Go toolchain are uploaded.
upload:
#!/usr/bin/env bash
set -eou pipefail
{{ _with_debug }}
just _with_docker ./scripts/upload.bash {{ _cmd }}

# Executes commands in ./web/Justfile. Equivalent to `just web/dist`, but inside of Docker (i.e., just _with_docker web/dist).
web *args:
#!/usr/bin/env bash
Expand Down Expand Up @@ -309,6 +318,8 @@ _with_docker *args:
-e GOCACHE=/work/.cache/go/cache\
-e GOMODCACHE=/work/.cache/go/mod\
-e MODE={{ _mode }}\
--env-file <(env | grep AWS_)\
--env-file <(env | grep GITHUB_)\
--platform {{ _docker_platform }}\
-v "$(pwd)":/work\
-w /work\
Expand Down
18 changes: 18 additions & 0 deletions scripts/get-vscode-extension-path.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "${DEBUG:-false}" = true ];
then
set -x
fi

if [ "$#" -ne 2 ]; then
echo "usage: $0 <cmd> <version>"
exit 1
fi

cmd=$1
version=$2

name=$(basename "$cmd")

printf "$(pwd)/packages/%s" "$name-$version.vsix"
45 changes: 0 additions & 45 deletions scripts/release.bash

This file was deleted.

59 changes: 59 additions & 0 deletions scripts/upload.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "${DEBUG:-false}" = true ];
then
set -x
fi

cmd=$1
if [[ -z "$cmd" ]]; then
echo "usage: $0 <cmd>"
exit 1
fi
echo "Command: $cmd" 1>&2

version=$(./scripts/get-version.bash)
echo "Version: $version" 1>&2

name=$(basename "$cmd")
echo "Name: $name" 1>&2

ref="${GITHUB_REF}"
echo "Git Reference: $ref" 1>&2
echo "Git Commit Hash: $(git show-ref --heads --hash "$ref")"
ref=${ref#"refs/"}

object_path="s3://posit-publisher/$name/releases/$ref"
echo "Object Path: $object_path" 1>&2

platforms=()
while IFS='' read -r line; do platforms+=("$line"); done < <(./scripts/get-platforms.bash)
for platform in "${platforms[@]}"
do
echo
echo "Release: $platform" 1>&2
os=${platform/\/*} # retain the part before the slash
arch=${platform/*\/} # retain the part after the slash
archive=$(./scripts/get-archive-path.bash "$name" "$version" "$os" "$arch" )
echo "Archive: $archive" 1>&2
if ! [ -f "$archive" ];
then
echo "Not Found. Skipping..." 1>&2
else
object="$object_path/$(basename "$archive")"
echo "Object: $object" 1>&2
aws s3 cp "$archive" "$object" > /dev/null 2>&1
fi
done

extension=$(./scripts/get-vscode-extension-path.bash "$name" "$version")
echo
echo "VSCode Extension: $extension"
if ! [ -f "$extension" ];
then
echo "Not Found. Skipping..." 1>&2
else
object="$object_path/$(basename "$extension")"
echo "Object: $object" 1>&2
aws s3 cp "$extension" "$object" > /dev/null 2>&1
fi

0 comments on commit 340856a

Please sign in to comment.