Skip to content

Commit

Permalink
Add possibility to run .cicd scripts from different environments
Browse files Browse the repository at this point in the history
### With FORCE_LINUX=true, you can run the Docker version in `build.sh` an `package.sh` scripts

When running the scripts from Mac OS X, it was always picking up the Darwin conditional
paths in the script. `FORCE_LINUX=true` can be used on a Mac OS X environment to run
the scripts.

### Fix shell expansion in `package.sh`

In the packaging script, there is a Docker pre command that add executable bit permission
on *.sh scripts in the `build/packages` folder. However, without the escaping, the shell
expansion was performed in the `package.sh` script execution directly instead of being
delayed so that the expansion is performed inside the `docker run` call!

I suspect this worked in your CI system because execution of `package.sh` in performed in
such way that filesystem layout of `package.sh` and filesystem layout of `docker run` were
the same and as such, the expanded `chmod 755 file1.sh file2.sh` command was correctly
working inside the container.

The shell expansion via the `*` is now escaped which correctly delays the expansion until
it's in the `docker run` execution environment.

### Conditional buildkite-agent upload

Now, the upload task is performed only  if BUILDKITE env is set to `true`, so it's possible
to run the script without having to upload the actual package.
  • Loading branch information
Matthieu Vachon committed Apr 17, 2020
1 parent e303192 commit e693d54
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 6 deletions.
4 changes: 3 additions & 1 deletion .cicd/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ set -eo pipefail
. ./.cicd/helpers/general.sh
mkdir -p $BUILD_DIR
CMAKE_EXTRAS="-DCMAKE_BUILD_TYPE='Release' -DENABLE_MULTIVERSION_PROTOCOL_TEST=true -DBUILD_MONGO_DB_PLUGIN=true"
if [[ "$(uname)" == 'Darwin' ]]; then
if [[ "$(uname)" == 'Darwin' && $FORCE_LINUX != true ]]; then
# You can't use chained commands in execute
if [[ "$GITHUB_ACTIONS" == 'true' ]]; then
export PINNED=false
Expand Down Expand Up @@ -39,6 +39,8 @@ else # Linux
elif [[ "$GITHUB_ACTIONS" == 'true' ]]; then
ARGS="$ARGS -e JOBS"
COMMANDS="$BUILD_COMMANDS"
else
COMMANDS="$BUILD_COMMANDS"
fi
. $HELPERS_DIR/file-hash.sh $CICD_DIR/platforms/$PLATFORM_TYPE/$IMAGE_TAG.dockerfile
COMMANDS="$PRE_COMMANDS && $COMMANDS"
Expand Down
2 changes: 1 addition & 1 deletion .cicd/helpers/file-hash.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ set -eo pipefail
FILE_NAME=$(basename $1 | awk '{split($0,a,/\.(d|s)/); print a[1] }')
export DETERMINED_HASH=$(sha1sum $1 | awk '{ print $1 }')
export HASHED_IMAGE_TAG="eos-${FILE_NAME}-${DETERMINED_HASH}"
export FULL_TAG="eosio/ci:$HASHED_IMAGE_TAG"
export FULL_TAG="${IMAGE_NAME:-"eosio/ci"}:$HASHED_IMAGE_TAG"
12 changes: 8 additions & 4 deletions .cicd/package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@
set -eo pipefail
. ./.cicd/helpers/general.sh
mkdir -p $BUILD_DIR
if [[ $(uname) == 'Darwin' ]]; then
if [[ $(uname) == 'Darwin' && $FORCE_LINUX != true ]]; then
bash -c "cd build/packages && chmod 755 ./*.sh && ./generate_package.sh brew"
ARTIFACT='*.rb;*.tar.gz'
cd build/packages
[[ -d x86_64 ]] && cd 'x86_64' # backwards-compatibility with release/1.6.x
buildkite-agent artifact upload "./$ARTIFACT" --agent-access-token $BUILDKITE_AGENT_ACCESS_TOKEN
if [[ "$BUILDKITE" == 'true' ]]; then
buildkite-agent artifact upload "./$ARTIFACT" --agent-access-token $BUILDKITE_AGENT_ACCESS_TOKEN
fi
for A in $(echo $ARTIFACT | tr ';' ' '); do
if [[ $(ls $A | grep -c '') == 0 ]]; then
echo "+++ :no_entry: ERROR: Expected artifact \"$A\" not found!"
Expand All @@ -19,7 +21,7 @@ if [[ $(uname) == 'Darwin' ]]; then
else # Linux
ARGS=${ARGS:-"--rm --init -v $(pwd):$MOUNTED_DIR"}
. $HELPERS_DIR/file-hash.sh $CICD_DIR/platforms/$PLATFORM_TYPE/$IMAGE_TAG.dockerfile
PRE_COMMANDS="cd $MOUNTED_DIR/build/packages && chmod 755 ./*.sh"
PRE_COMMANDS="cd $MOUNTED_DIR/build/packages && chmod 755 \"./*.sh\""
if [[ "$IMAGE_TAG" =~ "ubuntu" ]]; then
ARTIFACT='*.deb'
PACKAGE_TYPE='deb'
Expand All @@ -34,7 +36,9 @@ else # Linux
eval docker run $ARGS $(buildkite-intrinsics) $FULL_TAG bash -c \"$COMMANDS\"
cd build/packages
[[ -d x86_64 ]] && cd 'x86_64' # backwards-compatibility with release/1.6.x
buildkite-agent artifact upload "./$ARTIFACT" --agent-access-token $BUILDKITE_AGENT_ACCESS_TOKEN
if [[ "$BUILDKITE" == 'true' ]]; then
buildkite-agent artifact upload "./$ARTIFACT" --agent-access-token $BUILDKITE_AGENT_ACCESS_TOKEN
fi
for A in $(echo $ARTIFACT | tr ';' ' '); do
if [[ $(ls $A | grep -c '') == 0 ]]; then
echo "+++ :no_entry: ERROR: Expected artifact \"$A\" not found!"
Expand Down

0 comments on commit e693d54

Please sign in to comment.