Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Action] Add actions to replace TAOS-CI #226

Merged
merged 4 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/actions/check-rebuild/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Check if rebuild required
description:

inputs:
mode:
description: build mode to be checked
required: false
default: build

runs:
using: composite
steps:
- run: |
tmpfile=$(mktemp)
git show --pretty="format:" --name-only --diff-filter=AMRC ${{ github.event.pull_request.head.sha}} -${{ github.event.pull_request.commits }} | sort | uniq | awk NF > ${tmpfile}
echo "changed_file_list=${tmpfile}" >> "$GITHUB_ENV"
rebuild=`bash .github/actions/check-rebuild/check_if_rebuild_requires.sh ${tmpfile} ${{ inputs.mode }} | grep "REBUILD=YES" | wc -l`
echo "Rebuild required: ${rebuild}"
echo "rebuild=${rebuild}" >> "$GITHUB_ENV"
shell: sh
71 changes: 71 additions & 0 deletions .github/actions/check-rebuild/check_if_rebuild_requires.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env bash

##
# Copyright (c) 2024 Samsung Electronics Co., Ltd. All Rights Reserved.
#
# @file: check_if_rebuild_requires.sh
# @brief Check if rebuild & unit-test is required with the given PR.
# @see https://github.com/nnstreamer/nnstreamer
# @author MyungJoo Ham <[email protected]>
#
# Argument 1 ($1): the file containing list of files to be checked.
# Argument 2 ($2): build mode to be checked
# gbs: check if Tizen GBS build is required
# debian: check if pdebuild is required
# build (default): check if general meson rebuild is required.

if [ -z $1 ]; then
echo "::error The argument (file path) is not given."
exit 1
fi

if [ -z $2 ]; then
mode="build"
else
mode=$2
fi

rebuild=0
regbs=0
redebian=0

for file in `cat $1`; do
case $file in
*.md|*.png|*.webp|*.css|*.html )
;;
packaging/* )
regbs='1'
;;
debian/* )
redebian='1'
;;
* )
rebuild='1'
regbs='1'
redebian='1'
;;
esac
done

case $mode in
gbs)
if [[ "$regbs" == "1" ]]; then
echo "REBUILD=YES"
exit 0
fi
;;
debian)
if [[ "$redebian" == "1" ]]; then
echo "REBUILD=YES"
exit 0
fi
;;
*)
if [[ "$rebuild" == "1" ]]; then
echo "REBUILD=YES"
exit 0
fi
;;
esac

echo "REBUILD=NO"
38 changes: 38 additions & 0 deletions .github/actions/gitpush/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: 'Push to github.io'
description: 'Update the github action result to nnstreamer.github.io'
inputs:
source:
description: 'source path of the file or directory to be copied'
required: true
dest:
description: 'destination directory in nnstreamer.github.io repository'
required: true
message:
description: 'commit message'
required: false
default: 'Update the result from nnstreamer github action.'
taos_account:
required: true
taos_account_token:
required: true

runs:
using: "composite"
steps:
- name: update the result
run: |
git clone https://${{ inputs.taos_account }}:${{ inputs.taos_account_token }}@github.com/nnstreamer/nnstreamer.github.io.git
pushd nnstreamer.github.io
mkdir -p ${{ inputs.dest }}
cp -r ${{ inputs.source }} ${{ inputs.dest }}
if git diff --shortstat | grep -q changed; then
git config user.email "[email protected]"
git config user.name "nnsuite"
git add *
git commit -s -m "${{ inputs.message }}"
git push origin main -f
else
echo "Nothing to commit. Skip the push operation."
fi
popd
shell: bash
78 changes: 78 additions & 0 deletions .github/workflows/daily-build-gbs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: gbs daily build and test

on:
schedule:
# 05:00 AM (KST) Mon-Fri
- cron: "00 20 * * 0-4"

# Allow manually triggering the workflow
workflow_dispatch:

jobs:
build:
outputs:
x86_64: ${{ steps.gbs-result.outputs.x86_64 }}
armv7l: ${{ steps.gbs-result.outputs.armv7l }}
aarch64: ${{ steps.gbs-result.outputs.aarch64 }}
strategy:
fail-fast: false
matrix:
include:
- gbs_build_arch: "x86_64"
gbs_build_option: "--define \"unit_test 1\" --define \"testcoverage 1\""
- gbs_build_arch: "armv7l"
gbs_build_option: "--define \"unit_test 1\""
- gbs_build_arch: "aarch64"
gbs_build_option: "--define \"unit_test 1\""

runs-on: ubuntu-22.04

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v1

- name: prepare deb sources for GBS
run: echo "deb [trusted=yes] http://download.tizen.org/tools/latest-release/Ubuntu_22.04/ /" | sudo tee /etc/apt/sources.list.d/tizen.list

- name: install GBS
run: sudo apt-get update && sudo apt-get install -y gbs

- name: configure GBS
run: cp .github/workflows/tizen.gbs.conf ~/.gbs.conf

- name: get date
id: get-date
run: |
echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT

- name: run GBS
id: gbs-build
run: gbs build ${{ matrix.gbs_build_option }} --define "_skip_debug_rpm 1" -A ${{ matrix.gbs_build_arch }}

- name: save gbs cache
uses: actions/cache/save@v4
if: ${{ always() && github.ref == 'refs/heads/main' }}
with:
path: ~/GBS-ROOT/local/cache
key: gbs-cache-${{ matrix.gbs_build_arch }}-${{ steps.get-date.outputs.date }}

- if: matrix.gbs_build_arch == 'x86_64' && steps.gbs-build.outcome == 'success'
name: extract test coverage result
run: |
pip install pybadges beautifulsoup4 setuptools
mkdir -p ~/testresult/
pushd ~/testresult/
cp ~/GBS-ROOT/local/repos/tizen/x86_64/RPMS/*-coverage*.rpm .
rpm2cpio *-coverage*.rpm | cpio -idv
popd
python3 .github/workflows/gen_coverage_badge.py ~/testresult/usr/share/nnstreamer-edge/unittest/result/index.html ~/testresult/usr/share/nnstreamer-edge/unittest/result/coverage_badge.svg

- if: matrix.gbs_build_arch == 'x86_64' && steps.gbs-build.outcome == 'success'
name: update test coverage result to github.io
uses: ./.github/actions/gitpush
with:
source: ~/testresult/usr/share/nnstreamer-edge/unittest/result/*
dest: testresult/nnstreamer-edge
message: "${{ steps.get-date.outputs.date }} : Update test coverage result."
taos_account: ${{ secrets.TAOS_ACCOUNT }}
taos_account_token: ${{ secrets.TAOS_ACCOUNT_TOKEN }}
57 changes: 57 additions & 0 deletions .github/workflows/gbs_build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Build and unit test/ Tizen/GBS

on:
pull_request:
branches: [ main ]

jobs:
build:
name: Tizen GBS build on Ubuntu
strategy:
matrix:
include:
- gbs_build_arch: "x86_64"
gbs_build_option: "--define \"unit_test 1\""
- gbs_build_arch: "armv7l"
gbs_build_option: "--define \"unit_test 0\""
- gbs_build_arch: "aarch64"
gbs_build_option: "--define \"unit_test 0\""

runs-on: ubuntu-22.04

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: -${{ github.event.pull_request.commits }}
- name: Check if rebuild required
uses: ./.github/actions/check-rebuild
with:
mode: gbs
- uses: actions/setup-python@v1

- name: prepare GBS
if: env.rebuild == '1'
run: |
echo "deb [trusted=yes] http://download.tizen.org/tools/latest-release/Ubuntu_22.04/ /" | sudo tee /etc/apt/sources.list.d/tizen.list
sudo apt-get update && sudo apt-get install -y gbs
cp .github/workflows/tizen.gbs.conf ~/.gbs.conf

- name: get date
id: get-date
run: |
echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT

- name: restore gbs cache from main branch
if: env.rebuild == '1'
uses: actions/cache/restore@v4
with:
path: ~/GBS-ROOT/local/cache
key: gbs-cache-${{ matrix.gbs_build_arch }}-${{ steps.get-date.outputs.date }}
restore-keys: |
gbs-cache-${{ matrix.gbs_build_arch }}-

- name: run GBS
if: env.rebuild == '1'
run: |
gbs build --skip-srcrpm --define "_skip_debug_rpm 1" -A ${{ matrix.gbs_build_arch }} ${{ matrix.gbs_build_option }}
110 changes: 110 additions & 0 deletions .github/workflows/pdebuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: Build and unit test/ Pdebuild Ubuntu 22.04

on:
pull_request:
branches: [ main ]

schedule:
# 04:00 AM (KST) Mon-Fri
- cron: "00 19 * * 0-4"

# Allow manually triggering the workflow
workflow_dispatch:

jobs:
build:
strategy:
matrix:
os: [ ubuntu-22.04 ]
arch: [ amd64 ]
include:
- distroname: jammy
os: ubuntu-22.04

runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: -${{ github.event.pull_request.commits }}
- name: Check if rebuild required
uses: ./.github/actions/check-rebuild
with:
mode: build
- uses: actions/setup-python@v5

- name: Check trigger event
id: rebuild
run: |
if ${{ env.rebuild == '1' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}; then
echo "rebuild=1" >> $GITHUB_OUTPUT
else
echo "rebuild=0" >> $GITHUB_OUTPUT
fi

- name: get date
id: get-date
run: |
echo "date=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT

- name: make cache dir for pbuilder
## prevent permission error
run: sudo mkdir --mode a=rwx --parents /var/cache/pbuilder

- name: restore pbuilder cache for pull-request
id: restore-pbuilder-cache
if: env.rebuild == '1' && github.event_name == 'pull_request'
uses: actions/cache/restore@v4
with:
path: |
/var/cache/pbuilder/aptcache
/var/cache/pbuilder/base.tgz
key: pbuilder-cache-${{ matrix.os }}-${{ matrix.arch }}-${{ hashFiles('**/debian/control') }}-${{ steps.get-date.outputs.date }}
restore-keys: |
pbuilder-cache-${{ matrix.os }}-${{ matrix.arch }}-${{ hashFiles('**/debian/control') }}-
pbuilder-cache-${{ matrix.os }}-${{ matrix.arch }}-

- name: prepare pdebuild
if: steps.rebuild.outputs.rebuild == '1'
run: |
echo "Installing build tools"
sudo add-apt-repository ppa:nnstreamer/ppa
echo "::group::apt-get update && apt-get install"
sudo apt-get update && sudo apt-get install -y pbuilder debootstrap curl ubuntu-dev-tools qemu-user-static debian-archive-keyring ubuntu-keyring debhelper
echo "::endgroup::"
echo "DISTRIBUTION=${{ matrix.distroname }}" > ~/.pbuilderrc
echo "OTHERMIRROR=\"deb [trusted=yes] http://archive.ubuntu.com/ubuntu ${{ matrix.distroname }}-backports universe |deb [trusted=yes] http://ppa.launchpad.net/nnstreamer/ppa/ubuntu ${{ matrix.distroname }} main\"" >> ~/.pbuilderrc
cat ~/.pbuilderrc
sudo mkdir -p /root/
sudo ln -s ~/.pbuilderrc /root/

- name: make pbuilder base.tgz
if: ${{ steps.rebuild.outputs.rebuild == '1' && steps.restore-pbuilder-cache.outputs.cache-hit != 'true' }}
run: |
echo "=== pbuilder create"
echo "::group::pbuilder create --allow-untrusted"
sudo pbuilder create --allow-untrusted
echo "::endgroup::"
echo "=== pbuilder update"
echo "::group::pbuilder update --distribution ${{ matrix.distroname }}"
sudo pbuilder update --distribution ${{ matrix.distroname }}
echo "::endgroup"
echo "::group::pbuilder update"
sudo pbuilder update
echo "::endgroup"

- name: run pdebuild
if: steps.rebuild.outputs.rebuild == '1'
id: pdebuild
run: |
mkdir -p ~/daily_build/ubuntu
pdebuild --logfile ~/daily_build/pdebuild_log.txt --buildresult ~/daily_build/ubuntu --architecture ${{ matrix.arch }} -- --distribution ${{ matrix.distroname }}

- name: save pbuilder cache
if: always() && ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main' }}
uses: actions/cache/save@v4
with:
path: |
/var/cache/pbuilder/aptcache
/var/cache/pbuilder/base.tgz
key: pbuilder-cache-${{ matrix.os }}-${{ matrix.arch }}-${{ hashFiles('**/debian/control') }}-${{ steps.get-date.outputs.date }}
Loading
Loading