llm-ray inference & finetune

.github/workflows/workflow_orders_nightly.yml #13

Triggered via schedule December 19, 2023 21:04

carsonwang

⁠ 83cb052

main

Status Failure

Total duration 2m 45s

Artifacts –

workflow_orders_nightly.yml

on: schedule

Matrix: call-finetune-on-intel-gpu / finetune on gpu test

Matrix: call-finetune / finetune test

Matrix: call-inference / inference test

Annotations

21 errors and 12 warnings

call-finetune-on-intel-gpu / finetune on gpu test (pythia-6.9b)

Command failed: rm -rf /home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/ORIG_HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/shallow': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/config': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-commit.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-applypatch.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-push.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/fsmonitor-watchman.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/prepare-commit-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-receive.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-merge-commit.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/push-to-checkout.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/commit-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/applypatch-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/post-update.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/update.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-rebase.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/refs/remotes/pull/5/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/refs/remotes/pull/6/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/heads': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/remotes/pull/5/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/remotes/pull/6/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/tags': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/index': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/info/exclude': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/10/a5975f33936365d4e1c3feaf7c5362c3707818': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/7b/8625e3b2e2575e34aaa89645a1e5123e4276d9': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/50/8ab0ccfc0a8f5198bce9d460bac03f6d6b3500': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/3c/3a09fe9a4cbcbf82f64ebf66c8118ffa03e8c1': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/f3/28dc2fbe11f274aedac02a07054ddbe1d8dfb1': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/7c/393369d4902a0d0547bdd02a0dd7d863049e1b': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/f1/8920c21233dd55fc75facfaa490cbb69b60d15': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/l

call-finetune / finetune test (bigscience/bloom-560m, false)

Command failed: rm -rf /home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/post-update.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/commit-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-rebase.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-applypatch.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/update.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/fsmonitor-watchman.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-receive.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/push-to-checkout.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-merge-commit.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/prepare-commit-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/applypatch-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-push.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-commit.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/description': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/config': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/FETCH_HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/94/5c21d812d6e899eda5f6f7a86adfd1ebcc7ad8': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/40/384887615de9a8165910c7a6557b1afc415d45': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/f3/28dc2fbe11f274aedac02a07054ddbe1d8dfb1': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/info': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/d3/e6cb5d7b838d6f601ca310d9ed056a48d075d1': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/e6/aad62f0ba7965e284c0168844fd8edc0b6645c': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/pack/pack-505a401220d6dc8464f8ba20cfb1ddb156eb9605.idx': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/pack/pack-505a401220d6dc8464f8ba20cfb1ddb156eb9605.pack': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/branches': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/info/exclude': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/refs/remotes/pull/6/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/remotes/pull/6/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/heads': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/tags': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-o

call-finetune / finetune test (EleutherAI/gpt-j-6b, false)

Command failed: rm -rf /home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/tags': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/remotes/pull/6/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/refs/heads': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/branches': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/refs/remotes/pull/6/merge': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/logs/HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-applypatch.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/update.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-commit.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/push-to-checkout.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/fsmonitor-watchman.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/applypatch-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/commit-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-rebase.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-merge-commit.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/prepare-commit-msg.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-push.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/pre-receive.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/hooks/post-update.sample': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/description': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/ORIG_HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/HEAD': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/index': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/e6/aad62f0ba7965e284c0168844fd8edc0b6645c': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/d3/e6cb5d7b838d6f601ca310d9ed056a48d075d1': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/pack/pack-505a401220d6dc8464f8ba20cfb1ddb156eb9605.pack': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/pack/pack-505a401220d6dc8464f8ba20cfb1ddb156eb9605.idx': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/f3/28dc2fbe11f274aedac02a07054ddbe1d8dfb1': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/40/384887615de9a8165910c7a6557b1afc415d45': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/94/5c21d812d6e899eda5f6f7a86adfd1ebcc7ad8': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/objects/info': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.git/shallow': Permission denied rm: cannot remove '/home/ci/actions-runner/_work/llm-on-ray/llm-on-ray/.

call-finetune / finetune test (gpt2, false)