Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added remote sync updates #399

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

added remote sync updates #399

wants to merge 3 commits into from
+105 −37

Conversation

zackbradys
Copy link
Member

Please check below, if the PR fulfills these requirements:

  • Commit(s) and code follow the repositories guidelines.
  • Test(s) have been added or updated to support these change(s).
  • Doc(s) have been added or updated to support these change(s).

Associated Links:

Types of Changes:

  • Feature
  • Breaking Change

Proposed Changes:

  • [Feature] Added support for hauler store sync to accept multiple local and remote archives of http/https
  • [Feature] Improved logging for related commands and outputs of hauler store sync
  • [Breaking Change] Updated the flag from --files/-f to --filename/-f
    • defaulted local file of `hauler-manifest.yaml (new behavior)
    • matches hauler store load/hauler store save

Verification/Testing of Changes:

zackbradys@Zacks-MacBook-Pro test % ls -la
total 40
drwxr-xr-x    5 zackbradys  staff    160 Feb  3 02:41 .
drwxr-xr-x  213 zackbradys  staff   6816 Feb  2 10:17 ..
-rwxr-xr-x    1 zackbradys  staff   1266 Feb  3 02:38 hauler-manifest.yaml
-rw-r--r--    1 zackbradys  staff  14970 Feb  3 02:33 rancher-cluster-templates-0.5.2.tgz
drwxr-xr-x    8 zackbradys  staff    256 Feb  3 00:29 tmp
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store sync         
2025-02-03 02:41:24 INF processing manifest [hauler-manifest.yaml] to store [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:41:24 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images]
2025-02-03 02:41:24 INF adding 'image' [busybox] to the store
2025-02-03 02:41:30 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2025-02-03 02:41:30 INF adding 'image' [busybox:stable] to the store
2025-02-03 02:41:31 INF successfully added 'image' [index.docker.io/library/busybox:stable]
2025-02-03 02:41:31 INF adding 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5] to the store
2025-02-03 02:41:33 INF successfully added 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5]
2025-02-03 02:41:33 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts]
2025-02-03 02:41:33 INF adding 'chart' [rancher] to the store
2025-02-03 02:41:35 INF successfully added 'chart' [hauler/rancher:2.10.2]
2025-02-03 02:41:35 INF adding 'chart' [rancher] to the store
2025-02-03 02:41:36 INF successfully added 'chart' [hauler/rancher:2.8.4]
2025-02-03 02:41:36 INF adding 'chart' [rancher] to the store
2025-02-03 02:41:38 INF successfully added 'chart' [hauler/rancher:2.8.3]
2025-02-03 02:41:38 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.1.1
Digest: sha256:3957fc2bf560f2ccd64e38b59651b097dddb38863124745df264053cae1b807a
2025-02-03 02:41:38 INF successfully added 'chart' [hauler/hauler-helm:1.1.1]
2025-02-03 02:41:38 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.6
Digest: sha256:c4befb2f4939f4b077e3fa3986ab11090a603ac8dc7f8f4ac61dc7a896d20455
2025-02-03 02:41:39 INF successfully added 'chart' [hauler/hauler-helm:1.0.6]
2025-02-03 02:41:39 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.4
Digest: sha256:6a0a7317980309f4551404bd11ee43d1a9d1f1fe97294c7ab051625e435c3d81
2025-02-03 02:41:39 INF successfully added 'chart' [hauler/hauler-helm:1.0.4]
2025-02-03 02:41:39 INF adding 'chart' [rancher-cluster-templates-0.5.2.tgz] to the store
2025-02-03 02:41:39 INF successfully added 'chart' [hauler/rancher-cluster-templates:0.5.2]
2025-02-03 02:41:39 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files]
2025-02-03 02:41:39 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/install.sh:latest]
2025-02-03 02:41:39 INF successfully added 'file' [hauler/install.sh:latest]
2025-02-03 02:41:39 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/rke2-install.sh:latest]
2025-02-03 02:41:39 INF successfully added 'file' [hauler/rke2-install.sh:latest]
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % rm -rf store
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store info
+-----------+------+----------+----------+------+
| REFERENCE | TYPE | PLATFORM | # LAYERS | SIZE |
+-----------+------+----------+----------+------+
+-----------+------+----------+----------+------+
|                                TOTAL   | 0 B  |
+-----------+------+----------+----------+------+
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store sync -l debug
2025-02-03 02:41:55 DBG running cli command [hauler store sync]
2025-02-03 02:41:55 DBG using store at [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:41:55 DBG using temporary directory at [/var/folders/5f/lyy8pc6s62dbckp8sy31ft5r0000gn/T/hauler1037744451]
2025-02-03 02:41:55 INF processing manifest [hauler-manifest.yaml] to store [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:41:55 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images]
2025-02-03 02:41:55 INF adding 'image' [busybox] to the store
2025-02-03 02:41:56 DBG multi-arch image [true]
2025-02-03 02:42:00 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2025-02-03 02:42:00 INF adding 'image' [busybox:stable] to the store
2025-02-03 02:42:00 DBG multi-arch image [true]
2025-02-03 02:42:00 DBG platform for image [linux/amd64]
2025-02-03 02:42:00 INF successfully added 'image' [index.docker.io/library/busybox:stable]
2025-02-03 02:42:00 INF adding 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5] to the store
2025-02-03 02:42:01 DBG multi-arch image [false]
2025-02-03 02:42:02 INF successfully added 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5]
2025-02-03 02:42:02 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts]
2025-02-03 02:42:02 INF adding 'chart' [rancher] to the store
2025-02-03 02:42:04 INF successfully added 'chart' [hauler/rancher:2.10.2]
2025-02-03 02:42:04 INF adding 'chart' [rancher] to the store
2025-02-03 02:42:05 INF successfully added 'chart' [hauler/rancher:2.8.4]
2025-02-03 02:42:05 INF adding 'chart' [rancher] to the store
2025-02-03 02:42:07 INF successfully added 'chart' [hauler/rancher:2.8.3]
2025-02-03 02:42:07 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.1.1
Digest: sha256:3957fc2bf560f2ccd64e38b59651b097dddb38863124745df264053cae1b807a
2025-02-03 02:42:08 INF successfully added 'chart' [hauler/hauler-helm:1.1.1]
2025-02-03 02:42:08 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.6
Digest: sha256:c4befb2f4939f4b077e3fa3986ab11090a603ac8dc7f8f4ac61dc7a896d20455
2025-02-03 02:42:08 INF successfully added 'chart' [hauler/hauler-helm:1.0.6]
2025-02-03 02:42:08 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.4
Digest: sha256:6a0a7317980309f4551404bd11ee43d1a9d1f1fe97294c7ab051625e435c3d81
2025-02-03 02:42:08 INF successfully added 'chart' [hauler/hauler-helm:1.0.4]
2025-02-03 02:42:08 INF adding 'chart' [rancher-cluster-templates-0.5.2.tgz] to the store
2025-02-03 02:42:08 INF successfully added 'chart' [hauler/rancher-cluster-templates:0.5.2]
2025-02-03 02:42:08 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files]
2025-02-03 02:42:08 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/install.sh:latest]
2025-02-03 02:42:09 INF successfully added 'file' [hauler/install.sh:latest]
2025-02-03 02:42:09 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/rke2-install.sh:latest]
2025-02-03 02:42:09 INF successfully added 'file' [hauler/rke2-install.sh:latest]
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % rm -rf store
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store info
+-----------+------+----------+----------+------+
| REFERENCE | TYPE | PLATFORM | # LAYERS | SIZE |
+-----------+------+----------+----------+------+
+-----------+------+----------+----------+------+
|                                TOTAL   | 0 B  |
+-----------+------+----------+----------+------+
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store sync -f https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml         
2025-02-03 02:43:17 INF processing manifest [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml] to store [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:43:17 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images]
2025-02-03 02:43:17 INF adding 'image' [busybox] to the store
2025-02-03 02:43:23 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2025-02-03 02:43:23 INF adding 'image' [busybox:stable] to the store
2025-02-03 02:43:24 INF successfully added 'image' [index.docker.io/library/busybox:stable]
2025-02-03 02:43:24 INF adding 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5] to the store
2025-02-03 02:43:26 INF successfully added 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5]
2025-02-03 02:43:26 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts]
2025-02-03 02:43:26 INF adding 'chart' [rancher] to the store
2025-02-03 02:43:27 INF successfully added 'chart' [hauler/rancher:2.10.2]
2025-02-03 02:43:27 INF adding 'chart' [rancher] to the store
2025-02-03 02:43:29 INF successfully added 'chart' [hauler/rancher:2.8.4]
2025-02-03 02:43:29 INF adding 'chart' [rancher] to the store
2025-02-03 02:43:30 INF successfully added 'chart' [hauler/rancher:2.8.3]
2025-02-03 02:43:30 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.1.1
Digest: sha256:3957fc2bf560f2ccd64e38b59651b097dddb38863124745df264053cae1b807a
2025-02-03 02:43:31 INF successfully added 'chart' [hauler/hauler-helm:1.1.1]
2025-02-03 02:43:31 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.6
Digest: sha256:c4befb2f4939f4b077e3fa3986ab11090a603ac8dc7f8f4ac61dc7a896d20455
2025-02-03 02:43:31 INF successfully added 'chart' [hauler/hauler-helm:1.0.6]
2025-02-03 02:43:31 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.4
Digest: sha256:6a0a7317980309f4551404bd11ee43d1a9d1f1fe97294c7ab051625e435c3d81
2025-02-03 02:43:31 INF successfully added 'chart' [hauler/hauler-helm:1.0.4]
2025-02-03 02:43:31 INF adding 'chart' [rancher-cluster-templates-0.5.2.tgz] to the store
2025-02-03 02:43:31 INF successfully added 'chart' [hauler/rancher-cluster-templates:0.5.2]
2025-02-03 02:43:31 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files]
2025-02-03 02:43:32 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/install.sh:latest]
2025-02-03 02:43:32 INF successfully added 'file' [hauler/install.sh:latest]
2025-02-03 02:43:32 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/rke2-install.sh:latest]
2025-02-03 02:43:32 INF successfully added 'file' [hauler/rke2-install.sh:latest]
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % rm -rf store                                                                                  
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store info
+-----------+------+----------+----------+------+
| REFERENCE | TYPE | PLATFORM | # LAYERS | SIZE |
+-----------+------+----------+----------+------+
+-----------+------+----------+----------+------+
|                                TOTAL   | 0 B  |
+-----------+------+----------+----------+------+
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store sync -l debug -f https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml
2025-02-03 02:44:13 DBG running cli command [hauler store sync]
2025-02-03 02:44:13 DBG using store at [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:44:13 DBG using temporary directory at [/var/folders/5f/lyy8pc6s62dbckp8sy31ft5r0000gn/T/hauler3233246142]
2025-02-03 02:44:13 INF processing manifest [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml] to store [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:44:13 DBG detected remote manifest... starting download... [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml]
2025-02-03 02:44:13 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images]
2025-02-03 02:44:13 INF adding 'image' [busybox] to the store
2025-02-03 02:44:14 DBG multi-arch image [true]
2025-02-03 02:44:18 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2025-02-03 02:44:18 INF adding 'image' [busybox:stable] to the store
2025-02-03 02:44:18 DBG multi-arch image [true]
2025-02-03 02:44:18 DBG platform for image [linux/amd64]
2025-02-03 02:44:19 INF successfully added 'image' [index.docker.io/library/busybox:stable]
2025-02-03 02:44:19 INF adding 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5] to the store
2025-02-03 02:44:19 DBG multi-arch image [false]
2025-02-03 02:44:21 INF successfully added 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5]
2025-02-03 02:44:21 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts]
2025-02-03 02:44:21 INF adding 'chart' [rancher] to the store
2025-02-03 02:44:23 INF successfully added 'chart' [hauler/rancher:2.10.2]
2025-02-03 02:44:23 INF adding 'chart' [rancher] to the store
2025-02-03 02:44:24 INF successfully added 'chart' [hauler/rancher:2.8.4]
2025-02-03 02:44:24 INF adding 'chart' [rancher] to the store
2025-02-03 02:44:26 INF successfully added 'chart' [hauler/rancher:2.8.3]
2025-02-03 02:44:26 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.1.1
Digest: sha256:3957fc2bf560f2ccd64e38b59651b097dddb38863124745df264053cae1b807a
2025-02-03 02:44:27 INF successfully added 'chart' [hauler/hauler-helm:1.1.1]
2025-02-03 02:44:27 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.6
Digest: sha256:c4befb2f4939f4b077e3fa3986ab11090a603ac8dc7f8f4ac61dc7a896d20455
2025-02-03 02:44:27 INF successfully added 'chart' [hauler/hauler-helm:1.0.6]
2025-02-03 02:44:27 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.4
Digest: sha256:6a0a7317980309f4551404bd11ee43d1a9d1f1fe97294c7ab051625e435c3d81
2025-02-03 02:44:27 INF successfully added 'chart' [hauler/hauler-helm:1.0.4]
2025-02-03 02:44:27 INF adding 'chart' [rancher-cluster-templates-0.5.2.tgz] to the store
2025-02-03 02:44:27 INF successfully added 'chart' [hauler/rancher-cluster-templates:0.5.2]
2025-02-03 02:44:27 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files]
2025-02-03 02:44:27 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/install.sh:latest]
2025-02-03 02:44:28 INF successfully added 'file' [hauler/install.sh:latest]
2025-02-03 02:44:28 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/rke2-install.sh:latest]
2025-02-03 02:44:28 INF successfully added 'file' [hauler/rke2-install.sh:latest]
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % rm -rf store                
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store info

+-----------+------+----------+----------+------+
| REFERENCE | TYPE | PLATFORM | # LAYERS | SIZE |
+-----------+------+----------+----------+------+
+-----------+------+----------+----------+------+
|                                TOTAL   | 0 B  |
+-----------+------+----------+----------+------+
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % export HAULER_TEMP_DIR=/Users/zackbradys/Downloads/RGS/test/tmp
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store sync -l debug -f https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml
2025-02-03 02:45:38 DBG running cli command [hauler store sync]
2025-02-03 02:45:38 DBG using store at [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:45:38 DBG using temporary directory at [/Users/zackbradys/Downloads/RGS/test/tmp/hauler61401760]
2025-02-03 02:45:38 INF processing manifest [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml] to store [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:45:38 DBG detected remote manifest... starting download... [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml]
2025-02-03 02:45:39 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images]
2025-02-03 02:45:39 INF adding 'image' [busybox] to the store
2025-02-03 02:45:39 DBG multi-arch image [true]
2025-02-03 02:45:43 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2025-02-03 02:45:43 INF adding 'image' [busybox:stable] to the store
2025-02-03 02:45:44 DBG multi-arch image [true]
2025-02-03 02:45:44 DBG platform for image [linux/amd64]
2025-02-03 02:45:44 INF successfully added 'image' [index.docker.io/library/busybox:stable]
2025-02-03 02:45:44 INF adding 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5] to the store
2025-02-03 02:45:45 DBG multi-arch image [false]
2025-02-03 02:45:46 INF successfully added 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5]
2025-02-03 02:45:46 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts]
2025-02-03 02:45:46 INF adding 'chart' [rancher] to the store
2025-02-03 02:45:48 INF successfully added 'chart' [hauler/rancher:2.10.2]
2025-02-03 02:45:48 INF adding 'chart' [rancher] to the store
2025-02-03 02:45:50 INF successfully added 'chart' [hauler/rancher:2.8.4]
2025-02-03 02:45:50 INF adding 'chart' [rancher] to the store
2025-02-03 02:45:51 INF successfully added 'chart' [hauler/rancher:2.8.3]
2025-02-03 02:45:51 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.1.1
Digest: sha256:3957fc2bf560f2ccd64e38b59651b097dddb38863124745df264053cae1b807a
2025-02-03 02:45:52 INF successfully added 'chart' [hauler/hauler-helm:1.1.1]
2025-02-03 02:45:52 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.6
Digest: sha256:c4befb2f4939f4b077e3fa3986ab11090a603ac8dc7f8f4ac61dc7a896d20455
2025-02-03 02:45:52 INF successfully added 'chart' [hauler/hauler-helm:1.0.6]
2025-02-03 02:45:52 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.4
Digest: sha256:6a0a7317980309f4551404bd11ee43d1a9d1f1fe97294c7ab051625e435c3d81
2025-02-03 02:45:52 INF successfully added 'chart' [hauler/hauler-helm:1.0.4]
2025-02-03 02:45:52 INF adding 'chart' [rancher-cluster-templates-0.5.2.tgz] to the store
2025-02-03 02:45:52 INF successfully added 'chart' [hauler/rancher-cluster-templates:0.5.2]
2025-02-03 02:45:52 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files]
2025-02-03 02:45:53 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/install.sh:latest]
2025-02-03 02:45:53 INF successfully added 'file' [hauler/install.sh:latest]
2025-02-03 02:45:53 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/rke2-install.sh:latest]
2025-02-03 02:45:53 INF successfully added 'file' [hauler/rke2-install.sh:latest]
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % rm -rf store
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % unset HAULER_TEMP_DIR
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % 
zackbradys@Zacks-MacBook-Pro test % hauler store sync -l debug -t /Users/zackbradys/Downloads/RGS/test/tmp -f https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml 
2025-02-03 02:46:34 DBG running cli command [hauler store sync]
2025-02-03 02:46:34 DBG using store at [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:46:34 DBG using temporary directory at [/Users/zackbradys/Downloads/RGS/test/tmp/hauler1717808495]
2025-02-03 02:46:34 INF processing manifest [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml] to store [/Users/zackbradys/Downloads/RGS/test/store]
2025-02-03 02:46:34 DBG detected remote manifest... starting download... [https://carbide.s3.us-gov-east-1.amazonaws.com/tests/hauler-manifest.yaml]
2025-02-03 02:46:35 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images]
2025-02-03 02:46:35 INF adding 'image' [busybox] to the store
2025-02-03 02:46:35 DBG multi-arch image [true]
2025-02-03 02:46:39 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2025-02-03 02:46:39 INF adding 'image' [busybox:stable] to the store
2025-02-03 02:46:39 DBG multi-arch image [true]
2025-02-03 02:46:39 DBG platform for image [linux/amd64]
2025-02-03 02:46:40 INF successfully added 'image' [index.docker.io/library/busybox:stable]
2025-02-03 02:46:40 INF adding 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5] to the store
2025-02-03 02:46:40 DBG multi-arch image [false]
2025-02-03 02:46:42 INF successfully added 'image' [gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5]
2025-02-03 02:46:42 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts]
2025-02-03 02:46:42 INF adding 'chart' [rancher] to the store
2025-02-03 02:46:43 INF successfully added 'chart' [hauler/rancher:2.10.2]
2025-02-03 02:46:43 INF adding 'chart' [rancher] to the store
2025-02-03 02:46:45 INF successfully added 'chart' [hauler/rancher:2.8.4]
2025-02-03 02:46:45 INF adding 'chart' [rancher] to the store
2025-02-03 02:46:46 INF successfully added 'chart' [hauler/rancher:2.8.3]
2025-02-03 02:46:46 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.1.1
Digest: sha256:3957fc2bf560f2ccd64e38b59651b097dddb38863124745df264053cae1b807a
2025-02-03 02:46:47 INF successfully added 'chart' [hauler/hauler-helm:1.1.1]
2025-02-03 02:46:47 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.6
Digest: sha256:c4befb2f4939f4b077e3fa3986ab11090a603ac8dc7f8f4ac61dc7a896d20455
2025-02-03 02:46:47 INF successfully added 'chart' [hauler/hauler-helm:1.0.6]
2025-02-03 02:46:47 INF adding 'chart' [hauler-helm] to the store
Pulled: ghcr.io/hauler-dev/hauler-helm:1.0.4
Digest: sha256:6a0a7317980309f4551404bd11ee43d1a9d1f1fe97294c7ab051625e435c3d81
2025-02-03 02:46:48 INF successfully added 'chart' [hauler/hauler-helm:1.0.4]
2025-02-03 02:46:48 INF adding 'chart' [rancher-cluster-templates-0.5.2.tgz] to the store
2025-02-03 02:46:48 INF successfully added 'chart' [hauler/rancher-cluster-templates:0.5.2]
2025-02-03 02:46:48 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files]
2025-02-03 02:46:48 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/install.sh:latest]
2025-02-03 02:46:48 INF successfully added 'file' [hauler/install.sh:latest]
2025-02-03 02:46:48 INF adding 'file' [https://get.rke2.io/install.sh] to the store as [hauler/rke2-install.sh:latest]
2025-02-03 02:46:48 INF successfully added 'file' [hauler/rke2-install.sh:latest]

Additional Context:

  • N/A

@zackbradys zackbradys requested a review from a team February 3, 2025 07:53
@zackbradys zackbradys self-assigned this Feb 3, 2025
@zackbradys zackbradys added enhancement New feature or request size/M Denotes an issue/PR requiring a relatively moderate amount of work labels Feb 3, 2025
@zackbradys zackbradys added this to the Hauler v1.2.0 milestone Feb 3, 2025
@zackbradys zackbradys linked an issue Feb 3, 2025 that may be closed by this pull request
[feature] Allow hauler to load and sync remote files #208
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@zackbradys zackbradys

Labels
enhancement New feature or request size/M Denotes an issue/PR requiring a relatively moderate amount of work
Projects
Hauler
Status: Testing
Milestone
Hauler v1.2.0
Development

Successfully merging this pull request may close these issues.

[feature] Allow hauler to load and sync remote files
1 participant
@zackbradys