diff --git a/QPKGs/sherpa/build/sherpa.qpkg b/QPKGs/sherpa/build/sherpa.qpkg index 85bde29d0..422b9b8a4 100644 Binary files a/QPKGs/sherpa/build/sherpa.qpkg and b/QPKGs/sherpa/build/sherpa.qpkg differ diff --git a/QPKGs/sherpa/build/sherpa_240428.qpkg b/QPKGs/sherpa/build/sherpa_240428.qpkg new file mode 100644 index 000000000..422b9b8a4 Binary files /dev/null and b/QPKGs/sherpa/build/sherpa_240428.qpkg differ diff --git a/QPKGs/sherpa/build/sherpa_240428.qpkg.md5 b/QPKGs/sherpa/build/sherpa_240428.qpkg.md5 new file mode 100644 index 000000000..dc59aa7a1 --- /dev/null +++ b/QPKGs/sherpa/build/sherpa_240428.qpkg.md5 @@ -0,0 +1 @@ +8bac70224e620d0a267b681fbdd69c4f build/sherpa_240428.qpkg diff --git a/QPKGs/sherpa/qpkg.cfg b/QPKGs/sherpa/qpkg.cfg index d79080969..a35e347de 100644 --- a/QPKGs/sherpa/qpkg.cfg +++ b/QPKGs/sherpa/qpkg.cfg @@ -1,5 +1,5 @@ QPKG_NAME="sherpa" -QPKG_VER="240427" +QPKG_VER="240428" QPKG_AUTHOR="OneCD" QPKG_RC_NUM="500" QPKG_SERVICE_PROGRAM="sherpa-service.sh" diff --git a/QPKGs/sherpa/shared/sherpa-loader.sh b/QPKGs/sherpa/shared/sherpa-loader.sh index adf92b0d3..32a7a24ac 100755 --- a/QPKGs/sherpa/shared/sherpa-loader.sh +++ b/QPKGs/sherpa/shared/sherpa-loader.sh @@ -24,7 +24,7 @@ readonly USER_ARGS_RAW=$* Init() { -export LOADER_SCRIPT_VER='240427' +export LOADER_SCRIPT_VER='240428' export LOADER_SCRIPT_PPID=$PPID readonly QPKG_NAME=sherpa readonly CHARS_REGULAR_PROMPT='$ ' diff --git a/docs/QNAP-forum-announcement.bbcode b/docs/QNAP-forum-announcement.bbcode index 9f46ad22b..e62cc85e1 100644 --- a/docs/QNAP-forum-announcement.bbcode +++ b/docs/QNAP-forum-announcement.bbcode @@ -7,7 +7,7 @@ The world's first multi-action CLI package-manager! Package management via [b]sherpa[/b] provides features like easy application backup, upgrading, service and daemon management, multi-threaded operation, self-checking and repair, and all operations may be automated via cron. -[list][b][color=#A371F7]Important:[/color][/b] this is a command-line package and service manager, it's in beta status, and packages have been known to break due to auto-upgrades going wrong. If you would like-to (and are able-to) help by diagnosing and providing logs, and don't mind things breaking from time-to-time, please use this package. If you're looking for complete stability and want a "set-and-forget" solution, it won't be found here just yet. [color=#FF0000][b]Do not[/b][/color] use [b]sherpa[/b] in production environments, unless you're comfortable with the CLI and debugging bash and Python scripts, and/or can afford for applications to be out-of-order for extended periods of time. +[list][b][color=#A371F7]Important:[/color][/b] this is a command-line package and service manager, it's in beta status, and packages have been known to break due to auto-upgrades going wrong. If you would like-to (and are able-to) help by diagnosing and providing logs, and don't mind things breaking from time-to-time, please use this package. If you're looking for complete stability and want a "set-and-forget" solution, it won't be found here just yet. [color=#F85149][b]Do not[/b][/color] use [b]sherpa[/b] in production environments, unless you're comfortable with the CLI and debugging bash and Python scripts, and/or can afford for applications to be out-of-order for extended periods of time. That said: the majority of development is now complete, and I'm currently working-on increasing stability during auto-package upgrades. So, [b]sherpa[/b] will work beautifully on a fresh (or new) system, but can experience issues when individual application updates are released. [/list] diff --git a/docs/README.md b/docs/README.md index 683c3c4ef..1b25c0839 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,11 +1,8 @@ ![icon](images/sherpa.readme.png) - sherpa: a mini-package-manager for QNAP NAS ============================================ - [![Latest Release](https://img.shields.io/github/v/release/OneCDOnly/sherpa?logo=github&label=latest%20release)](https://github.com/OneCDOnly/sherpa/releases/latest) ![Static Badge](https://img.shields.io/badge/release_status-beta-orange?logo=github) ![Project Launch](https://img.shields.io/date/1494050732?logo=github&label=project%20launch) - The world's first multi-action CLI package-manager! Package management via **sherpa** provides features like easy application backup, upgrading, service and daemon management, multi-threaded operation, self-checking and repair, and all operations may be automated via cron. @@ -16,25 +13,21 @@ Package management via **sherpa** provides features like easy application backup > That said: the majority of development is now complete, and I'm currently working-on increasing stability during auto-package upgrades. So, sherpa will work beautifully on a fresh (or new) system, but can experience issues when individual application updates are released. [Click here for installable packages](https://github.com/OneCDOnly/sherpa/wiki/Packages) - - ## Installation - - [SSH](https://www.qnap.com/en/how-to/faq/article/how-do-i-access-my-qnap-nas-using-ssh) into your NAS, and install the QPKG manually at the command-prompt: ``` curl -skL https://tinyurl.com/get-sherpa > /share/Public/sherpa.qpkg; sudo sh /share/Public/sherpa.qpkg ``` - ## Usage - - - At the command-prompt, run: - ``` sudo sherpa ``` - ... and follow the help from there. +If you have suggestions, advice, comments or concerns, please either create a new [issue](https://github.com/OneCDOnly/sherpa/issues/new), or you are most welcome to start a new [discussion](https://github.com/OneCDOnly/sherpa/discussions/new/choose) topic. + +This project is a community effort, and has been built with the combined efforts of many community members on the [QNAP](https://forum.qnap.com/viewtopic.php?f=320&t=132373) community forum. Thank you to everyone who has contributed. 🤓 + Checkout the wiki for more information: [https://github.com/OneCDOnly/sherpa/wiki](https://github.com/OneCDOnly/sherpa/wiki) diff --git a/objects.tar.gz b/objects.tar.gz index c5f46b027..30b7950e7 100644 Binary files a/objects.tar.gz and b/objects.tar.gz differ diff --git a/packages.tar.gz b/packages.tar.gz index 8ea4926cc..24f9d1cc6 100644 Binary files a/packages.tar.gz and b/packages.tar.gz differ diff --git a/sherpa-manager.tar.gz b/sherpa-manager.tar.gz index d117b9421..6196b2dd6 100644 Binary files a/sherpa-manager.tar.gz and b/sherpa-manager.tar.gz differ diff --git a/support/build-wiki-package-abbreviations.sh b/support/build-wiki-package-abbreviations.sh index 40f6a0aa6..3880e4445 100755 --- a/support/build-wiki-package-abbreviations.sh +++ b/support/build-wiki-package-abbreviations.sh @@ -104,9 +104,14 @@ target_pathfile=$wiki_path/Package-abbreviations.md Objects:Load Packages:Load 2>/dev/null # packages source file throws a lot of syntax errors until it's processed - ignore these. -printf 'These abbreviations are recognised by **sherpa** and may be used in-place of each [package name](Packages):\n\n' > "$target_pathfile" -echo '| package name | acceptable abbreviations |' >> "$target_pathfile" -echo '| ---: | :--- |' >> "$target_pathfile" + { + + echo -e '![Static Badge](https://img.shields.io/badge/page_status-live-green?style=for-the-badge)\n' + echo -e 'These abbreviations are recognised by **sherpa** and may be used in-place of each [package name](Packages):\n' + echo '| package name | acceptable abbreviations |' + echo '| ---: | :--- |' + + } > "$target_pathfile" for package_name in $(QPKGs-GRall:Array); do abs=$(QPKG.Abbrvs "$package_name") diff --git a/support/highest_package_versions_found.tbl b/support/highest_package_versions_found.tbl index 4ce6914fc..77226f4ef 100644 --- a/support/highest_package_versions_found.tbl +++ b/support/highest_package_versions_found.tbl @@ -83,8 +83,8 @@ SABnzbd_240426_x86_64.qpkg.md5 SABnzbd_240426_x86_64.qpkg SABnzbd SABnzbd_240426_x86.qpkg.md5 SABnzbd_240426_x86.qpkg SABnzbd 240426 i86 845d18b588e8eac34f917ef137fb4fd5 sha3sum_230312_x86_64.qpkg.md5 sha3sum_230312_x86_64.qpkg sha3sum 230312 i64 6ceba9116e0dbcbdb4e6112f0c02f8c0 sha3sum_230312_x86.qpkg.md5 sha3sum_230312_x86.qpkg sha3sum 230312 i86 fdba9d16b88a5b6e3b04483ca77706dd -sherpa_240427.qpkg.md5 sherpa_240427.qpkg sherpa 240427 all 8f97e0a94dd0d08c9b09b805eca8d915 -SortMyQPKGs_240225.qpkg.md5 SortMyQPKGs_240225.qpkg SortMyQPKGs 240225 all 44643edd1a10db13caa48a46f0a8d3c5 +sherpa_240428.qpkg.md5 sherpa_240428.qpkg sherpa 240428 all 8bac70224e620d0a267b681fbdd69c4f +SortMyQPKGs_240427.qpkg.md5 SortMyQPKGs_240427.qpkg SortMyQPKGs 240427 all 9392283f02079fa586f8d03d96f0be17 Unrar_6.2.5_arm_64.qpkg.md5 Unrar_6.2.5_arm_64.qpkg Unrar 6.2.5 a64 e2382b80908fc9549bbb00058968082a Unrar_6.2.5_arm-x41.qpkg.md5 Unrar_6.2.5_arm-x41.qpkg Unrar 6.2.5 a41 ac1ff1149258db5e4c0b8165774eda37 Unrar_6.2.5_x86_64.qpkg.md5 Unrar_6.2.5_x86_64.qpkg Unrar 6.2.5 i64 f29dc4c271a0a1341d2dad78eb34d2ca diff --git a/support/sherpa-manager.source b/support/sherpa-manager.source index a705c433e..9a2afb710 100755 --- a/support/sherpa-manager.source +++ b/support/sherpa-manager.source @@ -40,6 +40,7 @@ readonly SCRIPT_STARTSECONDS=$(/bin/date +%s) Init() { + SetTraps OsIsOk || return UserIsOk || return LoadConsts @@ -47,7 +48,6 @@ Init() UpdateCapabilities LoadCMDs CMDsIsOk || return - SetTraps HideKeystrokes HideCursor LoadEnv || return @@ -573,10 +573,11 @@ LoadEnv() readonly CACHE_PATH=$THIS_PACKAGE_PATH/cache readonly ACTION_TIMES_PATH=$CACHE_PATH/action.times + readonly BG_PROCS_PATH=$CACHE_PATH/proc readonly DISPLAY_INHIBIT_PATHFILE=$CACHE_PATH/display.inhibit readonly IPK_CACHE_PATH=$CACHE_PATH/IPKs - readonly IPK_DL_PATH=$CACHE_PATH/IPKs.downloads - readonly IPK_DOWNGRADE_DL_PATH=$CACHE_PATH/IPKs.downgrade + readonly IPK_DL_PATH=$IPK_CACHE_PATH/downloads + readonly IPK_DOWNGRADE_PATH=$IPK_CACHE_PATH/downgrade readonly PIP_CACHE_PATH=$CACHE_PATH/PIPs readonly OBJECTS_ARCHIVE_PATHFILE=$CACHE_PATH/objects.tar.gz readonly OBJECTS_PATHFILE=$CACHE_PATH/objects @@ -584,7 +585,8 @@ LoadEnv() readonly PACKAGES_PATHFILE=$CACHE_PATH/packages readonly PREV_IPK_LIST=$CACHE_PATH/ipk.list.save readonly PREV_PIP_LIST=$CACHE_PATH/pip.list.save - readonly QPKG_DL_PATH=$CACHE_PATH/QPKGs.downloads + readonly QPKG_CACHE_PATH=$CACHE_PATH/QPKGs + readonly QPKG_DL_PATH=$QPKG_CACHE_PATH/downloads readonly LOGS_PATH=$THIS_PACKAGE_PATH/logs readonly RAMDISKS_FREESPACE_PATHFILE=$LOGS_PATH/ramdisks.freespace readonly SCREEN_SESSIONS_PATHFILE=$LOGS_PATH/screen.sessions @@ -687,15 +689,17 @@ LoadLists() CreatePaths() { + ClearPath "$CACHE_PATH" "$BG_PROCS_PATH" ClearPath "$CACHE_PATH" "$IPK_CACHE_PATH" ClearPath "$CACHE_PATH" "$IPK_DL_PATH" ClearPath "$CACHE_PATH" "$PIP_CACHE_PATH" MakePath "$ACTION_TIMES_PATH" 'action times' || return + MakePath "$BG_PROCS_PATH" 'background process tracking' || return MakePath "$CACHE_PATH" cache || return MakePath "$IPK_CACHE_PATH" 'IPK cache' || return MakePath "$IPK_DL_PATH" 'IPK download' || return - MakePath "$IPK_DOWNGRADE_DL_PATH" 'IPK downgrade' || return + MakePath "$IPK_DOWNGRADE_PATH" 'IPK downgrade' || return MakePath "$LOGS_PATH" logs || return MakePath "$PIP_CACHE_PATH" 'PIP cache' || return MakePath "$REPORTS_PATH" reports || return @@ -1427,8 +1431,11 @@ Action:Proc() ShowKeystrokes # Enable this before removing Entware & GNU `stty`. fi + pidfile=$($MKTEMP_CMD "$BG_PROCS_PATH"/bgproc_XXXXXX) # Set pidfile here, before launching background process so it's inherited by that process. + _LaunchOneActionWithManyForks_ "_${PACKAGE_TYPE}:${TARGET_ACTION}_" "${target_packages[@]}" & - fork_pid=$! + + echo "$!" > "$pidfile" # Read message pipe and process QPKGs and actions as-per requests contained within. while [[ ${#target_packages[@]} -gt 0 ]]; do @@ -1582,17 +1589,19 @@ CloseActionMsgPipe() # Restore original file descriptors, and remove message pipe. - # Restore stdin FD. - [[ $backup_stdin_fd != none ]] && eval "exec 0>&$backup_stdin_fd" + if [[ -n ${backup_stdin_fd:-} ]]; then + # Restore stdin FD. + [[ $backup_stdin_fd != none ]] && eval "exec 0>&$backup_stdin_fd" - # Release backup of stdin FD. - [[ $backup_stdin_fd != none ]] && eval "exec $backup_stdin_fd>&-" + # Release backup of stdin FD. + [[ $backup_stdin_fd != none ]] && eval "exec $backup_stdin_fd>&-" + fi # Release message pipe FD. - [[ $action_msg_pipe_fd != none ]] && eval "exec $action_msg_pipe_fd>&-" + [[ -n ${action_msg_pipe_fd:-} && $action_msg_pipe_fd != none ]] && eval "exec $action_msg_pipe_fd>&-" # Delete message pipe. - [[ -p $ACTION_MSG_PIPE ]] && rm -f "$ACTION_MSG_PIPE" + [[ -n ${ACTION_MSG_PIPE:-} && -p $ACTION_MSG_PIPE ]] && rm -f "$ACTION_MSG_PIPE" } @@ -3473,7 +3482,7 @@ IPKs:downgrade() ((ok_count++)) remote_url=${url_prefix}${name}${url_suffix} - local_pathfile=$IPK_DOWNGRADE_DL_PATH/$($BASENAME_CMD "$remote_url") + local_pathfile=$IPK_DOWNGRADE_PATH/$($BASENAME_CMD "$remote_url") RunAndLog "$CURL_CMD --location --output $local_pathfile $remote_url" "$log_pathfile" log:failure-only done @@ -3492,7 +3501,7 @@ IPKs:downgrade() ShowAsProc "downgrade $desc" ShowAsPercentProgress "downgrade $total_count ${package_type}$(Pluralise "$total_count")" '' "$ok_count" 0 "$fail_count" "$total_count" - RunAndLog "$OPKG_CMD install --force-downgrade --cache $IPK_CACHE_PATH --tmp-dir $IPK_DOWNGRADE_DL_PATH $IPK_DOWNGRADE_DL_PATH/*.ipk" "$log_pathfile" log:failure-only + RunAndLog "$OPKG_CMD install --force-downgrade --cache $IPK_CACHE_PATH --tmp-dir $IPK_DOWNGRADE_PATH $IPK_DOWNGRADE_PATH/*.ipk" "$log_pathfile" log:failure-only z=$? if [[ $z -eq 0 ]]; then @@ -3643,11 +3652,15 @@ _LaunchOneActionWithManyForks_() # inputs: (global) # $fork_count = number of currently running forks. # $max_forks = maximum number of permitted concurrent forks given the current environment. + # $pidfile = pathfilename of a file containing the PID for this process. + + FuncForkInit local a=${1:-function null} shift # `shift` all arguments one position to the left. local -a c=("$@") + fork_id=0 for qpkg_name in "${c[@]}"; do while [[ $fork_count -ge $max_forks && ! -e $ACTION_ABORT_PATHFILE ]]; do # Don't fork until an empty spot becomes available. @@ -3660,8 +3673,14 @@ _LaunchOneActionWithManyForks_() IncForkProgressIndex MarkThisActionForkAsStarted # Must create runfile here, as it takes too-long to happen in background function. QpkgSetIndex - $a "$qpkg_name" & - DebugAsDone "forked $a() for '$qpkg_name'" + + action_pidfile=$($MKTEMP_CMD "$BG_PROCS_PATH"/bgproc_XXXXXX) # Set pidfile here, before launching background process so it's inherited by that process. + + $a & + + echo "$!" > "$action_pidfile" + + DebugAsDone "forked $a() instance for '$qpkg_name'" UpdateForkProgress done @@ -3674,6 +3693,8 @@ _LaunchOneActionWithManyForks_() # All forks have exited. + FuncForkExit + } _DirSizeMonitor_() @@ -5393,8 +5414,8 @@ Help.Basic.Examples:Show() DisplayAsProjSynIndentExam "to see available $(ShowAsPackages)" 'show packages' DisplayAsProjSynIndentExam '' p DisplayAsProjSynIndentExam "to see available $(ShowAsPackageGroup)s" 'help groups' - DisplayAsProjSynIndentExam "for more $(ShowAsOptions)" 'help options' - DisplayAsHelpTitle "there's also the wiki: $(ShowAsURL 'https://github.com/OneCDOnly/sherpa/wiki')" + DisplayAsProjSynIndentExam "or, for more $(ShowAsOptions)" 'help options' + DisplayAsHelpTitle "more in the wiki: $(ShowAsURL 'https://github.com/OneCDOnly/sherpa/wiki')" Display @@ -7711,7 +7732,7 @@ ModPathToEntware() HardwareGetCPUInfo() { - # QTS 4.5.1 & BusyBox 1.01 don't support `-m` option for `grep`, so extract first mention the hard-way with `head`. + # BusyBox 1.01 doesn't support `-m` option for `grep`, so extract first mention the hard-way with `head`. if $GREP_CMD -q '^model name' /proc/cpuinfo; then $GREP_CMD '^model name' /proc/cpuinfo | $HEAD_CMD -n1 | $SED_CMD 's|^.*: ||' | tr -s ' ' @@ -7857,9 +7878,7 @@ UserGetTimeInShell() local n=0 - if [[ -n ${LOADER_SCRIPT_PPID:-} ]]; then - n=$($PS_CMD -o pid,etime | $GREP_CMD $LOADER_SCRIPT_PPID | $HEAD_CMD -n1) - fi + [[ -n ${LOADER_SCRIPT_PPID:-} ]] && n=$($PS_CMD -o pid,etime | $GREP_CMD $LOADER_SCRIPT_PPID | $HEAD_CMD -n1) FormatLongMinutesSecs "${n:6}" @@ -8063,7 +8082,7 @@ OsIsSupported() OsIsSupportSecureDownload() { - # `curl` with SSL certificate verification enabled: + # `/sbin/curl` with SSL certificate verification enabled: # fails in: # QTS 4.2.6 # QTS 4.4.1 @@ -8100,7 +8119,7 @@ OsIsSupportSudo() OsIsSupportSedExtRegex() { - # Test if QTS `sed` can handle extended regexes (early BusyBox versions cannot). + # Test if QTS `/bin/sed` can handle extended regexes (early BusyBox versions cannot). [[ $(echo -en "\033[1;97ma" | /bin/sed -r 's/\x1b\[[0-9;]*m//g' | /usr/bin/wc -c) -eq 1 ]] } @@ -8108,7 +8127,7 @@ OsIsSupportSedExtRegex() OsIsSupportDecimalSleepSeconds() { - # Test if QTS `sleep` can handle decimal seconds (early BusyBox versions cannot). + # Test if QTS `/bin/sleep` can handle decimal seconds (early BusyBox versions cannot). /bin/sleep .01 &>/dev/null } @@ -8248,7 +8267,7 @@ ClaimLockfile() ReleaseLockfile() { - rm -f "$LOCK_PATHFILE" + [[ -n ${LOCK_PATHFILE:-} ]] && rm -f "$LOCK_PATHFILE" } @@ -8355,17 +8374,17 @@ _QPKG:reassign_() { # * This function runs autonomously * - # Removes the `storeid` assignment for the QPKG named in $1. + # Removes the `storeid` assignment for the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local -i z=0 @@ -8379,7 +8398,7 @@ _QPKG:reassign_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "reassigning $(ShowAsPackageName)" RunAndLog "/sbin/setcfg -e $qpkg_name store -f /etc/config/qpkg.conf" "$LOGS_PATH/$qpkg_name.$REASSIGN_LOG_FILE" log:failure-only @@ -8394,7 +8413,7 @@ _QPKG:reassign_() z=1 # remap to 1 fi - FuncFork:Exit $z + FuncForkExit $z } @@ -8402,17 +8421,17 @@ _QPKG:download_() { # * This function runs autonomously * - # Downloads the QPKG named in $1. + # Downloads the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local -r REMOTE_HASH=$(QpkgGetHash) local -r REMOTE_URL=$(QpkgGetURL) @@ -8437,7 +8456,7 @@ _QPKG:download_() fi fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z if [[ ! -f $LOCAL_PATHFILE ]]; then DebugAsProc "downloading $(ShowAsFileName "$REMOTE_FILENAME")" @@ -8466,7 +8485,7 @@ _QPKG:download_() fi fi - FuncFork:Exit $z + FuncForkExit $z } @@ -8477,14 +8496,14 @@ _QPKG:install_() # Installs the QPKG named in $qpkg_name. # Input: - # $qpkg_name (global) = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -8507,7 +8526,7 @@ _QPKG:install_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z local local_pathfile=$(QpkgGetPathFilename) @@ -8517,7 +8536,7 @@ _QPKG:install_() z=4 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z if [[ $qpkg_name = Entware ]] && ! QPKGs-ISinstalled.Exist Entware && QPKGs-ACinstall-to.Exist Entware; then local -r OPT_PATH=/opt @@ -8580,7 +8599,7 @@ _QPKG:install_() ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -8588,20 +8607,19 @@ _QPKG:reinstall_() { # * This function runs autonomously * - # Reinstalls the QPKG named in $1. + # Reinstalls the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' - qpkg_name=${1:?${FUNCNAME[0]}'()': undefined package name} local -i z=0 if ! QPKGs-ISinstalled.Exist "$qpkg_name"; then @@ -8614,7 +8632,7 @@ _QPKG:reinstall_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z local local_pathfile=$(QpkgGetPathFilename) @@ -8624,7 +8642,7 @@ _QPKG:reinstall_() z=4 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z local target_path='' @@ -8658,7 +8676,7 @@ _QPKG:reinstall_() ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -8666,19 +8684,18 @@ _QPKG:rebuild_() { # * This function runs autonomously * - # Meta-action: rebuilds the QPKG named in $1. This is a `download`, `install` and `restore`, but only if a backup file exists for this QPKG. + # Meta-action: rebuilds the QPKG named in $qpkg_name. This is a `download`, `install` and `restore`, but only if a backup file exists for this QPKG. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit - qpkg_name=${1:?${FUNCNAME[0]}'()': undefined package name} local -i z=0 if ! QpkgIsCanBackup; then @@ -8699,7 +8716,7 @@ _QPKG:rebuild_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "meta-rebuilding $(ShowAsPackageName)" # Nothing-to-do here, real QPKG actions have already been assigned in CheckEnv(). @@ -8708,7 +8725,7 @@ _QPKG:rebuild_() MarkThisActionForkAsOk ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -8716,17 +8733,17 @@ _QPKG:upgrade_() { # * This function runs autonomously * - # Upgrades the QPKG named in $1. + # Upgrades the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -8745,7 +8762,7 @@ _QPKG:upgrade_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z local local_pathfile=$(QpkgGetPathFilename) @@ -8755,7 +8772,7 @@ _QPKG:upgrade_() z=4 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z local prev_ver=$(QpkgGetInstalledVer) local target_path='' @@ -8799,7 +8816,7 @@ _QPKG:upgrade_() ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -8807,17 +8824,17 @@ _QPKG:uninstall_() { # * This function runs autonomously * - # Uninstalls the QPKG named in $1. + # Uninstalls the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -8836,7 +8853,7 @@ _QPKG:uninstall_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z local -r QPKG_UNINSTALLER_PATHFILE=$(QpkgGetInstallationPath)/.uninstall.sh @@ -8877,7 +8894,7 @@ _QPKG:uninstall_() MarkThisActionForkAsFailed fi - FuncFork:Exit $z + FuncForkExit $z } @@ -8885,17 +8902,17 @@ _QPKG:activate_() { # * This function runs autonomously * - # Activates/starts the service script for the QPKG named in $1. + # Activates/starts the service script for the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -8914,7 +8931,7 @@ _QPKG:activate_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z ClearQpkgServiceStatus local -r LOG_PATHFILE=$LOGS_PATH/$qpkg_name.$ACTIVATE_LOG_FILE @@ -8928,7 +8945,7 @@ _QPKG:activate_() SaveActionResultToLog QPKG "$qpkg_name" activate skipped-error 'QPKG service script file undefined' MarkThisActionForkAsSkippedError - FuncFork:Exit 4 + FuncForkExit 4 elif [[ $useropt_debug = true ]]; then a='DEBUG_QPKG=true ' RunAndLog "${a}${service_pathfile} start" "$LOG_PATHFILE" log:failure-only @@ -8962,7 +8979,7 @@ _QPKG:activate_() ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -8970,20 +8987,19 @@ _QPKG:reactivate_() { # * This function runs autonomously * - # Reactivates/restarts the service script for the QPKG named in $1. + # Reactivates/restarts the service script for the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' - qpkg_name=${1:?${FUNCNAME[0]}'()': undefined package name} local -i z=0 if QPKGs-ISNTinstalled.Exist "$qpkg_name"; then @@ -9000,7 +9016,7 @@ _QPKG:reactivate_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z ClearQpkgServiceStatus local -r LOG_PATHFILE=$LOGS_PATH/$qpkg_name.$REACTIVATE_LOG_FILE @@ -9014,7 +9030,7 @@ _QPKG:reactivate_() SaveActionResultToLog QPKG "$qpkg_name" reactivate skipped-error 'QPKG service script file undefined' MarkThisActionForkAsSkippedError - FuncFork:Exit 4 + FuncForkExit 4 elif [[ $useropt_debug = true ]]; then a='DEBUG_QPKG=true ' RunAndLog "${a}${service_pathfile} restart" "$LOG_PATHFILE" log:failure-only @@ -9039,7 +9055,7 @@ _QPKG:reactivate_() ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -9047,17 +9063,17 @@ _QPKG:deactivate_() { # * This function runs autonomously * - # Deactivates/stops the service script for the QPKG named in $1. + # Deactivates/stops the service script for the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -9076,7 +9092,7 @@ _QPKG:deactivate_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z ClearQpkgServiceStatus local -r LOG_PATHFILE=$LOGS_PATH/$qpkg_name.$DEACTIVATE_LOG_FILE @@ -9090,7 +9106,7 @@ _QPKG:deactivate_() SaveActionResultToLog QPKG "$qpkg_name" deactivate skipped-error 'QPKG service script file undefined' MarkThisActionForkAsSkippedError - FuncFork:Exit 4 + FuncForkExit 4 elif [[ $useropt_debug = true ]]; then a='DEBUG_QPKG=true ' RunAndLog "${a}${service_pathfile} stop" "$LOG_PATHFILE" log:failure-only @@ -9123,7 +9139,7 @@ _QPKG:deactivate_() ClearQpkgAppCenterNotifier - FuncFork:Exit $z + FuncForkExit $z } @@ -9131,17 +9147,17 @@ _QPKG:enable_() { # * This function runs autonomously * - # Enables the QPKG named in $1. + # Enables the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local -i z=0 @@ -9159,7 +9175,7 @@ _QPKG:enable_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z ClearQpkgServiceStatus local -r LOG_PATHFILE=$LOGS_PATH/$qpkg_name.$ENABLE_LOG_FILE @@ -9174,7 +9190,7 @@ _QPKG:enable_() SaveActionResultToLog QPKG "$qpkg_name" enable ok MarkThisActionForkAsOk - FuncFork:Exit $z + FuncForkExit $z } @@ -9182,17 +9198,17 @@ _QPKG:disable_() { # * This function runs autonomously * - # Disabled the QPKG named in $1. + # Disables the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local -i z=0 @@ -9214,7 +9230,7 @@ _QPKG:disable_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z ClearQpkgServiceStatus local -r LOG_PATHFILE=$LOGS_PATH/$qpkg_name.$DISABLE_LOG_FILE @@ -9230,7 +9246,7 @@ _QPKG:disable_() SaveActionResultToLog QPKG "$qpkg_name" disable ok MarkThisActionForkAsOk - FuncFork:Exit $z + FuncForkExit $z } @@ -9238,17 +9254,17 @@ _QPKG:enableau_() { # * This function runs autonomously * - # Enables auto-updating of the QPKG named in $1. + # Enables auto-updating of the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -9267,7 +9283,7 @@ _QPKG:enableau_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "enabling auto-update $(ShowAsPackageName)" [[ $useropt_debug = true ]] && a='DEBUG_QPKG=true ' @@ -9285,7 +9301,7 @@ _QPKG:enableau_() z=1 # Remap to 1. fi - FuncFork:Exit $z + FuncForkExit $z } @@ -9293,17 +9309,17 @@ _QPKG:disableau_() { # * This function runs autonomously * - # Disables auto-updating of the QPKG named in $1. + # Disables auto-updating of the QPKG named in $qpkg_name. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -9322,7 +9338,7 @@ _QPKG:disableau_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "disabling auto-update $(ShowAsPackageName)" [[ $useropt_debug = true ]] && a='DEBUG_QPKG=true ' @@ -9340,7 +9356,7 @@ _QPKG:disableau_() z=1 # Remap to 1. fi - FuncFork:Exit $z + FuncForkExit $z } @@ -9348,17 +9364,17 @@ _QPKG:backup_() { # * This function runs autonomously * - # Calls the service script for the QPKG named in $1 and runs a `backup` action. + # Calls the service script for the QPKG named in $qpkg_name and runs a `backup` action # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -9377,7 +9393,7 @@ _QPKG:backup_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "backing-up $(ShowAsPackageName) configuration" [[ $useropt_debug = true ]] && a='DEBUG_QPKG=true ' @@ -9396,7 +9412,7 @@ _QPKG:backup_() z=1 # Remap to 1. fi - FuncFork:Exit $z + FuncForkExit $z } @@ -9404,17 +9420,17 @@ _QPKG:restore_() { # * This function runs autonomously * - # Calls the service script for the QPKG named in $1 and runs a `restore` action. + # Calls the service script for the QPKG named in $qpkg_name and runs a `restore` action # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -9437,7 +9453,7 @@ _QPKG:restore_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "restoring $(ShowAsPackageName) configuration" [[ $useropt_debug = true ]] && a='DEBUG_QPKG=true ' @@ -9456,7 +9472,7 @@ _QPKG:restore_() z=1 # Remap to 1. fi - FuncFork:Exit $z + FuncForkExit $z } @@ -9464,17 +9480,17 @@ _QPKG:clean_() { # * This function runs autonomously * - # Calls the service script for the QPKG named in $1 and runs a `clean` action. + # Calls the service script for the QPKG named in $qpkg_name and runs a `clean` action # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local -i z=0 @@ -9493,7 +9509,7 @@ _QPKG:clean_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "cleaning $(ShowAsPackageName)" [[ $useropt_debug = true ]] && a='DEBUG_QPKG=true ' @@ -9511,7 +9527,7 @@ _QPKG:clean_() z=1 # Remap to 1. fi - FuncFork:Exit $z + FuncForkExit $z } @@ -9526,14 +9542,14 @@ _QPKG:sign_() # Should only be required for QTS 4.3.5-and-later firmwares. # Input: - # $1 = QPKG name + # $qpkg_name (global) # Output: # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local b='' @@ -9545,7 +9561,7 @@ _QPKG:sign_() z=3 fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugVar SQLITE_CMD @@ -9580,7 +9596,7 @@ _QPKG:sign_() fi fi - [[ $z -eq 0 ]] || FuncFork:Exit $z + [[ $z -eq 0 ]] || FuncForkExit $z DebugAsProc "\"signing\" $(ShowAsPackageName)" @@ -9614,7 +9630,7 @@ _QPKG:sign_() z=1 # Remap to 1. fi - FuncFork:Exit $z + FuncForkExit $z } @@ -9625,16 +9641,14 @@ _QPKG:status_() # Query a QPKG for its 'status'. Each compatible QPKG will return 0 if application process is active or ready-to-run, 0 if not. # Input: - # $qpkg_default_index (global, optional) - # $qpkg_index (global, optional) - # $qpkg_name (global, required) + # $qpkg_name (global) # Output: - # $? = none, this function executes in the background. + # $? = none, this function executes in the background. But an exitcode is recorded in the debug log. [[ $useropt_verbose != true ]] && exec &>/dev/null - FuncFork:Init + FuncForkInit local a='' local b='' @@ -9646,7 +9660,7 @@ _QPKG:status_() z=1 fi - [[ $z -eq 0 ]] || FuncFork:Exit + [[ $z -eq 0 ]] || FuncForkExit DebugAsProc "status $(ShowAsPackageName)" @@ -9690,7 +9704,7 @@ _QPKG:status_() SaveActionResultToLog QPKG "$qpkg_name" status ok MarkThisActionForkAsOk - FuncFork:Exit + FuncForkExit } @@ -11677,11 +11691,13 @@ FuncExit() } -FuncFork:Init() +FuncForkInit() { # Debug forked function entry. + trap '[[ -n ${action_pidfile:-} && -e $action_pidfile ]] && rm -f "$action_pidfile"; [[ -n ${pidfile:-} && -e ${pidfile:-} ]] && rm -f "$pidfile"; exit' SIGINT + # Redirect debug output to a temporary log, then add it to session log before exiting. This will keep action log progress in-order. original_sess_active_pathfile=$sess_active_pathfile MakePath "$ACTION_LOGS_PATH" 'action logs' @@ -11696,7 +11712,7 @@ FuncFork:Init() } -FuncFork:Exit() +FuncForkExit() { # Debug forked function exit. @@ -11719,6 +11735,8 @@ FuncFork:Exit() sess_active_pathfile=$original_sess_active_pathfile original_sess_active_pathfile='' + [[ -n ${action_pidfile:-} && -e $action_pidfile ]] && rm -f "$action_pidfile"; [[ -n ${pidfile:-} && -e ${pidfile:-} ]] && rm -f "$pidfile" + exit ${1:-0} } @@ -12456,7 +12474,7 @@ HideKeystrokes() ShowKeystrokes() { - [[ -e $GNU_STTY_CMD && -t 0 ]] && $GNU_STTY_CMD 'echo' + [[ -n ${GNU_STTY_CMD:-} && -e $GNU_STTY_CMD && -t 0 ]] && $GNU_STTY_CMD 'echo' } @@ -12723,11 +12741,20 @@ SetTraps() RunOnSIGINT() { - $TOUCH_CMD "$DISPLAY_INHIBIT_PATHFILE" + local a='' + + [[ -n ${DISPLAY_INHIBIT_PATHFILE:-} ]] && ${TOUCH_CMD:-/bin/touch} "$DISPLAY_INHIBIT_PATHFILE" EraseThisLine ShowAsAbort 'caught SIGINT' KillActiveFork CloseActionMsgPipe + + if [[ -n ${BG_PROCS_PATH:-} && -d ${BG_PROCS_PATH:-} ]]; then + for a in "$BG_PROCS_PATH"/*; do + [[ -n $a && $(${BASENAME_CMD:-/usr/bin/basename} "$a") != '*' && -s $a ]] && kill -9 "$(<$a)" + done + fi + exit } @@ -12736,6 +12763,7 @@ RunOnEXIT() { trap - INT + ShowKeystrokes ShowCursor ReleaseLockfile diff --git a/workshop/ideas.txt b/workshop/ideas.txt index c65d53379..11c78c5c0 100644 --- a/workshop/ideas.txt +++ b/workshop/ideas.txt @@ -1,3 +1,8 @@ +* Add new QPKG 'features' report. + - This would show the separate features available to each QPKG. Backup, clean restart-to-update, etc... + +* Include QPKG version numbers in progress messages. + * Report helper descriptions should only be shown if-required. - Don't show helpers for words not used in the report. diff --git a/workshop/issues.txt b/workshop/issues.txt index 5339d7a06..c9f92232a 100644 --- a/workshop/issues.txt +++ b/workshop/issues.txt @@ -1,12 +1,9 @@ Observed issues: - * Old NZBGet QPKG was 'stop'ped, then a 'backup' was run. + * Old NZBGet QPKG was already 'stop'ped when a 'backup' was run. - This was then followed by a 'start', which is incorrect. - This QPKG is using library functions, so other QPKGs will behave this way too. - * Need to make a cleaner exit during SIGINT when in 'proc: env ...' stage. - - Must set traps earlier. - * armv5 (Helga) is cutting-off debug log early when running 'sherpa s olive verbose' due to SIGINT. * Allow Kapowarr to follow source git branch, release or tag. diff --git a/workshop/QPKGs remaining to add library function support.txt b/workshop/sherpa-enhanced QPKGs without function library support.txt similarity index 100% rename from workshop/QPKGs remaining to add library function support.txt rename to workshop/sherpa-enhanced QPKGs without function library support.txt diff --git a/workshop/test-bg-tracking.sh b/workshop/test-bg-tracking.sh new file mode 100755 index 000000000..b6da266ea --- /dev/null +++ b/workshop/test-bg-tracking.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash + +# get background procs to cleanup their own pid files. + +runme() + { + + # Input: + # $pidfile (global) = the pidfile for this process, containing the PID. + + trap '[[ -e $pidfile ]] && rm "$pidfile"; exit' SIGINT + +# [[ -e $pidfile ]] && echo "pidfile contains: '$(<$pidfile)'" + echo "pidfile: [$pidfile], start sleep" + cmd='sleep 15' + + eval "$cmd" + + echo "pidfile: [$pidfile], end sleep" + + [[ -e $pidfile ]] && rm "$pidfile" + + } + +basepath=~/workspace/proc +mkdir -p "$basepath" +echo "root PID: [$$]" + +for ((i=1; i<=10; i++)); do + pidfile=$(mktemp "$basepath"/bgproc_XXXXXX) # Set $pidfile here, before launching background process so it's inherited by child. + + runme & + echo "$!" > "$pidfile" + sleep 1 +done + +echo 'waiting for background procs to exit...' +wait 2>/dev/null + +echo 'done waiting for background procs, now waiting to exit script...' +sleep 5 + +echo "and exit" +