-
Notifications
You must be signed in to change notification settings - Fork 308
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
DAOS-16111 rebuild: enhance leader update_and_warn_for_slow_engines()
With this change, the existing warning logic invoked by the PS leader engine during rebuild_leader_status_check() is enhanced to check for "almost done" and likely hung rebuilds. When 95% or more engines have completed their scan or pull phase, this logic will set a deadline of 2 minutes. If the relevant rebuild phase has not finished by the deadline, warning messages will be logged to indicate a potentially stuck rebuild, including a list of engines that have not completed. The determination of "almost done" is such that, depending on the scale of the system, the number of remaining engines that are being waited for is in a reasonable range of 1-20 engines. The daos_test -r rebuild tests include a new rebuild_long_scan_hang() to inject a single-engine scan hang with a > 2 minute delay, to exercise the new warning logic. Features: rebuild Signed-off-by: Kenneth Cain <kenneth.cain@hpe.com>
- Loading branch information
Showing
5 changed files
with
183 additions
and
45 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters