DAOS-7273 pool: Fix dmg pool exclude on ranks

[liw]

The command

dmg pool exclude --rank=R P

only excludes all targets of rank R, but not rank R itself, from pool P. This patch is a quick fix, such that

dmg pool exclude --rank=R P
  # exclude R and all its targets from P

dmg pool exclude --rank=R --target-idx=T0,T1 P
  # exclude T0 and T1, but not R, from P

Before requesting gatekeeper:

• Two review approvals and any prior change requests have been resolved.
• Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
• Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
• Commit messages follows the guidelines outlined here.
• Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

• You are the appropriate gatekeeper to be landing the patch.
• The PR has 2 reviews by people familiar with the code, including appropriate owners.
• Githooks were used. If not, request that user install them and check copyright dates.
• Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
• All builds have passed. Check non-required builds for any new compiler warnings.
• Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
• If applicable, the PR has addressed any potential version compatibility issues.
• Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
• Extra checks if forced landing is requested
  • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
  • No new NLT or valgrind warnings. Check the classic view.
  • Quick-build or Quick-functional is not used.
• Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

[github-actions]

Ticket title is '"dmg pool exclude" unable to exclude a rank'
Status is 'Open'
https://daosio.atlassian.net/browse/DAOS-7273

[daosbuild1]

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-15585/1/testReport/

[daosbuild1]

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-15585/1/execution/node/1519/log

[liw]

I think I may have a clue on (some of) the test failures.

With my PR, during reint, pool_discard is called, but when the collective determines on which targets to discard, pool_map_find_by_rank_status finds that all targets (on rank 7) are UPIN(a status excluded from the collective)!

During the rebuild earlier, I guess that the pool map cached on rank 7 is not updated because rank 7 has been excluded. Therefore, it is still at version 1, where all targets are UPIN.

Without my PR, rank 7 itself is not excluded, and its cached pool map is updated during the rebuild for the exclusion.

I'm not sure whether this is a bug of my PR... Sounds like the reint protocol should have checked the cached pool map version on rank 7.