Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAOS-14909 pool: property for rdb svc_ops KVS entry max age #13639

Merged
merged 13 commits into from
Jan 30, 2024

Conversation

kccain
Copy link
Contributor

@kccain kccain commented Jan 19, 2024

To specify the maximum age of an entry in the svc_ops KVS (that is within rdb, and used for duplicate operation detection), this change introduces a pool property, DAOS_PROP_PO_SVC_OPS_ENTRY_AGE. This replaces the previous engine environment variable interface. The property may be specified during pool create and is then immutable. The default max age is 300 seconds, with a valid range of 150-600.

Features: pool

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate watchers.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

- DAOS_MD_OPS_AGE_SEC environment variable to specify a maximum age of
  svc_ops KVS entries (if default 300 seconds is insufficient).
- test-only pool property "svc_ops_enabled" that can take on the
  boolean values 0 (disabled) or 1 (enabled). So that tests may
  create pools without the duplicate operations detection feature.

Signed-off-by: Kenneth Cain <[email protected]>
Signed-off-by: Kenneth Cain <[email protected]>
Signed-off-by: Kenneth Cain <[email protected]>
since co_op_retry() is taking about a minute longer due to several
raft elections called as part of the test.

Test-tag: test_daos_container
Skip-unit-tests: true
Skip-fault-injection-test: true

Signed-off-by: Kenneth Cain <[email protected]>
Copy link

github-actions bot commented Jan 19, 2024

Bug-tracker data:
Ticket title is 'pool rdb configuration improvement: pool properties for rdb size, duplicate ops detection age/capacity'
Status is 'In Review'
https://daosio.atlassian.net/browse/DAOS-14909

@kccain kccain force-pushed the kccain/daos_14909 branch from 2d7a3b7 to e76ac2f Compare January 19, 2024 19:22
@daosbuild1
Copy link
Collaborator

Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13639/2/testReport/

@kccain kccain force-pushed the kccain/daos_14909 branch from e76ac2f to c5b351b Compare January 20, 2024 02:48
@daosbuild1
Copy link
Collaborator

Test stage NLT on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13639/3/testReport/

@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13639/3/execution/node/1187/log

To specify the maximum age of an entry in the svc_ops KVS (that is
within rdb, and used for duplicate operation detection), this change
introduces a pool property, DAOS_PROP_PO_SVC_OPS_ENTRY_AGE. This
replaces the previous engine environment variable interface. The
property may be specified during pool create and is then immutable.
The default max age is 300 seconds, with a valid range of 150-600.

Features: pool

Signed-off-by: Kenneth Cain <[email protected]>
@kccain kccain force-pushed the kccain/daos_14909 branch from a2ab43c to fb948e4 Compare January 22, 2024 16:52
Base automatically changed from kccain/daos_14021_phase2 to master January 23, 2024 17:00
@kccain kccain marked this pull request as ready for review January 25, 2024 16:07
@kccain kccain requested review from a team as code owners January 25, 2024 16:07
@kccain kccain requested review from mjmac, tanabarr and liw and removed request for a team January 25, 2024 16:07
Copy link
Contributor

@tanabarr tanabarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go changes LGTM

Copy link
Contributor

@mjmac mjmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just a note that it would be good to add some unit test cases for the new property. We haven't been as diligent as we should have been about insisting on these when new properties are added.

@@ -120,6 +120,31 @@ func PoolProperties() PoolPropertyMap {
valueMarshaler: numericMarshaler,
},
},
"svc_ops_entry_age": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be a good idea to add some test cases for this to TestControl_PoolProperties. Looks like the svc_ops_enabled property could use some, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a separate PR with some candidate unit test cases for these two new properties, #13676

@kccain kccain requested a review from a team January 26, 2024 18:41
@mjmac mjmac merged commit f27d15d into master Jan 30, 2024
53 checks passed
@mjmac mjmac deleted the kccain/daos_14909 branch January 30, 2024 16:38
brianjmurrell pushed a commit that referenced this pull request Jan 31, 2024
To specify the maximum age of an entry in the svc_ops KVS (that is
within rdb, and used for duplicate operation detection), this change
introduces a pool property, DAOS_PROP_PO_SVC_OPS_ENTRY_AGE. This
replaces the previous engine environment variable interface. The
property may be specified during pool create and is then immutable.
The default max age is 300 seconds, with a valid range of 150-600.


#Pragmas from previous commit message:
Skip-checkpatch: true
Skip-python-bandit: true
Skip-build: true
Quick-build: true
Quick-Functional: true
Allow-unstable-test: true
#RPM-test-version: version[-release]
#RPM-test-version: 2.5.100
# VM1-label: ci_vm1
# Ubuntu-VM9-label: ci_vm9
# Leap15-VM9-label: ci_vm9
# EL8-VM9-label: ci_vm9
# HW-medium-label: ci_nvme5
# HW-large-label: ci_nvme9
Signed-off-by: Kenneth Cain <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

5 participants