Add recursive dataset mounting support to pam_zfs_key

[jkolo]

PR Title

Add recursive dataset mounting support to pam_zfs_key

Motivation and Context

This change introduces the ability to recursively mount ZFS datasets using a new configuration option, mount_recursively. This enhancement solves the need for hierarchical or nested dataset management in environments that utilize pam_zfs_key, ensuring all eligible child datasets are mounted as part of the session functionality.
This feature improves the flexibility and usability of pam_zfs_key when managing complex dataset organizations.

Description

• Added support for a new configuration option, mount_recursively, which ensures child datasets of a given root can also be mounted.
• Refactored the decrypt_mount() function to integrate recursive mounting functionality, leveraging zfs_iter_filesystems_v2 for child dataset traversal.
• Updated the dataset mounting logic to ensure properties such as canmount and mountpoint are respected during recursive mounting.
• Modified configuration handling (via zfs_key_config_t) to include the mount_recursively option.
• Enhanced the session logic to include recursive mounting without breaking existing behavior.
• Added a functional test, pam_mount_recursively.ksh, to validate recursive mounting behavior and edge cases (e.g., "none", "legacy", "noauto", and "off" mountpoints).

How Has This Been Tested?

• Added a functional test (tests/zfs-tests/tests/functional/pam/pam_mount_recursively.ksh) that:

  • Validates recursive mounting for nested datasets with the mount_recursively flag enabled.
  • Tests behavior across various child dataset configurations (e.g., different canmount and mountpoint properties).
  • Ensures correct key loading status through session management (open_session and close_session) via PAM.
  • Verifies datasets are only mounted when they satisfy canmount=on and appropriate mountpoint properties.

• Manual testing performed on Linux configurations using pamtester to ensure real-world applicability.

• Verified backward compatibility where mount_recursively is not enabled.

Types of Changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly. (Implementation details documented in code comments. Additional documentation updates suggested separately if needed.)
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

Let me know if additional details are required.

[jkolo]

Pull request is ready for merge. Is there anything else I should do to get it to pass review? @behlendorf sorry for calling directly. I didn't know who or where best to ask :)

[behlendorf]

This makes sense. Thanks for working on this and adding the test case.