DAOS-14826 test: Add a dfuse/pil4dfs test for bash fd usage. #13681
Conversation
|
Bug-tracker data: |
Add a test for pil4dfs that tries to replicate what configure scripts do with file descriptors. Required-githooks: true Test-tag: FdTest Skip-init-tests: true Signed-off-by: Ashley Pittman <[email protected]>
ashleypittman
force-pushed
the
amd/bash-fd-test
branch
from
c98d317 to
a145af4
Compare
|
Test stage Build RPM on EL 9 completed with status UNSTABLE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13681/2/execution/node/358/log |
|
Test stage Build RPM on EL 8 completed with status UNSTABLE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13681/2/execution/node/361/log |
|
Test stage Build RPM on Leap 15.5 completed with status UNSTABLE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13681/2/execution/node/278/log |
|
Test stage Build DEB on Ubuntu 20.04 completed with status UNSTABLE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-13681/2/execution/node/346/log |
|
Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13681/2/testReport/ |
Required-githooks: true Test-tag: FdTest Skip-unit-tests: true Signed-off-by: Ashley Pittman <[email protected]>
|
Test stage Functional on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13681/3/testReport/ |
Required-githooks: true Signed-off-by: Ashley Pittman <[email protected]>
|
Results are at https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13681/3/testReport/FTEST_dfuse/ Test passed for dfuse and ioil, failed for pil4dfs as expected. The test is weekly so could probably land before #13532 |
|
Test stage Unit Test on EL 8.8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-13681/4/testReport/ |
|
The new test works fine in my local test. |
|
please rebase with latest master and the test-tag |
src/tests/ftest/dfuse/bash_fd.py
Outdated
| def run_bashfd(self, il_lib=None): | ||
| """Run a shell script which opens and writes to files. | ||
|
|
||
| This attempts to replicate the way that configure scripts manipulate fds in bash. | ||
| """ |
There was a problem hiding this comment.
We try to document all arguments
| def run_bashfd(self, il_lib=None): | |
| """Run a shell script which opens and writes to files. | |
| This attempts to replicate the way that configure scripts manipulate fds in bash. | |
| """ | |
| def run_bashfd(self, il_lib=None): | |
| """Run a shell script which opens and writes to files. | |
| This attempts to replicate the way that configure scripts manipulate fds in bash. | |
| Args: | |
| il_lib (str, optional): interception library to run with. Defaults to None | |
| """ |
src/tests/ftest/dfuse/bash_fd.py
Outdated
| self.add_pool(connect=False) | ||
| self.add_container(self.pool) | ||
| mount_dir = f"/tmp/{self.pool.identifier}_daos_dfuse" | ||
| self.start_dfuse(self.hostlist_clients, self.pool, self.container, mount_dir=mount_dir) |
There was a problem hiding this comment.
We're trying to move away from these "magic" pool and container functions and instead use explicit variables. Also, you could let the framework generic a mount_dir for you.
| self.add_pool(connect=False) | |
| self.add_container(self.pool) | |
| mount_dir = f"/tmp/{self.pool.identifier}_daos_dfuse" | |
| self.start_dfuse(self.hostlist_clients, self.pool, self.container, mount_dir=mount_dir) | |
| pool = self.get_pool(connect=False) | |
| container = self.get_container(pool) | |
| self.start_dfuse(self.hostlist_clients, pool, container) |
| with open(os.path.join(fuse_root_dir, "bash_fd_inner.sh"), "w") as fd: | ||
| fd.write(INNER) | ||
|
|
||
| os.chmod(os.path.join(fuse_root_dir, "bash_fd_inner.sh"), stat.S_IXUSR | stat.S_IRUSR) | ||
|
|
||
| with open(os.path.join(fuse_root_dir, "bash_fd_outer.sh"), "w") as fd: | ||
| fd.write(OUTER) | ||
|
|
||
| os.chmod(os.path.join(fuse_root_dir, "bash_fd_outer.sh"), stat.S_IXUSR | stat.S_IRUSR) | ||
|
|
||
| cmd = f"cd {fuse_root_dir}; ./bash_fd_outer.sh" | ||
|
|
||
| result = run_remote(self.log, self.hostlist_clients, env_str + cmd) | ||
| if not result.passed: | ||
| self.fail(f'"{cmd}" failed on {result.failed_hosts}') |
There was a problem hiding this comment.
Just pointing out that this creates local files and then issues a remote command to execute them. This only works because the test runner is also the only client. That's okay for now, but eventually (not sure when) we'd like to not rely on the test runner being a client.
There was a problem hiding this comment.
Is there a "write remote file" option in ftest yet? There's no other way to write fuse tests now, and there never has been.
There was a problem hiding this comment.
No, there isn't. But it's something we'll likely need in the future. For now, I think this approach is okay.
src/tests/ftest/dfuse/bash_fd.py
Outdated
| # stop dfuse | ||
| self.stop_dfuse() | ||
| # destroy container | ||
| self.container.destroy() | ||
| # destroy pool | ||
| self.pool.destroy() |
There was a problem hiding this comment.
This generally isn't necessary. It will all be handled in teardown
src/tests/ftest/dfuse/bash_fd.py
Outdated
| Test Description: | ||
| Test a typical I/O pattern for bash based configure scripts. | ||
|
|
||
| :avocado: tags=all,pr,full_regression |
There was a problem hiding this comment.
pr and weekly? Is that intentional?
There was a problem hiding this comment.
Forgot to update this one, no that's not intentional.
There was a problem hiding this comment.
Re-pushed. This was the only change.
Required-githooks: true Signed-off-by: Ashley Pittman <[email protected]>
Test-tag: DFuseFdTest Required-githooks: true
Test-tag: DFuseFdTest Required-githooks: true Signed-off-by: Ashley Pittman <[email protected]>
ashleypittman
dismissed stale reviews from wiliamhuang and daltonbohning
via
419803c
Add a test for pil4dfs that tries to replicate what configure
scripts do with file descriptors.
Required-githooks: true
Test-tag: FdTest
Skip-init-tests: true
Signed-off-by: Ashley Pittman [email protected]