Sync meeting on EESSI test suite (2023 03 30)

EESSI sync meeting on software testing (2023-03-30)

Attending

• Caspar, Sam, Satish, Kenneth

Notes

• PRs
  • #22 just does some code cleanup
    • merged!
  • #23 add support for custom executable opts
    • needs to be synced with main by Sam after merging of #22
    • enables control over additional GROMACS options
    • Satish hopes to find time to take a look
  • #24 add hortense config file
    • Kenneth will test & merge
• Is GROMACS test good enough as example for other tests?
  • There's still room for improvement
    • cfr. https://github.com/EESSI/test-suite/issues/18
    • my_find_modules should be moved to utils
    • come up with multiple general scales
      • 1_core (single-core)
      • sanity_check (1 node)
      • 1_node (1 node, all available cores)
      • 2_nodes (2 nodes)
      • 4_nodes (4 nodes)
      • 8_nodes (8 nodes)
      • => should define a list of known scales somewhere centrally, which defines both tag + node count
      • make a utils function that tags the test with x_nodes tags (+ additional tags like 1_core, sanity_check, ...)
    • set_test_scale is a very general method
      • could be part of an EESSITestCommon base class (known_scales)? or a constant that it's imported and used in base class
      • using reframe.core.pipeline.RegressionMixin? see https://reframe-hpc.readthedocs.io/en/stable/regression_test_api.html#reframe.core.pipeline.RegressionMixin + https://reframe-hpc.readthedocs.io/en/stable/tutorial_advanced.html#grouping-parameter-packs
    • body of filter_tests could be hoisted into a general function that takes self.nb_impl as input
      • filter_tests_by_device_type(device_type, mod_name)
        • checks whether device_type == 'gpu'
        • checks whether mod_name has CUDA support
    • code refactoring options:
      • Class inheritance
      • Use fixtures
      • Move functionality to dedicated functions
• feature request made to ReFrame
  • cfr. https://github.com/EESSI/test-suite/issues/21 + https://github.com/reframe-hpc/reframe/issues/2820
  • this is just "nice to have" to be able to filter more easily
• adding two/three additional tests could help figure out which parts of a test are common, and can be put in a base class
  • (Caspar) TensorFlow - quite similar to GROMACS on CPU/GPU aspect
  • (Satish) OSU - quite different compared to GROMACS + TensorFlow
    • point-to-point
    • collectives: 1-to-many, many-to-1, all-to-all
    • small/large packet size
• effort of trying to generalise (Sam) + add more tests can be done in parallel by different people (Caspar: TensorFlow, Satish: OSU)
• (Kenneth) GitHub Actions workflow to test 'pip install' in old/new Ubuntu
• next steps
  • tests for software used in MultiXscale (ESPResSo, waLBerla, LAMMPS)
    • https://github.com/espressomd/espresso
    • https://i10git.cs.fau.de/walberla/walberla
    • https://docs.easybuild.io/version-specific/supported-software/#lammps
  • integration with the bot (test step in between build/deploy)