EESSI SKA 2023 09 25

EESSI/SKA sync meeting (2023-09-25)

attendees:

• Tim Kok (SURF, SKA)
• Kenneth Hoste (HPC-UGent, EESSI)
• Lara Peeters (HPC-UGent, EESSI)
• Caspar van Leeuwen (SURF)
• Raymond Oonk (SURF, SKA)

Notes

• Square Kilometer Array (SKA) project
  • distributing software across nodes used to run software
  • ~1PB of software per year in LOFA project
  • SKA will be order of magnitude larger in terms of produced data
  • mostly high throughput computing
• radio astronomy software
  • mix of software from 70s + bleeding edge
  • Fortran, Python, BLAS, ...
  • ~2 month runtime for large dataset (15TB) on 60 cores
  • container images are mostly built manually (by PhD students)
• containers
  • experience with distributing Singularity containers via CVMFS (since 2014)
    • both container image files and unpacked images
  • see for example https://github.com/tikk3r/flocs/tree/fedora-py3/singularity
• collaboration between EESSI & SKA
  • first SKA data should become available in 2026
  • ship container images for SKA via EESSI?
  • reuse build bot effort
  • reuse detection of host CPU
  • own ska repo in EESSI (ska.eessi.io)?
  • letting SKA explore under EESSI umbrella could be interesting in view of potential funding post 2026 for EESSI