-
Notifications
You must be signed in to change notification settings - Fork 0
meeting July 2nd 2020
Kenneth Hoste edited this page Jul 2, 2020
·
12 revisions
- date & time: July 2nd 2020 - 3pm CEST (1pm UTC)
- venue: (online)
- agenda:
- Short introduction by new partners (if any)
- ...
- Logo, website, docs, Twitter
- Feedback on HPCKP'20 talk [Kenneth]
- Update on filesystem layer [Bob, Thomas, Jaco]
- Update on compatibility layer [Bob, Peter]
- Update on software layer [Kenneth, Caspar, Alan]
- Pilot setup [Kenneth]
- Applying for resources within the Fenix Infrastructure (deadline 13 July) [Alan]
- Software licenses [Kenneth]
- What to do with our new members (meetings/mailing list)? [All]
- Who does what the coming weeks?
- Short introduction by new partners (if any)
- slides available at https://github.com/EESSI/meetings/blob/master/meetings/EESSI_meeting_20200702.pdf
- Maxim (SURF) briefly introduced himself, planned to help out with software layer (together with Caspar)
- Davide (Microsoft) joining as EasyBuild maintainer
- see https://github.com/EESSI/docs/tree/master/talks/20200618_HPCKP20
- lots of interest, over a dozen people joined the Slack/mailing list via https://www.eessi-hpc.org/join/
- Oriol: recording should be available soon (this month)
- logo, made by graphical designer
- website: https://www.eessi-hpc.org
- docs: https://eessi.github.io/docs
- Twitter account: https://twitter.com/eessi_hpc
- quote requested to finish up logo work (raw materials), align website with logo, get stickers printed
- Jaco (Dell Technologies) may be able to cover those costs
- working pilot setup with Stratum-0, Stratum-1 at Univ. of Groningen
- automated setup with Ansible playbooks
- procedure is documented in README of
filesystem-layer
repo - tested by Thomas at UiO
- tested by Jaco on several platforms, including WSL (Linux env in Windows client)
- now also .deb/.rpm packages available for EESSI CVMFS configuration
- see https://github.com/EESSI/filesystem-layer/pull/24
- only central CVMFS package + this EESSI config package + some local configuring required to get started
- (Singularity) container in the works to allow easy testing
- see https://github.com/EESSI/filesystem-layer/issues/10
- NOT for production usage (w.r.t. CVMFS caching, etc.)
- also look into Docker container (which can be used via Singularity)?
- TODO:
- HTTPS support?
- look into publisher & gateway nodes: https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html
- experimenting with Gentoo Prefix bootstrap installation
- bootstrap script is not 100% reliable
- ran into issues with broken
bzip2
installation, circular dependency, silly typos, ... - not all problems are caught upstream quickly (although they do have some CI setup)
- but response from Prefix developers in case of problems is really quick
- ran into issues with broken
- we may need a better way of controlling the Prefix, especially for a production setup
- Alex: using snapshot of Prefix repository may help
- current approach of getting Prefix installed with bootstrap script and hoping for the best is good enough for pilot setup?
- contact Prefix mailing list to ask about how we could get more control over the installation mechanism if we wanted to?
- bootstrap script is not 100% reliable
- Prefix installation for pilot setup now available at
/cvmfs/pilot.eessi-hpc.org/test/gentoo/2020/startprefix
- documentation to start using it available in README in
compatibility-layer
repo - see also https://github.com/EESSI/compatibility-layer/pull/4
- documentation to start using it available in README in
- planning to leverage ComputeCanada's gentoo-overlay repository for additional Gentoo packages (ebuilds)
- Lua + Lmod is now a manual installation (not part of Prefix)
- brainstorm sessions with Kenneth/Caspar/Alan on June 9th and June 26th on organizing
software-layer
repo- planning to leverage easyconfigs/easyblocks from EasyBuild release as much as possible
- customization specific to EESSI project via EasyBuild hooks
- only include easyconfigs/easyblocks in
software-layer
repo if we have to- try to actively contribute back to central EasyBuild repositories
- working on script that consumes a "build list" which defines the software stack + script to process it
- idea is to leverage EasyBuild as a library
- see https://github.com/EESSI/software-layer/pull/5
- also working on init script to easily set up environment to start using the EESSI software stack
- initialize Prefix environment, set any additional required environment variables, run correct
module use
based on host architecture (CPU/GPU) - see https://github.com/EESSI/software-layer/pull/4
- initialize Prefix environment, set any additional required environment variables, run correct
- see https://github.com/EESSI/software-layer/wiki/Pilot-setup
- limited setup as proof-of-concept
- only Linux clients
- small set of software: GROMACS, TensorFlow, OpenFOAM
- Xavier: also ParaView (GUI, GPU-capable)?
- brings in GUI libraries...
- Xavier: also ParaView (GUI, GPU-capable)?
- limited set of CPU/GPU architectures
- Alan would like to apply for resources in FENIX infrastructure
- see https://fenix-ri.eu/news/prace-icei-calls-proposals-0
- overview of planned resources: https://fenix-ri.eu/infrastructure/resources/planned-resources
- project proposal is short, deadline is July 13th
- Henk-Jan is willing to act as PI for EESSI proposal
- Alan is also planning related proposal w.r.t. scalable training incl. LAMMPS & Magic Castle
- Mark was willing to help review proposal w.r.t. technical aspects
- Which license do we use for the scripts we produce ourselves?
- need to take into account projects we leverage (anything that leverages EasyBuild => GPLv2)
- Some control mechanism to ensure we don't redistribute software we're not allowed to?
- Anyone who submits a join request via https://www.eessi-hpc.org/join/ is added to Slack + mailing list and is welcome to join the meetings
- also outside of Europe
- active contributors can get write permissions to GitHub repositories at some point
- Kenneth: software layer (driver script + init script), but also 3 weeks of vacation
- Caspar/Maxim: software layer
- Xavier: testing in different layers
- has expertise with OpenFOAM, useful for pilot setup
- Mark: test filesystem in OpenStack cloud env
- Thomas: more testing with filesystem and compatibility layer
- national project in Norway w.r.t. setting up shared software stack
- hope to get people involved in EESSI in that context
- Henk-Jan:
- University of Groningen has agreed to fund infrastructure for pilot phase of EESSI w.r.t. hosting Stratum-0