Skip to content

Commit

Permalink
fix(build): The number of build jobs should depend on RAM size (#1939)
Browse files Browse the repository at this point in the history
## Problem

- Package build fails in OBS with OOM killer
-
https://build.suse.de/package/live_build_log/Devel:YaST:Agama:Head/agama/SLES-16.0/s390x

## Details

Sometimes the build might fail in OBS because Rust compilation requires
huge amount of RAM. The problem happens when running many parallel jobs
on a machine with not enough RAM. The build on S390 failed when running
8 jobs with 8GB RAM.

Originally I wanted to increase the requirement for RAM in the
`_constraints` file from 8GB to 16GB. But that would decrease the number
of available workers significantly, esp. on exotic archs like s390. And
in most cases the workers run 4 or 8 jobs so requiring 16GB would be an
overkill.

## Solution

So to decrease the needed amount of RAM decrease the maximum number of
parallel jobs.

My experimental run `/usr/bin/time -v cargo build -j1` showed maximum
used memory 1.1GB on x86_64. The compilation on S390 failed with 1GB per
job. To be on the safe side let's require 1.3GB per job, different
architectures might require more and we need to also leave something for
the system and other services.

That means with 8GB RAM it should run at most 6 parallel jobs. That
should hopefully avoid triggering the OOM killer.

## Notes

- We can adjust the RAM per job constant later if needed, this initial
value is rather an experimental value, let's see how it will work.
- If some architecture needs quite different amount of RAM we can make
the setting arch dependent using the `%ifarch` macro.
  • Loading branch information
lslezak authored Jan 23, 2025
1 parent 3c73623 commit d7e5108
Showing 1 changed file with 13 additions and 4 deletions.
17 changes: 13 additions & 4 deletions rust/package/agama.spec
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#
# spec file for package agama
#
# Copyright (c) 2023-2024 SUSE LLC
# Copyright (c) 2023-2025 SUSE LLC
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
Expand All @@ -12,9 +12,15 @@
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.

# Please submit bugfixes or comments via http://bugs.opensuse.org/
# Please submit bugfixes or comments via https://bugs.opensuse.org/
#

# Require at least 1.3GB RAM per each parallel job (the size is in MB)
%global _smp_tasksize_proc 1300

# Redefine the _smp_mflags macro so it takes the amount of RAM into account
%define _smp_mflags "-j%{getncpus:proc}"

Name: agama
# This will be set by osc services, that will run after this.
Version: 0
Expand All @@ -23,7 +29,7 @@ Summary: Agama Installer
# If you know the license, put it's SPDX string here.
# Alternately, you can use cargo lock2rpmprovides to help generate this.
License: GPL-2.0-or-later
Url: https://github.com/opensuse/agama
Url: https://github.com/agama-project/agama
Source0: agama.tar
Source1: vendor.tar.zst

Expand Down Expand Up @@ -70,7 +76,7 @@ Version: 0
Release: 0
Summary: Agama command-line interface
License: GPL-2.0-only
Url: https://github.com/opensuse/agama
Url: https://github.com/agama-project/agama

%description -n agama-cli
Command line program to interact with the Agama installer.
Expand Down Expand Up @@ -129,6 +135,9 @@ package contains a systemd service to run scripts when booting the installed sys
# find vendor -type f -name \*.rs -exec chmod -x '{}' \;

%build
# just for debugging failures with low memory
cat /proc/meminfo | head -n 3

%{cargo_build}
cargo run --package xtask -- manpages
gzip out/man/*
Expand Down

0 comments on commit d7e5108

Please sign in to comment.