Enable NixOS cross-compiling from aarch64-linux to x86_64-linux and vice versa

[prinzdezibel]

Disk image cross compilation fails on ARM64

It's not possible to create cross-platform disk images with nixos/lib/make-disk-image, because runInLinuxVM does only support architectures with same CPU architecture.

Using binfmt emulation alone is not sufficient and will segfault when trying to build an image for x86_64 on aarch64 build machines. Segfaults also happened when I tried to build a barebone configuration with nixos-generators.

To sucessfully build cross-platform image modifications and several cross-compile fixes for the nixpkgs repository are necessary. The proposed changes allow make-disk-image to make usage of a fully fledged QEMU qemu-system-x86_64 instance with TCG fallback instead of the qemu-kvm package which only supports emulation of
machines with the same CPU architecture.

Reproduction of the error

• Clone https://github.com/prinzdezibel/nixos-qemu-image

• Enable official nixpkgs repository as flake input and disable this line

• Enable next line

• Necessary steps (find more detail more here)

  1. Enable binfmt emulation for cross compiling in your system's configuration.nix. Example for a x86_64-linux build machine:

  {
    # Enable binfmt emulation of aarch64-linux.
    boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
  }
  

  2. Specify your machine's system in flake.nix. For example:

    buildSystem = "x64_64-linux"; # Change this if you're building on ARM64 arch
  

  3. Try to cross-compile image to different CPU architecture:

    nix build .#nixosConfigurations.x86_64-linux.config.system.build.qcow
    # or
    nix build .#nixosConfigurations.aarch_64-linux.config.system.build.qcow
  

  4. Result: Lots of errors. Some are related to cross-compiling cloud-init, others because runInLinuxVM does not actually switch standard build environment in stage 2 to targeted host environment. This causes segfaults and additionally there are "activation failed" errors that prevent the VM from sucessfully installing the bootloader.

Fix error: Switch to working nixpkgs repository

• Switch flake input in flake.nix

# Cross-compile image from ARM to Intel:
nix build .#nixosConfigurations.x86_64-linux.config.system.build.qcow
 
# Cross-compile image from Intel to ARM64:
nix build .#nixosConfigurations.aarch_64-linux.config.system.build.qcow

Nix-info

nix-shell -p nix-info --run "nix-info -m"

• system: "aarch64-linux"
• host os: Linux 6.11.6-asahi, NixOS, 24.11 (Vicuna), 24.11.20241123.0c58267
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.24.10
• channels(root): ""
• nixpkgs: /etc/nixpkgs/channels/nixpkgs

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[prinzdezibel]

The PR was created with the wrong head branch selected (it's prinzdezibel:master instead of prinzdezibel:staging or prinzdezibel:feature-branch respectively). I'm going to close this one and create a new PR.