How to cross compile? #125
Comments
|
Can you try something like this: channels.cross.config = {
pkgs = nixpkgs.legacyPackages.x86_64-linux.pkgsCross.aarch64-multiplatform; // what does this do? botstrap nixpkgs?
localSystem = { config = "x86_64-unknown-linux-gnu"; };
crossSystem = { config = "aarch64-unknown-linux-gnu"; };
};
``` |
This defines localSystem, crossSystem and pkgs on
I think all the |
|
@gytis-ivaskevicius I've done some experiments, and commenting out https://github.com/gytis-ivaskevicius/flake-utils-plus/blob/master/lib/mkFlake.nix#L147 led to the system build picking up the relevant options (e.g. setting nixpkgs.pkgs to Also, updating https://github.com/gytis-ivaskevicius/flake-utils-plus/blob/master/lib/mkFlake.nix#L212 to not blanketly inheriting system, but passing either crossSystem and localSystem or system also led to cross compilation correctly (nixpkgs assertion prevents passing system and localSystem at the same time). All that said, I got some compilation issues, and cross compiling led to a full rebuild of all packages without getting any susbstitutions which defeats the purpose. I abandoned the project and I'm looking into ways to selectively cross-compile problem packages, and use qemu aarch64 emulation for the others (and benefit from substituions from nixos binary cache if I just want to use packages as is). Maybe some issues in fup still remain: e.g. I can't just use pkgsCross. because system==localSystem and consequently e.g. |
|
I think I might have figured targeted cross compilation out: {
…
pkgsCrossAargh64 = import nixpkgs {
localSystem = "x86_64-linux";
crossSystem = "aarch64-linux";
overlays = sharedOverlays;
};
channels.cross.input = nixpkgs;
channels.cross.overlaysBuilder = _: [
(final: prev: {
inherit (self.pkgsCrossAargh64) grafana …;
})
];
hosts.archimedes.system = "aarch64-linux";
hosts.archimedes.channelName = "cross";
…
} |
|
thanks @pschyska for all the hints! I seem to have cross compilation working with the modifications your described here #125 (comment) .
I think it is still useful and fup should not prevent building systems like that. I'll see if I can pick up your work and polish it for a PR.
that looks great! If I understand correctly nix will evaluate this and query the caches even on a non-aarach64 host without binfmt emulation, correct? It will probably fail later when assembling the system images unless everything (incl the system derivation, the systemd units, etc dir) depending on those packages is also explicitely listed for cross-compilation. But the approach of taking almost everything from the native caches and just cross-building the few needed packages and the resulting system looks very interesting! |
|
Just to document this here:
I don't have the capacity to do anything about this currently. I'm also not quite sure what the "right" way of implementing this would be. nixpkgs is switching away from specifying Footnotes |
Sorry for the late reply; I'm actually not 100% clear what happens. I'm still using binfmt emulation, and I see some derivations that I pull in via this cross channel doing the main compilation with a cross compiler (e.g. I'm mainly using this to save on build times. At the time, grafana didn't build for aarch64 in nixos-unstable, and it took over an hour to do it via qemu. I also have some custom packages or overlays with patches of nixpkgs packages that were very slow to build (neovim master, the bupstash backup tool, systemd-udevd, from the top of my head). In general, the derivations are substitutable and are indeed substituted. Bear in mind that the cross drvs are different, e.g.: I just built grafana with the config above, and it did build Not sure why it built |
I just tried
Built → Still not clear to my why fish itself wasn't substituted.
Substituted a lot (~3 GB 😲 ) but tried to rebuild around 160 packages, and ultimately failed with: I suspect the binary cache for pkgsCross is populated on a best-effort basis, and there will be many broken things in there because cross compilation is hard. I'll probably keep building franken-pkgs for my raspberry and hope I don't hit these issues. Fingers crossed 😆 |
I'm trying to get cross-compilation working but without any luck so far. I tried every incantation of buildPlatform, hostPlatform, localSystem, crossSystem in either channels config or host's config, but I just can't get it to work.
In particular:
This should cross-compile according to the nixpkgs documentation, but it doesn't.
Also this (unsure what the difference to above is supposed to be):
Additionally, setting pkgs seems to have zero effect when using fup:
I also tried it in channels config:
Trying to read the code it looks like on is merely able to influence
nixpkgs.configfrom the flake level, however all these options go into options directly undernipkgs.Do you know of anyone successfully using fup for aarch64?
My aarch64 hosts either wrongly build as x86_64, or use qemu to build. The latter is extremely slow to a point it's unusable, and stopped working alltogether recently trying to build "grafana" (just stops doing anything without load).
Any pointers?
The text was updated successfully, but these errors were encountered: