Skip to content

Latest commit

 

History

History
375 lines (365 loc) · 12.5 KB

nixtalk.org

File metadata and controls

375 lines (365 loc) · 12.5 KB

Nix Introduction

Sales Pitch

  • Why use nix to install packages?
  • Reproducible builds
  • Full dependency integrity
  • Large integrated language ecosystem
  • Parallel versions of the same package
  • Global caching
  • Encompasses: builds, deployments, system management

nix is a package manager

  • (… and a language, and an OS)
  • nix is a package manager
    • nix-env high level: calls the other tools.
    • nix-shell, nix-store, nix-build
  • nix language
    • declarative
    • dynamic typing
      • resolved at runtime
      • no implicit coersion
    • A DSL to generate IO PKG
  • Invoke builders
    • derivation: recorded pure result of evaluation
    • output: running an engine to realize the derivation

Roadmap

  1. Define a pure language
  2. Evaluate language to determine goals: derivations
  3. Storing builds
  4. Realize derivations (IO)
  5. Compose results into environments

nix language

https://nixos.org/nix/manual

  • sets of attr/val 
    > a = { x = 1; y = "foo"; z = https://lots.of/pipes; };
  • standard set operation: // 
    > b = { x = 3; r = [2 "hi"]; blob = { t=1; }; };
> a // b
{ r = [...]; x = 3; y = "foo"; z = ... }
  • functions 
    > f = i: j: i + j.x
> f 9 a
10
  • lazy evaluation
  • let and rec bindings

nix language files

  • convention: file stores a function 
    $ cat myfun.nix
{ i, j }:
i + j.x
  • conventionally one argument: a set of values 
    > g = import myfun.nix
> g { i=9; j=a; }
10
> c = {i=9;}
> g (c // { j=a; })
10
  • “inherit x y;” is syntactic sugar for “x=x; y=y;” 
    > j = a
> g { inherit j; i=9; } == g { i=9; j=j; }
true

Roadmap

  1. Define a pure language
  2. Evaluate language to determine goals: derivations
  3. Storing builds
  4. Realize Derivations (IO)
  5. Compose results into environments

Create Derivations

  • What is a Derivation?
    • instructions on how to build something + references to every dependency
    • Implemented as sets, but treated specially during evaluation.
      • Perform side effects like producing build output
    • derivation is the most important primitive in nix
      • Requires a name
      • Requires a builder (script for building the package)
      • All attributes are passed as environment variables to the build.
      • Requires a src
        • can use local source, or perform IO and fetch over network

Writing Derivations

  • In practice you’ll use wrappers
    • e.g., runCommand, writeScriptBin, mkDerivation
    • mkDerivation uses setup.sh by default
    • fetchUrl, fetchFromGitHub, fetchFromGitLab, etc.
      • fixed output derivations.
      • Perform network IO, but “pure” because they perform “hash matching”.

Sample derivation

{ pkgs ? import <nixpkgs> {} }:

  # pkgs parameter has a default; it lazily imports
  # all of nixpkgs namespace into scope

  pkgs.stdenv.mkDerivation {

    name = "hello-2.9";

    # src itself is a derivation
    # All derivations are stored in the /nix/store, before being realized
    src = pkgs.fetchurl {
      url = "mirror://gnu/hello/${name}.tar.gz";
      sha256 = "19qy37gkasc4csb1d3bdiz9snn8mir2p3aj0jgzmfv0r2hi7mfzc";
    };

  };
}

Roadmap

  1. Define a pure language
  2. Evaluate language to determine goals: derivations
  3. Storing builds
  4. Realize Derivations (IO)
  5. Compose results into environments

nix store

  • Building a derivation puts build output into the “store”
    • System global directory tree
  • Nothing is in scope
    • dependencies passed in to derivation as args
    • args reference store locations

Unique hashes

  • each store entry identified by hash 
    $ ls -CF /nix/store/q0crs4bg5vgl9cjpp9yxysd1w97inr0-git-2.13.2/
bin/ etc/ lib/ libexec/ share/
  • every package and every version has unique hash
    • closure over build-time inputs
  • can have multiple versions
    • each store location is isolated
    • all dependencies are explicit in the derivation

Roadmap

  1. Define a pure language
  2. Evaluate language to determine goals: derivations
  3. Storing builds
  4. Realize Derivations (IO)
  5. Compose results into environments

Realize derivations

  • Phase distinction, instantiation & realization
  • nix (https://github.com/nixos/nix)
    • Heart of nix system, 4 C++ libraries, dozen or so utilities
  • nix-instantiate
    • Creates build instructions, calculates hash
    • Will exploit network binary caches of store path
  • nix-store -r
    • Build (“realize”) package from derivation description
    • Can watch evaluation through setup.sh by tracing “set -x”
  • nix-build
    • Creates derivation, builds package, installs in store
    • Equivalent to calling ‘nix-store -r $(nix-instantiate default.nix)’

Build phases

  • Builds go through phases
    • pre/postFetch, pre/postBuild, pre/postInstall. etc/
    • Package builds are split into phases to make it easier to override specific parts of the build (e.g., unpacking the sources or installing the binaries).
    • New phases can be defined (haskell packages does this, e.g. haddockPhase)
    • Default phase behavior can be seen in setup.sh
  • nix-shell 
    $ nix-shell $(nix-instantiate default.nix)
    • Useful for interactively building derivations
    • Puts a user into a shell with environment variables from the derivation present
    • Can run the build with `genericBuild`

Roadmap

  1. Define a pure language
  2. Evaluate language to determine goals: derivations
  3. Storing builds
  4. Realize derivations (IO)
  5. Compose results into environments

profiles expose packages

  • Each exe or lib has a unique store location
  • Want a useable environment to run multiple tools
  • nix-env creates profiles nix-user-environments.png

Minimizing for Effectiveness

  • Conventional package managers install globally
  • This is an anti-pattern using nix
  • nix installs into global store, and then exposes locally
    • use only for generics, like vi and emacs
  • let projects specify the packages and versions needed
    • nix-build
      • local ”result” symlink instead of profile
    • nix-shell
      • shell with build environment, but doesn’t build current package

nixpkgs

  • A large set of ready-made derivations
  • Each function is (ultimately) the mkDerivation
  • channels: tarball of packages
    • Updates of packages 
      $ nix-channel --update
    • Stored in the store
    • Can rollback
    • Multiple channels
  • hydra
    • public package builder daemon
    • supplies binaries and bdiffs to nix-env

Huge collection of files

NixOS

  • Built on nix language
  • Adds “system” profile, containing kernel, boot image
  • Builder to update grub to use new boot image
  • Builder to write /etc configuration files from nix
  • Uses same profile symlinks
    • All kernels and boots available in the store
    • Rollback (or forward) is trivial (grub menu entries)
  • Can Should use nixpkgs outside of NixOS

Language ecosystems

  • Each language has its own “package” concept
  • Nix has tools/language elements to support different languages
  • Check online documentation for your language

cabal2nix

  • Tool to builds a nix specification from a cabal file
  • hydra builds
    • cabal2nix run on LTS yaml file
    • Generates new packages specification
    • pkgs.haskellPackages.PKG (11,257 pkgs)
    • hydra rebuilds from new package descriptions
    • For new packages or newer versions
      • use cabal2nix to generate a local file
  • Input sources:
$ cabal2nix /home/kquick/crucible/crucible-matlab > crucible-matlab.nix
$ cabal2nix cabal://language-sally > language-sally.nix
$ cabal2nix cabal://containers-4.3.1 > containers-4.3.1.nix
$ cabal2nix https://github.com/GaloisInc/crucible [-rev 98ac3d5] > crucible.nix

INSTALLING a local haskell project

  • high-level config specifying new/missing components 
    $ nix-env -f myfile -iA haskellPackages.project_a
  • copies source to store
    • assigns hash
    • builds from that source
    • maintains closure chain/reproducability

DEVELOPING a local haskell project

  • nix-shell
  • high-level config specifying new/missing components
  • instantiates the build environment (i.e. builds all dependencies)
  • does not build current target
  • normal “cabal build” process
  • dependencies just need a .nix specification
    • + don’t need to be checked out to build locally
    • + built once
    • - full rebuild if changed, no partial rebuilds

crucible

Sample specification

{ stdenv, fetchurl, pkgconfig, libgphoto2, libexif, popt, gettext
, libjpeg, readline, libtool
}:

stdenv.mkDerivation rec {
  name = "gphoto2-2.5.11";

  src = fetchurl {
    url = "mirror://sourceforge/gphoto/${name}.tar.bz2";
    sha256 = "1sgr6rsvzzagcwhc8fxbnvz3k02wr2hab0vrbvcb04k5l3b48a1r";
  };

  nativeBuildInputs = [ pkgconfig gettext libtool ];
  buildInputs = [ libgphoto2 libexif popt libjpeg readline ];

  meta = with stdenv.lib; {
    description = "A ready to use set of digital camera software applications";
    homepage = http://www.gphoto.org/;
    license = licenses.gpl2Plus;
    platforms = platforms.unix;
    maintainers = [ maintainers.jcumming ];
  };
}

REPL

  • REPL tool for language and inspection 
    $ nix-repl
> :l <nixpkgs>
> pkgs.git.m<TAB>
pkgs.git.makeFlags pkgs.git.meta
> pkgs.git.meta.description
"Distributed version control system"

Miscellaneous

  • Local builds are not incremental
  • Attributes are specific but not versioned
    • When needed, a new version-specific attribute is created
  • Store can get big (that’s called efficiency) 
    $ nix-env --delete-generations 19 20 17
$ nix-env --delete-generations 30d
$ nix-collect-garbage
    • The GC roots are the profile versions.
    • Store locations unreachable from profile roots are removed
  • Prefer nix-shell to nix-env
    • Keep your main environment minimal

Examples

THE END