Mon 11 Apr 2016 — Fri 27 Apr 2018

Nix

Nix will solve all our problems, maybe.

There's definitely still some thinking to do.

The clever trick

A derivation produces a hash.

It also puts some outputs in a folder under that hash in the Nix Store.

In order to get the outputs so that you can use them, you need to either know the hash, or be able to re-run the derivation to get it.

The other part of the clever trick

The derivation depends only on the things it declares. So if you want to look at files in the directory where the derivation is, you've got to refer to their path in the derivation.

Inside the derivation's builder, where we create the output, we get access to things from the derivation as environment variables. We can't do direct file access, because each file will have been put in a special place based on our derivation's hash.

Nix Language

Objects are surrounded in braces. This is called an attribute set. These are like a dictionary literal, except that it can refer to itself. Presumably no cyclic dependencies though?

Function Definition

: means lambda, it defines a function.

On the left hand side of the : are the arguments in braces. On the right hand side is the definition.

Functions are all pure in Nix.

Technically, functions in Nix all take a single argument. The { a, b } brace syntax for arguments is a destructuring bind.

Derivation

A derivation is a function which returns a hash.

The hash represents a location in the /nix-store directory. This will be passed to the derivation's builder as the $out environment variable.

A derivation contains some special properties like name, version and builder. Other properties are treated as enivronment variables which the builder will receive.

The builder creates a shell script which sets up files in the $out location as needed.

Nixops

Provides the nixops command.

This lets you list, add to or sync the various machine definitions.

You need to write two definitions (.nix files):

  • machine definition (this is like /etc/nixos/configuration.nix)
  • deployment definition (how will it be run on your service

At CSE, we use a libvirtd deployment definition. However, you can also write them for cloud VPS things.

Nix on Debian

Nix config

/etc/nix/nix.conf

Probably nixos has a way to set these?

Or maybe they should be in your profile instead?

This is useful for a Debian install: buid-use-chroot = true

This is useful for developers: gc-keep-outputs = true

nix-shell

nix-shell -p package-a package-b --run command

You can also write a special hashbang inside your script files:

#!/usr/bin/env nix-shell
#! nix-shell -i python3 -p python3

Deployment

nix-copy-closure --to me@my.machine /nix/store/some-gibberish34985705ru34-x.y.z pushes a Nix package to another machine over ssh.

nix-copy-closure --from person@online.repo /nix/store/goop-5r34-x.y.z pulls a package.

You can also push to and pull from a location on your file system. This will use a serialized form of the Nix expressions.

Channels

Channels provide a list of what packages they offer.

They can offer multiple possible subsitutes (versions) for the same package.

There are two major sources of channels:

https://nixos.org/channels/nixos-yy.mm
stable, includes a cache
  • append /nixexprs.tar.xz?
https://https//github.com/NixOS/nixpkgs-channels/archive/nixos-yy.mm.tar.gz
always up to date

Binary Caches

This is a web server offering:

  1. the serialized form of our built packages
  2. a zip containing the expressions for those packages

We can then point nix.binaryCaches at the URL for (1) and NIXPATH at the URL for (2).

Or we can use nix-serve.

default.nix

Put this in the root of your project.

If you run nix-shell, it will look for a default.nix.

Javascript / Node.js

The nixpkgs repository explains how to port NPM packages using node2nix.

Although actually, this is the wrong way for me.

node2nix does the equivalent of npm shrinkwrap (finds all the versions you want).

Python

python.buildEnv.override let us make an environment to run out python stuff in

buildPythonPackage

  • buildInputs for build-time dependencies
  • propagatedBuildInputs for run-time dependencies (most things)

Hmm. Which do I use? Why?

Python 2 virtualenv problem

See: use virtualenv/pip with nix-shell

Python 2 has some C modules linked from its standard libraries.

These break using Nix (so e.g. we can't import sqlite3).

Make sure to install python27Full (attribute path nixos.python27Full)

Run the following when creating virtualenv to make sure we get the things we need. virtualenv -p $(readlink $(which python2.7)) venv --no-site-packages

Useful things to look up

nix-prefetch-url nix-prefetch-git fetchurl

Derivations

Derivations are a set {type = "derivation"}.

Derivations all have an outPath.

If you apply builtins.toString to a derivation, it knows to lookup the outPath.

Derivations have a builder and an optional args. The builder should be executable (e.g. bash), and the args will be passed to it (e.g. a bash script).

Steps

First, Nix does instantiation/evaluation of all of the derivations (works everything out).

Then it builds/realises them all.

Organisation

Make a folder to contain a group of related .nix files.

Each .nix package is a function mapping arguments to a derivation as follows:

{ stdenv, fetchurl, dependencyA, dependencyB }:

stdenv.mkDerivation {
    name = "package-1.2.3";
    src = fetchurl {
  url = "https://www.example.com/thing.tar.gz";
  sha256 = "1ymhvpa6zqg6kw8vf0q7vw4kdjb1pm2129arb9cgylgynfb36lkm":
    };

    # Excetera
}

Then there is a final file default.nix:

{ pkgs ? (import <nixpkgs> {}) }

with pkgs; rec {
    child = callPackage ./child.nix {};
    parent = callPackage ./parent.nix {
  inherit child;
    };
}

This layout works by use of rec and lazyness.

We can test each package individually using nix-shell -A package-name or nix-build -A package-name.

Versions

We can use environment variables to control which Nix channel we use.

For example:

export NIX_PATH="nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixpkgs-unstable.tar.gz"

nix-env weirdness

nix-env doesn't use NIXPATH by default.

Instead it looks in ~/.nix-defexpr

Fine.

Sandbox

To build in a sandbox:

nix-build --option build-use-sandbox true -A some.package

This –option syntax works with nix-shell too.

Nixos modules

https://github.com/NixOS/nixpkgs/blob/master/lib/modules.nix

Make a function which takes config {config, pkgs, ...}:.

Make an attribute set which contains options, config and imports.

Options are made using mkOption.

Config is made using mkIf, allowing it to be turned on or off based on sevices.<ourModule>.enabl.

There are a variety of other control flow statements we can use: https://nixos.wiki/wiki/NixOS:Properties.

packageOverrides

This is a fixed point function of nixpkgs.

It lets us changes one of our packages, and have that propagate down to all the others. It lazily evaluates one step at a time.

Nix User Files

~/.nix-channels
controlled with nix-channel. Doesn't seem important?
~/.nix-defexpr
this is an offline cache of the channel definition
  • used by nix-env unless you tell it otherwise
  • has a /channels folder, which points to /nix/var/nix/profiles/per-user/$USER/channels
  • changed by nix-channel --update
  • if you delete all the channels, the last one won't actually be removed
~/.nix-profile
a symlink to /nix/var/nix/profiles/default
  • has a Nix daemon
  • has nix.sh, a script to setup Nix on load
  • contains a manifest with all your installed things
  • contains symlinks to your installed things
  • NIXLINK environment variable points to it
NIXPATH
should point to ~/.nix-defexpr
NIXLINK
points to ~/.nix-profile by default, not clear what it does