Michael Maclean

Why Nix?

I’ve written a lot about Nix and NixOS over the last couple of years, and I use Nix in some way most days that I’m on a computer. Sometimes when I mention this, people ask me what’s so interesting about it. I’m going to try and answer that here, by running through what I find useful about Nix.

If you read the front page of the Nix website, you’ll see a quote like this:

Nix is a tool that takes a unique approach to package management and system configuration. Learn how to make reproducible, declarative and reliable systems.

All of the above is true, and some folks may be able to see what is attractive about that immediately. I was not entirely one of those people when I started off, though I’d heard a lot about it from others. I’m still not an advanced Nix user by any stretch. I don’t really properly understand the Nix language yet, but this hasn’t particularly stopped me finding value in it.

Background

Normally, on a Linux machine, you’ve got a package manager that can install whatever you like, whenever you need it. If you’re on a Debian or Ubuntu machine, you might want to do some basic resizing of an image, and end up doing something like this:

me@laptop:~$ convert -geometry 1024x big.jpg small.jpg
bash: convert: command not found

me@laptop:~$ sudo apt install imagemagick
# Much text ensues

me@laptop:~$ convert -geometry 1024x big.jpg small.jpg
# Success!

This demonstrates one of the key points I want to get across here: on a conventional Linux distribution, these little changes accrete. If you need to wipe your machine, move to a new one, or just use a different computer for a little while, many of the things that you expect to be there are not going to be present. You might end up encountering these little paper cuts for some time as you continue working. Additionally, you end up with tools installed that you might use once and never again, taking up space.

At work I generally use macOS, and on there I used to use Homebrew to install things. For 90% of what I do this was fine, but one thing tipped me over the edge into finally installing Nix. I was doing some data work which required me to use PostgreSQL. I’d just used brew install postgresql and got on with it at the time, without thinking too hard about what version was installed. It was a few weeks later that the version of PostgreSQL in Homebrew was upgraded as a side effect of installing something else, and the next time I looked at my Postgres DB it had been wiped and started from scratch. The data was still there, it was just in a different directory with the version number in the path, but the running daemon had been upgraded to a different one so it had started from an empty DB. There are other ways to solve this problem, but by this point I’d heard enough about “reproducibility” and “immutability” to wonder if this would be something that I could use.

I installed Nix on macOS, and found out how to install things with nix-env.

$ nix-env -iA nixpkgs.imagemagick

This is basically what I had before with apt or brew, except that it pulls from Nix’s own repository, called nixpkgs. You can install things, and they appear in your shell environment. It’s just different, not better.

What is better is nix-shell. With this, you can create a new shell with any set of tools that you like present. For example:

$ nix-shell -p imagemagick

This drops me into a new shell with ImageMagick present. When I close it again, it’s no longer there. If I don’t use it again eventually it’ll be garbage collected. I use this a bit for tools that I don’t need all of the time.

shell.nix

Shortly after this, I learned enough to discover shell.nix files. This is the part of Nix that I probably still use the most. These are files you can write and keep inside your projects that describe things you want to be available when you’re working on that bit of code. An example I have from the repository for this website looks like:

with import <nixpkgs> {};

pkgs.mkShell {
  buildInputs = [
    pkgs.zola
  ];
}

This just means that if I change into the directory for the site, I can type nix-shell and that file will be loaded, and a new shell will result with zola available. Once I quit that shell, zola disappears from the environment. This is more important than it may appear, and results in next two key points: I can have different versions of tools available in different projects, and you can ensure you have only the tools you require. If I stop working on a project that has a particular dependency, it can get garbage collected and it’s no longer present on the system.

I commit these shell.nix files to my projects, and I have blogged marginally more complex examples before for keeping a PostgreSQL environment per-project and also for the Go tools that VS Code wants. For other projects at work where I’m the only person using Nix to install the build tools, I’ll sometimes keep them to myself by adding them to .git/info/exclude.

A neat feature of this is that you can override how the packages you install are defined. For example, if you want to create a Python 3 environment with some packages from PyPI, you can do something like this:

with import <nixpkgs> {};

let
 my-python-packages = python-packages: with python-packages; [
    pillow
    numpy
    black
  ];
  python-with-my-packages = python3.withPackages my-python-packages;
in
pkgs.mkShell {
  buildInputs = [
    python-with-my-packages
  ];
}

Direnv and Lorri

nix-shell is pretty neat, but it’s a bit like Python virtualenvs where you need to remember to activate and deactivate them as you go. You can use a tool called direnv with its Nix addon to automate this for you.

Once you have that configured, setting up a project is pretty simple. You’ll need a shell.nix as above, and then you create a file to tell direnv what to do.

$ echo 'use nix' > .envrc
direnv: error .envrc is blocked. Run `direnv allow` to approve its content.

The error here demonstrates that you have to explicitly permit direnv to work in this directory. It prevents you accidentally running code in your shell directly from a Git checkout, or similar.

$ direnv allow

Then all of your Nix dependencies will be installed, and you’ll return to your prompt in the same shell. It can take a little while, and if you do nix-channel --update or similar it will often do it again, but it’s pretty neat.

To take some of the time out of this, you can combine direnv with lorri. It’s a daemon that runs in the background and can react live to updates in shell.nix. It also prevents your dependencies being cleared out if you run nix-collect-garbage–more on that command later. This also makes it easier to kick off new projects using lorri init instead of creating the files manually like I described.

These days, at work, I’ve switched roles into one where I don’t do so much development as I used to, so I don’t actually use lorri any more. It’s still a pretty cool tool though and I’ll probably use it again if I return to a full-time software development role.

Home Manager

With Nix, I can create individual environments for projects I’m working on. It’d be cool if I could extend this idea to my entire login environment. With Home Manager, I can. At a basic level, you can install it and do essentially the same thing as I described with shell.nix above, but it affects all of your shells. You can type home-manager edit and you’re dropped into an editor where you can make whatever changes you need, before using home-manager switch to apply them.

Here’s a basic example from one of my machines:

{ config, pkgs, ... }:

let
  pkgsUnstable = import <nixpkgs-unstable> {};
in
{
  home.stateVersion = "19.09";

  home.packages = with pkgs; [
    awscli
    direnv
    git
    htop
    wget
    ripgrep
    jq
    pkgsUnstable.asuka
  ];
}

Here I’m installing some of the tools I expect to be available all the time, including a newer version of git than is provided by macOS or Xcode. I normally use the stable version of a Nix release, but here I wanted to use a Gemini client called asuka which wasn’t available in a stable release when I set this up, so I installed that from the unstable release.

If you keep your configuration in Git or somewhere like that, you can share it across machines, and have the same environment present everywhere you do work.

Home Manager lets you do more than just install packages, though. You can configure them too, and it’s a convenient way to manage dotfiles. For example, I keep my Git configuration in there, and so Home Manager will write that to ~/.gitconfig for me.

programs.git = {
  enable = true;
  userName  = "Michael Maclean";
  userEmail = "michael at thisdomain dot net";
};

It can also start services for you at login. One example of this is gpg-agent:

services.gpg-agent = {
  enable = true;
  defaultCacheTtl = 1800;
  enableSshSupport = true;
};

You’re also not limited to using it to install command line tools. On one of my Linux machines, I use it to configure VS Code and all the extensions I use. (I don’t use this on macOS–I’ll talk about that later.)

I use this on every machine I have access to. The configuration is kept in Git so I can update it, and as long as I remember to pull the changes on my other machines, I end up having the same set of tools available.

NixOS

If you’ve read this far, you hopefully are able to see why I find Nix valuable. It lets me install the tools that I use and keep them configured the way I want in one place. It would be nice if I could build an entire system this way, which is where NixOS comes in. It’s a Linux distribution that fundamentally can be configured in the same way as I described above, using a Nix file to describe the state of the system you want. This differs from other configuration management tools, where sometimes you have to describe the set of actions that must be taken to make the system you want.

The nicest thing about NixOS for me is that some thought has gone into how the services on your machine should be configured. Instead of having to write separate files in different languages and scatter them all across /etc, they’re all in one place (/etc/nixos/configuration.nix) and use the same language. I run a Miniflux RSS reader, and configuring that was really simple and clear:

  services.miniflux = {
    enable = true;
  };

  services.nginx = {
    enable = true;

    recommendedGzipSettings = true;
    recommendedOptimisation = true;
    recommendedProxySettings = true;
    recommendedTlsSettings = true;

    virtualHosts."rss-reader" = {
      forceSSL = true;
      enableACME = true;
      locations."/" = {
        proxyPass = "http://127.0.0.1:8080";
      };
    };
  };

  security.acme = {
    acceptTerms = true;
    email = "blah";
  };

I’m not going to go into a lot more detail here as I’ve already covered some of what the rest of the system configuration looks like in previous posts. If you’d like to see what this might look like, check out these posts:

Using NixOS doesn’t mean you need to throw out your Home Manager setup, either. I use NixOS’s own configuration to set up the hardware and the display manager, and to run services like OpenSSH on the machine, but all the software I want as a user (including Firefox, VS Code, that sort of thing) is set up using Home Manager. They coexist well.

Things I don’t use

Nix is a package manager, so obviously it can be used to package software for distribution. It can also build machine images, Docker images, Snaps, Flatpaks, and probably other things I’m forgetting. I don’t actually use it for much of that, which is related to me not doing full-time software development at the moment.

I’ve also not tried to push the use of Nix for building software at work, partly due to the learning curve. I’m happy to help people as far as I can if they come to it themselves, but I’m not an evangelist for it there.

Although I use Nix on macOS quite happily, I don’t use Nix to install GUI software there. I do on Linux. On macOS, most of the packages I’m interested in (things like Virtualbox) are marked as Linux-only and while I could override that, I found it didn’t really work, so I’ve not directly replaced brew cask with Nix. There is a tool called nix-darwin that I believe can do some of this, but I’ve not tried it. On my work computer I suspect it may upset the MDM, and on my home Mac I’m happy enough with Home Manager.

In all three environments–nix-shell, Home Manager and NixOS–you can freeze the version of nixpkgs you are installing from, to specifically lock your environment to the versions of the dependencies that you know work. This is similar to a lock file in a package manager like Composer or NPM. I have used this a bit, but on the whole I’ve not had any trouble from just tracking the stable version of nixpkgs.

In summary

I get a lot of value out of Nix even at my current “advanced beginner” level of experience. I still don’t really know the Nix language. I’ve not used Flakes. I’ve not tried NixOps properly yet. All of these are things I am going to learn, but not knowing them has not held me back. I’ve been able to do all the things I’ve described, and do some contribution to nixpkgs itself, by learning from others and using the code and documentation that is already out there.

If you’re a user of Nix already, you may note I’ve not talked about the core parts of how Nix works. I’ve not mentioned the store, nor immutability, nor really reproducibility–these are things that are important to how Nix works, but I find you don’t really need to understand them to make use of it.

I have had a lot of assistance along the way. The Nix community has been pretty welcoming, although #nixos on Libera IRC is a pretty busy place. Graham has helped me and many others out a lot in the past, and I’ve learned a fair bit from discussions with Sean and Joël too. Thanks to all of them, and those in the community that have helped me in the past.