Deploying NixOS to Amazon EC2

We start on the NixOS homepage, which invites us in with a big friendly green “Get NixOS” button. This takes us to the download page, which has links to a handful of ways to install: USB sticks for your desktop, virtual machine images, and links to launch on Amazon EC2 and Microsoft Azure.

We’re using AWS just because it’s the platform I’m most familiar with. Amazon makes us choose which geographic region we want our server to run in; AWS does this because they power a lot of large businesses who care deeply about uptime – if we make sure to spread out our servers over multiple regions, then even if, say, a hurricane knocks out Amazon data centers in some part of the world, our business can keep running. This is a more sophisticated concern than we care to think about right now, so just choose any region, perhaps preferring one on the same continent as you.

When you click “Launch”, you’ll be prompted to log into the AWS console if you are not already logged in. Then you’ll get to the seven-step process for launching an instance. Seven steps seems like a lot, and indeed it’s a bit cumbersome. For AWS tasks that you perform repeatedly, you’ll want to eventually wean off of the web console and learn to do them with the command line interface.

1. The first step is selecting the virtual machine image (“AMI,” in Amazon parlance). The “Launch” button from the NixOS website included an appropriate choice of AMI and skipped us past this step, so we start on step 2.

2. The “instance type” determines (among other things) how many processors and how much RAM your server will get. It also affects the price. We aren’t particularly demanding on our servers, so one of the smaller types like t2.micro will suffice. You can run a t2.micro instance at no cost during your first year of using AWS, after which it will cost somewhere in the vicinity of $10/month. This gives us a single processor and 1 GB of memory.

3. A handful of configuration options we don’t need to worry about for now.

4. Storage - The disk capacity defaults to 3 GB. This is certainly too small. The main Type Classes server currently uses 5 GB of its storage. I like to provide plenty of padding here because you don’t want to run out of storage on a running server, It is possible to resize the disk of a running instance, but I’d rather not. so I think 30 GB is a good standard choice.

5. Tags - Another AWS feature we don’t need to think about right now.

6. Security group - By default, your instance is tightly firewalled and no incoming traffic can reach it. The security group is what lets you relax the rules and specify what ports you need to accept connections on. At minimum we’ll need port 22 (the standard port for SSH). Since Type Classes is a website, we also accept on ports 80 and 443 (the standard ports for HTTP and HTTPS). It’s very easy to forget about security groups; in general, if something among your EC2 instances isn’t working, it’s likely that it’s because you forgot to allow a port in some security group. If you don’t already have an appropriate security group created, this step lets you create a new one right here.

7. The last step just shows you a summary of what you’ve entered in the previous steps.

Once you click “Launch” in the bottom-right, you get an additional bonus step. AWS will automatically initialize the instance with a keypair that you can use to SSH into it as root. As with security groups, you might have a keypair set up on your AWS account already, or otherwise you can have AWS create one for you as part of this step. In this project we’re assuming the latter; choose “Create a new keypair” and click the “Download Key Pair” button. This will download a .pem file which contains the secret key that you will need to SSH into the machine. In this project we name the key “type-classes-demo-1” and so it downloads to the path ~/downloads/type-classes-demo-1.pem.

Set the permissions on the .pem file to avoid exposing it to other users on your system. Setting file permissions may seem unnecessary if the computer is, for example, your laptop that nobody else ever uses. But SSH requires the file permissions be set strictly, so you cannnot skip this step.

Finally, this “Launch” button will actually start the instance. On the next page, click on the instance identifier, An AWS instance identifier looks something like “i-0c434292979382418” and this will take you to the page that shows a bunch of information about the instance, including its status which will be “Initializing…” for a few minutes while the virtual machine boots up.

Once the machine is initialized and running, we can SSH into it. With the instance selected, you can find the IP address in the panel at the bottom of the AWS console. In this example, it is 35.174.155.99.

Once we start a new server, the first thing we probably want to do is open a remote shell on it to poke around and verify that it’s working. We’ll set up more user accounts later, but at the moment, the only user that exists on the server is root, and we need to authenticate using the key we just downloaded. So the form of the SSH command we’ll need to use for this is

which in this example looks like this:

Now that you have a remote shell, the next thing we probably want to do is start making changes to the NixOS configuration. The config file is in the conventional place, /etc/nixos/configuration.nix, and by default it looks like this:

The import of amazon-image.nix provides some default EC2-specific configuration, and you can look at it in the nixpkgs git repository if you’re curious.

We can try making some small changes to the configuration, like setting the hostname:

Then run nixos-rebuild switch to build and activate the new configuration. That isn’t how we want to deploy changes to the server normally, though. We gave this machine a pretty small amount of memory, so any nontrivial build is likely to run out of memory and fail. Plus, in the interest of keeping our servers as responsive to user requests as possible, we don’t really want to waste their time compiling packages. So next we’ll come up with a better way to deploy.

Fortunately, we can take advantage of the fact that “the entire server” is a Nix package, and Nix packages are extremely portable; we can build them on one machine and then ship them off to another. That’s what we’re going to do here: Build all of NixOS on our local development machine, and then upload it to the server.

If we copy the configuration.nix file that we saw a minute ago from the server, and throw this bit of boilerplate into a file called server.nix:

Then we can build NixOS for our server using the nix-build program, giving it server.nix as an argument to specify what we want it to build.

The files get created somewhere within /nix/store; the output of this command includes the path of the directory containing the build result. nix-build also creates a symlink called result which points to that path. This is convenient in a lot of cases, but we’re not going to take advantage of this feature here, so in the future I’ll always use nix-build with the --no-out-link flag to prevent it from creating result. It’s harmless, but we just don’t need it.

So that’s it – The build step is quite simple.

Getting the build result running on the server takes a few steps, so we wrote a Haskell script to do it. You could do this in a Bash script just as easily if that’s your thing, but I just love Haskell so much, and this will be good practice. We use

• the Stack script interpreter,
• turtle (which requires OverloadedStrings), and
• neat-interpolation (which requires QuasiQuotes).

Even in small scripts, there’s no need to sacrifice type safety. The deploy script involves two different kinds of strings, so let’s define a newtype for each.

We run the whole program inside turtle’s Shell monad, using sh :: Shell a -> IO () at the top of main to run it. I like when main serves as a concise outline of what the program does, and this one shows that the deploy process consists of exactly four steps:

For now we have a very simple setup with a single server. We keep the IP address of the server in a file called server-address.txt, and the getServer action reads the contents of that file.

The build action runs the nix-build command that we worked out in the previous section.

Once we’ve built NixOS, we need to upload it to the server. For this purpose, Nix provides the nix-copy-closure command. “Closure” refers to the fact that it copies the “transitive closure” of a package – that is, the package, all of its dependencies, all of their dependencies, etc. – everything that our build depends on either directly or transitively.

The --use-substitutes flag enables a cool feature: If any of the packages we need to upload is available in the main NixOS cache hosted at nixos.org, the server we’re uploading to will download the package from there instead. I use this to get faster deploys because uploads via my home internet connection aren’t particularly fast.

After the upload, the NixOS build is present on the server, but isn’t running yet. Our last step is to switch to it. We do this by

1. setting the new build as the system profile, and then
2. running its switch-to-configuration program to start it.

We construct a shell script that does both of these things (remoteCommand below) and use it as an argument to the ssh command to execute it on the server.

And our deploy.hs script is done! Now the process of making changes to the server is quite simple:

1. Edit configuration.nix.
2. Run ./deploy.hs.

So far we can only deploy as root, and only using the .pem file we downloaded from AWS. Let’s create some users.

Like most things in NixOS, we do this by editing configuration.nix and rebuilding. This is what our config looks like when we add two users chris and julie:

Set isNormalUser = true; for users that correspond to people. This sets a few defaults like specifying that the user should have a home directory under /home.

When we deploy, NixOS will automatically create, modify, or delete users on the system to match what is in the configuation.

We want those users to be able to deploy, so we need to give them some more permissions:

• Add each user to the wheel group to make them sudoers
• Specify what SSH keys the users will use to authenticate The SSH public keys in the example below are the strings starting with “ssh-rsa…” and they have been shortened for brevity here.
• Disable the sudo password prompt to make it easy for our deploy script to run with root permissions
• Allow sudoers to load unsigned packages into the Nix store

As an additional security measure, you may also wish to add

to force your users to always log in using an SSH key, never with a password.

If we want to launch a new EC2 instance, we can do so without repeating most of the steps we took in this project. This is due to a really handy feature of the NixOS AMI: You can copy the contents of configuration.nix into the “user data” field in step 3 of the EC2 launch process, and the machine will automatically launch into the specified configuration the first time it boots. Thus we can create new fully-configured machines in a single step.

Be aware that the user data initialization runs each time the machine starts, not just the first time. This is an open issue, so it may change in the future. If you launch a NixOS EC2 instance with config in the user data, deploy a different config, and then reboot the instance, it will revert back to its original configuration.

• Instructor: Chris Martin
• Window manager: xmonad (configuration: xmonad.hs)
• Text editor: Atom with the language-haskell package
• Font: Fira Code in Atom, Monospace Regular in the terminal
• Syntax theme: Atom Dark in Atom
• Parts of the output of nix-build and nix-copy-closure commands have been edited out of the videos for your viewing convenience. In reality they take a bit longer than we show here.