What’s the problem?

Go is a fantastic language and has been among my favorites for a while. I also enjoy building software with Nix (btw). These two combine seamlessly to provide smooth and reproducible software. You write the code, compile it into a binary, and then deploy it, easy peasy. If you’re involved in open-source, it’s even simpler with public modules acting as repositories, ready for use in your application.

Need functionality from a repository? Run go get -u <git-repo-url>, and voila, it’s available in your go.mod, ready for use in your code. This is the typical experience with Go, and it’s an excellent way to share or implement others’ solutions in your own work. However, what if you’re working at a company or on a private project? This is where things can get complicated, and integrating Go with Nix can become a bit more challenging which I needed to solve for.

How do private modules work with Go?

Private Go modules work similarly to public modules, with the primary difference being access control. When you use private modules, your Go code depends on repositories that are not publicly accessible. This could be a repository in a private GitHub organization, a Bitbucket team, or a self-hosted GitLab instance.

To use private Go modules, you can use the GOPRIVATE environment variable. This variable tells the Go command not to validate or proxy requests to modules with paths that match the given pattern (or patterns).

For instance, if you have a private module on GitHub, you can set GOPRIVATE to use it in your project. This environment variable should be set in your shell or within your Go environment. After setting GOPRIVATE, you can run go get -u <git-repo-url> to download and use the private module, much like you would with a public module.

export GOPRIVATE=github.com/myusername/*
go get -u github.com/myusername/example

To download and use private modules, you must authenticate with the Git server. Depending on your hosting provider, you can do this by using SSH keys, personal access tokens, or OAuth tokens. Go also supports the use of a .netrc file, which I discuss further in the next section.

How do private modules work with Nix?

If you’ve used Nix before, you may have had difficulties with private packages. By default, Nix only supports public locations that don’t require authentication. As a result, using private packages can lead to authentication errors, due to the way Nix sandboxes build environments. If you search online enough, you may stumble across a NixOS Wiki article titled “Enterprise” that addresses this issue.

Nix uses a .netrc file to download files with curl. The netrc file is a mechanism that facilitates automatic authentication to remote hosts when using curl in builds. This file contains the machine name, login, and password information of the remote server from which you want to download files.

machine github.com
login exampleuser
password mypassword

Next, the netrc file needs to be accessible during the builds. We’ll set up Nix to allow direct access to this file from the build sandboxes. To do this, edit your /etc/nix/nix.conf file to include the following lines.

netrc-file = /etc/nix/netrc

When you use functions within Nix to download files, it looks for the .netrc file in the home directory (or configured location) and uses the information provided in the file to authenticate with the server. This way, Nix can automatically handle the authentication process when downloading files.

mypackage = callPackage <mypackage.nix> {
  fetchurl = stdenv.fetchurlBoot;
};

However, it’s important to note that this only applies to one function in Nix, named fetchurlBoot. This function does not work with building Go applications natively, so the search continues.

How do sandboxes work with Nix?

The last piece of the puzzle is understanding sandboxed build environments. Sandboxed environments in Nix are isolated, controlled environments that provide a high degree of reproducibility. In a Nix sandbox, all dependencies are explicitly declared, which means that the environment only has access to the resources it needs to build or run a piece of software. This eliminates the possibility of hidden dependencies and ensures that the software will behave the same way across different systems.

The sandboxing mechanism restricts a process’s visibility and access to the external world. For instance, it usually disallows network access and only permits specific directories to be accessible. These directories often encompass the build inputs and a build directory designated for outputs:

nix build --sandbox ".#mypackage"

This offers several advantages such as ensuring the explicit declaration of all dependencies, preventing build contamination, and establishing a controlled environment for package creation. Nonetheless, it can pose challenges when dealing with private packages, as they frequently need network access for authentication.

How do we join the these together?

If you’ve read up to this point, you’re likely eager to find a solution to this problem, and I understand your struggle. The method I’ve shared reflects how I found the best solution for my needs. This is a significant aspect of working with Nix: finding a solution that best fits your requirements, rather than a “one-size-fits-all” or “only correct” one. I want to emphasize that the solution I’ve presented here worked best for what I aimed to achieve.

The following requirements were necessary for this project:

• It must be the same for both local development and CI pipelines.
• It must not use --impure or other non-pure approaches in Nix, allowing a fully restricted default sandbox environment.
• It must not store any artifacts that contain secrets in the Nix store or any other non-designated location on the system.

For awhile, I was going to give up on this because I couldn’t find a solution that fit all these requirements. I resulted in doing what most of us do, RTFM, or read the “fantastic” manual. Upon doing some research, I read this from the Nix CLI manual about sandboxes:

“Sandboxes are isolated from the normal file system hierarchy and will only see their dependencies in the Nix store, the temporary build directory, private versions of /proc, /dev, /dev/shm and /dev/pts (on Linux), and the paths configured with the sandbox-paths option.”

Eureka! There’s a sandbox-paths option that allows you to include specific paths from the filesystem into the build environment without saving them as artifacts. As I had previous experience with Docker and its volumes feature, I wondered if I could apply a similar strategy here. The idea is to store secrets in a file, then “mount” (or in this context, add) the path to the build environment during runtime.

nix build --sandbox --sandbox-paths "/host-path=/sandbox-path" ".#mypackage"

Almost there, but one more hurdle to overcome. The next issue I encountered was that the file must be accessible by the nix user that builds the environment. By default, Nix uses a specific user to manage build environments. This user may not have access to the file if it is located in your user folder or another secure location. To overcome this, I added a step to temporarily place the file in /tmp directory before the build and then remove it directly after. This also means if the system is rebooted it will automatically get removed if I happened to forget.

nix build --sandbox --sandbox-paths "/tmp/.netrc=/tmp/.netrc" ".#mypackage"

Do you need to put the file in the /tmp directory to get this to work? No, you do not and it’s okay if that’s not where you’d like to put it. You simply need to place it in a location that the Nix build user can read. I chose this location because I trust my development machine and I trust my CI/CD pipeline machines. In the situation that I forgot, the next time I reboot either machine it would be removed on clean up which happens automatically depending on the operating system.

To verify this works, I updated the Nix derivation function to include custom commands in the preBuild phase to check for the file in the sandboxed build environment:

buildGoModule {
  name = "example";
  src = ./.;
  
  env = {
	  # set env vars in the derivation like in development shell
	  GOPRIVATE = "github.com/myusername/*";
  };

  preBuild = ''
    ls -alh /tmp/.netrc
  '';
};

You can check the Nix build logs output and should notice the ls -alh of the file:

example> -rw-r--r-- 1 owner group 140 Apr 30 07:09 /tmp/.netrc

Great! The file is finally in the location it should be within the sandbox and accessible. Now we just need to make sure the Go binary in the sandbox can see the .netrc file.

In the Go documentation it states:

“The location of the file may be set with the NETRC environment variable. If NETRC is not set, the go command will read $HOME/.netrc on UNIX-like platforms or %USERPROFILE%_netrc on Windows.”

Whenever I tried using the NETRC environment variable to set the path it did not work within the Nix build environment. However, there is a workaround which finally gets our build working by setting the $HOME environment variable to the /tmp path as the docs suggest:

buildGoModule {
  name = "example";
  src = ./.;

  env = {
    GOPRIVATE = "github.com/myusername/*";
    HOME = "/tmp"; # set HOME so go binary finds .netrc
  };

  preBuild = ''
    ls -alh /tmp/.netrc
  '';
};

Voila! Now when we run our build inside of Nix the .netrc file is added to the sandbox, accessible by the Go binary and discoverable in the updated $HOME path.

To set this up in Continuous Integration (CI), like GitHub Actions, you can use the extractions/netrc@v2 action. This action sets up the netrc in the action, after which you can copy it to the desired path:

build:
  runs-on: ubuntu-latest
  steps:
    - uses: extractions/netrc@v2
      with:
        machine: github.com
        password: github_pat_1234567890 # personal access token
        username: myusername # username
    - run: cp --no-preserve=mode ~/.netrc /tmp/.netrc
    - run: nix build --sandbox --sandbox-paths "/tmp/.netrc=/tmp/.netrc" ".#mypackage"
    - run: rm -rf /tmp/.netrc
      if: success() || failure()

Conclusion

It’s important to note that while this solution worked well for my specific requirements, it may not be the perfect solution for everyone. The approach I took was tailored to solve a specific set of constraints and needs. Your situation might be different, and therefore, you might need a different solution.

However, the methodology and thought process I followed could serve as a guide when dealing with similar challenges in the Nix environment. Always remember, the best solution is the one that fits your unique requirements and constraints, not necessarily the one that is commonly accepted or used.