Page:Ludovic Courtès - Functional Package Management with Guix.djvu/2

2. BACKGROUND

This section describes the functional package management paradigm and its implementation in Nix. It then shows how Guix differs, and what the rationale is.

2.1 Functional Package Management

Functional package management is a paradigm whereby the build and installation process of a package is considered as a pure function, without any side effects. This is in contrast with widespread approaches to package build and installation where the build process usually has access to all the software installed on the machine, regardless of what its declared inputs are, and where installation modifies files in place.

Functional package management was pioneered by the Nix package manager^[1], which has since matured to the point of managing a complete GNU/Linux distribution^[2]. To allow build processes to be faithfully regarded as pure functions, Nix can run them in a chroot environment that only contains the inputs it explicitly declared; thus, it becomes impossible for a build process to use, say, Perl, if that package was not explicitly declared as an input of the build process. In addition, Nix maps the list of inputs of a build process to a statistically unique file system name; that file name is used to identify the output of the build process. For instance, a particular build of GNU Emacs may be installed in /nix/store/-v9zic07iar8w90zcy398r745w78a7lqs-emacs-24.2, based on a cryptographic hash of all the inputs to that build process; changing the compiler, configuration options, build scripts, or any other inputs to the build process of Emacs yields a different name. This is a form of on-disk memoization, with the /nix/store directory acting as a cache of "function results"—i.e., a cache of installed packages. Directories under /nix/store are immutable.

This direct mapping from build inputs to the result’s directory name is basis of the most important properties of a functional package manager. It means that build processes are regarded as referentially transparent. To put it differently, instead of merely providing pre-built binaries and/or build recipes, functional package managers provide binaries, build recipes, and in effect a guarantee that a given binary matches a given build recipe.

2.2 Nix

The idea of purely functional package started by making an analogy between programming language paradigms and software deployment techniques^[1]. The authors observed that, in essence, package management tools typically used on free operating systems, such as RPM and Debian’s APT, implement an imperative software deployment paradigm. Package installation, removal, and upgrade are all done in-place, by mutating the operating system’s state. Likewise, changes to the operating system’s configuration are done in-place by changing configuration files.

This imperative approach has several drawbacks. First, it makes it hard to reproduce or otherwise describe the OS state. Knowing the list of installed packages and their version is not enough, because the installation procedure of packages may trigger hooks to change global system configuration files^[3][4], and of course users may have done additional modifications. Second, installation, removal, and upgrade are not transactional; interrupting them may leave the system in an undefined, or even unusable state, where some of the files have been altered. Third, rolling back to a previous system configuration is practically impossible, due to the absence of a mechanism to formally describe the system’s configuration.

Nix attempts to address these shortcomings through the functional software deployment paradigm: installed packages are immutable, and build processes are regarded as pure functions, as explained before. Thanks to this property, it implements transparent source/binary deployment: the directory name of a build result encodes all the inputs of its build process, so if a trusted server provides that directory, then it can be directly downloaded from there, avoiding the need for a local build.

Each user has their own profile, which contains symbolic links to the /nix/store directories of installed packages. Thus, users can install packages independently, and the actual storage is shared when several users install the very same package in their profile. Nix comes with a garbage collector, which has two main functions: with conservative scanning, it can determine what packages a build output refers to; and upon user request, it can delete any packages not referenced via any user profile.

To describe and compose build processes, Nix implements its own domain-specific language (DSL), which provides a convenient interface to the build and storage mechanisms described above. The Nix language is purely functional, lazy, and dynamically typed; it is similar to that of the Vesta software configuration system^[5]. It comes with a handful of built-in data types, and around 50 primitives. The primitive to describe a build process is derivation.

derivation { 
  name = "example-1 .0"; 
  builder = "${./static-bash}"; 
  args = [ "-c" "echo hello > $out" ]; 
  system = "x86_64-linux"; 
}

Figure 1: Call to the derivation primitive in the Nix language.

Figure 1 shows code that calls the derivation function with one argument, which is a dictionary. It expects at least the four key/value pairs shown above; together, they define the build process and its inputs. The result is a derivation, which is essentially the promise of a build. The derivation has a low-level on-disk representation independent of the Nix language—in other words, derivations are to the Nix language what assembly is to higher-level programming languages. When this derivation is instantiated—i.e., built—, it runs the command static-bash -c "echo hello > $out" in a chroot that contains nothing but the static-bash file; in addition, each key/value pair of the derivation argument is reified in the build process as an environment variable, and the out environment variable is defined to point to the output /nix/store file name.

Before the build starts, the file static-bash is imported under /nix/store/…-static-bash, and the value associated

1. E. Dolstra, M. d. Jonge, E. Visser. Nix: A Safe and Policy-Free System for Software Deployment. In Proceedings of the 18th Large Installation System Administration Conference (LISA '04), pp. 79-92, USENIX, November 2004.
2. E. Dolstra, A. Löh, N. Pierron. NixOS: A Purely Functional Linux Distribution. In Journal of Functional Programming, (5-6) , New York, NY, USA, November 2010, pp. 577-615.
3. R. D. Cosmo, D. D. Ruscio, P. Pelliccione, A. Pierantonio, S. Zacchiroli. Supporting software evolution in component-based FOSS systems. In Sci. Comput. Program., 76(12) , Amsterdam, The Netherlands, December 2011, pp. 1144-1160.
4. O. Crameri, R. Bianchini, W. Zwaenepoel, D. Kostić. Staged Deployment in Mirage, an Integrated Software Upgrade Testing and Distribution System. In In Proceedings of the Symposium on Operating Systems Principles, 2007.
5. A. Heydon, R. Levin, Y. Yu. Caching Function Calls Using Precise Dependencies. In Proceedings of the ACM SIGPLAN 2000 conference on Programming Language Design and Implementation, PLDI '00, pp. 311-320, ACM, 2000.