Nix overlays and override pattern

Question

I have trouble understanding Nix overlays and the override pattern. What I want to do is add something to "patches" of gdb without copy/pasting the whole derivation.

From Nix Pills I kind of see that override just mimics OOP, in reality it is just another attribute of the set. But how does it work then? Override is a function from the original attribute set to a transformed one that again has a predefined override function?

And as Nix is a functional language you also don't have variables only bindings which you can shadow in a different scope. But that still doesn't explain how overlays achieve their "magic".

Through ~/.config/nixpkgs I have configured a test overlay approximately like this:

self: super:
  {
     test1 = super.gdb // { name = "test1"; buildInputs = [ super.curl ]; };
     test2 = super.gdb // { name = "test2"; buildInputs = [ super.coreutils ]; };
     test3 = super.gdb.override { pythonSupport = false; };
  };

And I get:

nix-repl> "${test1}"
"/nix/store/ib55xzrp60fmbf5dcswxy6v8hjjl0s34-gdb-8.3"

nix-repl> "${test2}"
"/nix/store/ib55xzrp60fmbf5dcswxy6v8hjjl0s34-gdb-8.3"

nix-repl> "${test3}"
"/nix/store/vqlrphs3a2jfw69v8kwk60vhdsadv3k5-gdb-8.3"

But then

$ nix-env -iA nixpkgs.test1
replacing old 'test1'
installing 'test1'

Can you explain me those results please? Am I correct that override can just alter the "defined interface" - that is all parameters of the function and as "patches" isn't a parameter of gdb I won't be able to change it? What is the best alternative then?

After playing in repl with // operator it seems it doesn't work on derivations the same way as on attribute sets. But then I wonder why I read in Nix Pills that in principle a derivation is just a normal set. — fiction, Aug 11 '19 at 10:45
I don't understand what is being asked. It's also not clear what 'behavior' you find surprising. — Chris Stryczynski, Aug 17 '19 at 16:12
Before doing nix-instantiate (or similar) a derivation is basically just an attribute set with a special "type" (at least according to Nix Pills). So I should be able to create a new one with a few parameters changed - "new = old // ..." What confuses me then is that when I try to check what new is, it is the same as old. Like test1 and test2 have different buildInputs (beside the name) so I expected them to have a totally different nix store path. But I guess the flaw in my thinking was that "out" is also a value in that attribute set (I somehow thought this would all happen at "build" time). — fiction, Aug 18 '19 at 00:00
It would probably be best to edit the question / title in that case. But just from my understanding - the hash is based on the output rather than the input. — Chris Stryczynski, Aug 18 '19 at 09:07

fiction · Accepted Answer · 2019-08-21T07:55:20.787

I will write an answer in case anyone else stumbles on this.

Edit 21.8.2019:

what I actually wanted is described in https://nixos.org/nixpkgs/manual/#sec-overrides

overrideDerivation and overrideAttrs

overrideDerivation is basically "derivation (drv.drvAttrs // (f drv))" and overrideAttrs is defined as part of mkDerivation in https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix

And my code then looks like:

gdb = super.gdb.overrideAttrs (oldAttrs: rec {
  patches = oldAttrs.patches ++ [
     (super.fetchpatch {
       name = "...";
       url = "...";
       sha256 = "...";
     })
 ];
});

The question title is misleading and comes from my fundamental misunderstanding of derivations. Overlays work exactly as advertised. And they are probably also not that magic. Just some recursion where endresult is result of previous step // output of last overlay function.

What is the purpose of nix-instantiate? What is a store-derivation?

Correct me please wherever I am wrong.

But basically when you evaluate Nix code the "derivation function" turns a descriptive attribute set (name, system, builder) into an "actual derivation". That "actual derivation" is again an attribute set, but the trick is that it is backed by a .drv file in the store. So in some sense derivation has side-effects. The drv encodes how the building is supposed to take place and what dependencies are required. The hash of this file also determines the directory name for the artefacts (despite nothing was built yet). So implicitly the name in the nix store also depends on all build inputs.

When I was creating a new derivation sort of like Frankenstein based on tying together existing derivations all I did was create multiple references to the same .drv file. As if I was copying a pointer with the result of getting two pointers pointing to the same value on the heap. I was able to change some metadata but in the end the build procedure was still the same. Infact as Nix is pure I bet there is no way to even write to the filesystem (to change the .drv file) - except again with something that wraps the derivation function.

Override on the other hand allows you to create a "new instance". Due to "inputs pattern" every package in Nix is a function from a dependencies attribute set to the actual code that in the end invokes the "derivation function". With override you are able to call that function again which makes "derivation function" get different parameters.

Nix overlays and override pattern

1 Answers1