1

I'm trying to create a flake for the peer to peer web browser, agregore ( https://github.com/AgregoreWeb/agregore-browser ). It's my first flake and I'm having major difficulties.

Here's the full flake, which I have placed in a clone of the above agregore git repo. I think the relevant section is all just this:

packages.default = stdenv.mkDerivation {
  pname = "agregore";
  version = (builtins.fromJSON (builtins.readFile ./package.json)).version;
  buildInputs = [
    pkgs.git
    pkgs.nodejs
    pkgs.nodePackages.node-gyp
    pkgs.nodePackages.npm
    pkgs.yarn
  ];
  buildPhase = ''
    git submodule update --init --recursive
    yarn
  '';
  installPhase = ''
    mkdir -p $out/bin
    #TODO find the build output
    ln -s build/agregore $out/bin/agregore
  '';
  src = ./.;
};

Logs produced:

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/va0hnynmqf8h29f580sb6h7d3z9kxcpm-1q603n538l>
source root is 1q603n538laqf3nvkgb5mdr6am8apbda-source
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
fatal: not a git repository (or any of the parent directories): .git

I've tried inserting commands like ls .git/ > out.txt into the buildPhase, and this produces ls: cannot access '.git/': No such file or directory, so they are indeed not being copied over to nix's temporary build directory for this flake (and running the same command with build/ instead of .git/ doesn't produce this error.). I can't understand why this would be.

The same command, git submodule update --init --recursive, works fine in the original project dir (and I'm running nixos btw), also within a nix develop session, so the .git directory really should to be there?

mako
  • 1,201
  • 14
  • 30
  • 1
    Yes, that's normal/correct/intended behavior. Nix has options to check out submodules itself; you can't/shouldn't do it after the fact. – Charles Duffy Jun 07 '23 at 23:35
  • 1
    In `fetchgit` and `builtins.fetchGit`, see `fetchSubmodules = true`. – Charles Duffy Jun 07 '23 at 23:39
  • 1
    Remember, Nix builds your code in a sandbox with no network access. You couldn't pull referenced submodules even if you _did_ have a `.git`. – Charles Duffy Jun 07 '23 at 23:39
  • 1
    BTW, if this code is from a flake, use `self` instead of `./.` – Charles Duffy Jun 07 '23 at 23:40
  • ... https://github.com/NixOS/nix/pull/4435 is also potentially relevant/interesting/useful background. – Charles Duffy Jun 07 '23 at 23:42
  • (the other thing is that flakes are pure-by-default; allowing content that isn't hash-addressed -- like the `.git` directory's contents -- would be defeating a big chunk of the point; thus, despite it being a bit of a pain, I'd argue that it's good practice to have everything that might otherwise be pulled as a submodule be listed as a separate flake input). – Charles Duffy Jun 07 '23 at 23:43
  • Hmm, `self` is never explained in https://nixos.wiki/wiki/Flakes . Is it the flake's directory in the store or something? – mako Jun 09 '23 at 00:27
  • 1
    Yes, the flake's directory is added to the store and passed as `self`, with all the same metadata you get from other flake inputs (so you can ask for the git committish and so forth). – Charles Duffy Jun 09 '23 at 12:56

1 Answers1

1

I believe mkDerivation by default cleans .git from the source, because it's not appropriate to include it in a reproducible build (.git contains things like dangling references to old branches and the like which are completely client-specific). That said, in my opinion git submodule update --init --recursive doesn't really belong in the Nix code, because the Nix code assumes that it's operating on a "sane" source structure.

l0b0
  • 55,365
  • 30
  • 138
  • 223
  • I see, that makes sense, though it's surprise to me, because I seem to remember hearing somehwere that .git directories are sometimes included in nix packages? (there was explicitly a warning to make sure your .git directory didn't contain any secrets) – mako Jun 08 '23 at 00:42
  • You have to explicitly include it (and it's probably a bad idea in any case); it looks like the relevant parameter is `leaveDotGit`. – l0b0 Jun 08 '23 at 01:32
  • Huh, I'm noticing that .gitmodules is empty, xD I think mauve already replaced the submodules with something else and that build instruction is no longer needed – mako Jun 08 '23 at 02:03