Duplicate variant directory before running SConscript

Question

I have SConscripts which recursively call SConscripts in all directories contained in the SConscript directory.

This does not work with variant builds as the entire source directory is not copied to the variant directory before evaluating the subsidiary SConscripts.

Example

.
├── SConstruct
└── src
    ├── a
    │   ├── a.c
    │   ├── a.h
    │   ├── SConscript
    │   ├── x
    │   │   ├── ax.h
    │   │   └── SConscript
    │   └── y
    │       ├── ay.h
    │       └── SConscript
    ├── b
    │   ├── b.c
    │   ├── b.h
    │   ├── SConscript
    │   ├── x
    │   │   ├── bx.h
    │   │   └── SConscript
    │   └── y
    │       ├── by.h
    │       └── SConscript
    ├── main.c
    └── SConscript

SConstruct

env = Environment()
objects = SConscript('src/SConscript', exports=['env'], variant_dir='build')
objects = [x for x in env.Flatten(objects) if x is not None]
program = env.Program(target='prog', source=objects)

All other SConscripts

import os

Import('env')
env.AppendUnique(CPPPATH=[Dir('.')])
objects = SConscript(
    dirs=[name for name in os.listdir('.')
          if os.path.isdir(os.path.join('.', name))],
    exports=['env'])
objects = objects + env.Object(env.Glob('*.c'))
Return('objects')

Output

$ scons . --tree=all
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
gcc -o build/main.o -c -Ibuild build/main.c
build/main.c:1:15: fatal error: a.h: No such file or directory
compilation terminated.
scons: *** [build/main.o] Error 1
+-.
  +-SConstruct
  +-build
  | +-build/SConscript
  | +-build/main.c
  | +-build/main.o
  |   +-build/main.c
  |   +-/usr/bin/gcc
  +-prog
  | +-build/main.o
  | | +-build/main.c
  | | +-/usr/bin/gcc
  | +-/usr/bin/gcc
  +-src
    +-src/SConscript
    +-src/main.c
scons: building terminated because of errors.

The src/a and src/b directories are not copied to build/a and build/b before calling build/SConscript. Hence, build/SConscript does not call the subsidiary SConscripts as those directories don't exist yet.

Output where the variant directory is removed from SConstruct

$ scons --tree=all
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
gcc -o src/b/b.o -c -Isrc -Isrc/b -Isrc/b/x -Isrc/b/y -Isrc/a -Isrc/a/x -Isrc/a/y src/b/b.c
gcc -o src/a/a.o -c -Isrc -Isrc/b -Isrc/b/x -Isrc/b/y -Isrc/a -Isrc/a/x -Isrc/a/y src/a/a.c
gcc -o src/main.o -c -Isrc -Isrc/b -Isrc/b/x -Isrc/b/y -Isrc/a -Isrc/a/x -Isrc/a/y src/main.c
gcc -o prog src/b/b.o src/a/a.o src/main.o
+-.
  +-SConstruct
  +-prog
  | +-src/b/b.o
  | | +-src/b/b.c
  | | +-src/b/b.h
  | | +-src/b/x/bx.h
  | | +-src/b/y/by.h
  | | +-/usr/bin/gcc
  | +-src/a/a.o
  | | +-src/a/a.c
  | | +-src/a/a.h
  | | +-src/a/x/ax.h
  | | +-src/a/y/ay.h
  | | +-/usr/bin/gcc
  | +-src/main.o
  | | +-src/main.c
  | | +-src/a/a.h
  | | +-src/b/b.h
  | | +-/usr/bin/gcc
  | +-/usr/bin/gcc
  +-src
    +-src/SConscript
    +-src/a
    | +-src/a/SConscript
    | +-src/a/a.c
    | +-src/a/a.h
    | +-src/a/a.o
    | | +-src/a/a.c
    | | +-src/a/a.h
    | | +-src/a/x/ax.h
    | | +-src/a/y/ay.h
    | | +-/usr/bin/gcc
    | +-src/a/x
    | | +-src/a/x/SConscript
    | | +-src/a/x/ax.h
    | +-src/a/y
    |   +-src/a/y/SConscript
    |   +-src/a/y/ay.h
    +-src/b
    | +-src/b/SConscript
    | +-src/b/b.c
    | +-src/b/b.h
    | +-src/b/b.o
    | | +-src/b/b.c
    | | +-src/b/b.h
    | | +-src/b/x/bx.h
    | | +-src/b/y/by.h
    | | +-/usr/bin/gcc
    | +-src/b/x
    | | +-src/b/x/SConscript
    | | +-src/b/x/bx.h
    | +-src/b/y
    |   +-src/b/y/SConscript
    |   +-src/b/y/by.h
    +-src/main.c
    +-src/main.o
      +-src/main.c
      +-src/a/a.h
      +-src/b/b.h
      +-/usr/bin/gcc
scons: done building targets.

So is there a way to make SCons duplicate the source directory BEFORE running subsidiary SConscripts, or a way to make the subsidiary SConscripts refer to the source directory.

EDIT: I want to keep the dynamic generation of the list of directories. Hardcoding the directory names works, but that isn't a feasible solution as the number of directories that I have is huge and will keep changing.

Yes, by following the basic setup as described in chap. 14 "Hierarchical Builds" and chap. 15 "Separating Source and Build Directories" in the [UserGuide](http://scons.org/doc/production/HTML/scons-user.html). — dirkbaechle, Oct 03 '16 at 11:39
@dirkbaechle The examples in SCons documentation do not consider the case where the subsidiary directories are dynamically generated by using os.listdir(). Hardcoding all the directory names is not a solution as the number of directories increases. — user80551, Oct 03 '16 at 16:58
There's very little value in doing what you are doing? Why not just have the top level SConstruct walk your directory tree and find all the SConscripts and invoke them, rather than recurse. — bdbaddog, Oct 04 '16 at 02:43

score 0 · Answer 1 · edited May 23 '17 at 10:27

Based on https://stackoverflow.com/a/29218216/2694385, the SConscripts must refer to files and folders as if they are already in the variant directory even though, they actually aren't.

The source and variant directories can be saved in the environment, and each SConscript can find the path to the corresponding source directory (based on its depth in the variant directory).

The modified files are

SConstruct

import os

env = Environment()
env['VARIANT_DIR'] = Dir("build")
env['SOURCE_DIR'] = Dir("src")
objects = SConscript(os.path.join(env['SOURCE_DIR'].path, 'SConscript'),
                     exports=['env'],
                     variant_dir=env['VARIANT_DIR'])
objects = [x for x in env.Flatten(objects) if x is not None]
program = env.Program(target='prog', source=objects)

All SConscripts

import os

Import('env')
env.AppendUnique(CPPPATH=[Dir('.')])

dir_depth_from_variant = env['VARIANT_DIR'].rel_path(Dir('.'))
corresponding_src_dir = os.path.join(Dir('.').rel_path(env['SOURCE_DIR']),
                                     dir_depth_from_variant)
dirs = [name for name in os.listdir(corresponding_src_dir)
        if os.path.isdir(os.path.join(corresponding_src_dir, name))]
objects = SConscript(
    dirs=dirs,
    exports=['env'])
objects = objects + env.Object(env.Glob('*.c'))
Return('objects')

Duplicate variant directory before running SConscript

1 Answers1