6

I am building an Electron app (running an Angular application) which acts as the User Interface for a python program underneath.

The python program uses anaconda for package management (I am using miniconda for development).

When the app boots up, it checks whether the required conda environment exists, and if not, creates it.

The following code is part of a Service which is responsible for managing the python program.

public doEnvironmentSetup() {
    let stdOutSub = new Subject<string>();
    let stdErrSub = new Subject<string>();
    let completeSubject = new Subject<string>();
    this.runningSetup = true;

    const TEMP_ENV_FILE = join(tmpdir(), "env.yml");

    return Promise.resolve()
      .then(() => {

        // Copy packaged environment.yml to TEMP_ENV_FILE

      })
      .then(() => this.electron.getApplicationStoragePath())
      .then((stor) => {

        setTimeout(() => {

          let runProcess = this.electron.childProcess.spawn("conda", ["env", "create", "--file", TEMP_ENV_FILE, "--name", CONDA_ENV_NAME], {
            cwd: stor
          });

          const stdOutReaderInterface = createInterface(runProcess.stdout);
          const stdErrReaderInterface = createInterface(runProcess.stderr);

          stdOutReaderInterface.on('line', (line) => {
            stdOutSub.next(line);
          });

          stdErrReaderInterface.on('line', (line) => {
            stdErrSub.next(line);
          });

          runProcess.on('close', (code: number) => {
            this.electron.fs.unlinkSync(TEMP_ENV_FILE);
            this.runningSetup = false;
            completeSubject.next("");
          });

        }, 2000);

        return {
          stdOut: stdOutSub,
          stdErr: stdErrSub,
          onComplete: completeSubject
        };

      });

  }

Now, when I need to run the actual python code, the piece of code which runs is (not giving the whole thing, since it is too long for our purpose here) :

        execCmd.push(
          `conda init ${this.electron.os.platform() === "win32" ? "powershell" : "bash"}`,
          `conda activate ${CONDA_ENV_NAME}`,
          // long python spawn command
          `conda deactivate`,
        )

        setTimeout(() => {

          logLineSubject.next({ out: "--- Setting up Execution Environment ---", err: "" });

          logLineSubject.next({ out: `Running in ${dir}`, err: "" });

          const cmd = execCmd.join(" && ");

          let runProcess = this.electron.childProcess.spawn(cmd, {
            detached: false,
            windowsHide: true,
            cwd: cwd,
            shell: this.getShell()
          });

          const stdOutInterface = createInterface(runProcess.stdout);
          const stdErrInterface = createInterface(runProcess.stderr);

          stdOutInterface.on('line', (line) => {
            // get this line back to the component
          });

          stdErrInterface.on('line', (line) => {
            // get this line back to the component
          });

          runProcess.on("error", (err) => {
            // get this back to the component
          });

          runProcess.on('close', (code: number) => {
            // get this line back to the component
          });

        }, 1000);

where getShell is defined as:

private getShell() {
  return process.env[this.electron.os.platform() === "win32" ? "COMSPEC" : "SHELL"];
}

However, whenever I try to run this, it comes back with:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
    $ conda init <SHELL_NAME>
blah blah blah ...

When I try with source activate ${CONDA_ENV_NAME}, it comes back with:

/bin/bash: activate: No such file or directory

Not really sure what I am doing wrong here. Can somebody please point me in the right direction?

PS: It works with source $(conda info --root)/etc/profile.d/conda.sh, but I can't really use it since I need to support Windows as well!

Disclaimer: I am new to python/anaconda.

Binaek Sarkar
  • 617
  • 8
  • 21

3 Answers3

4

I'm not sure what needs to get run in Windows Powershell, but for bash, you need to run the script that conda init configures bash to run at startup, rather than conda init. That is,

miniconda3/etc/profile.d/conda.sh

so it should be something like

execCmd.push(
    `. ${CONDA_ROOT}/etc/profile.d/conda.sh`,
    `conda activate ${CONDA_ENV_NAME}`,
    // long python spawn command
     `conda deactivate`,
)

I suspect the Windows case is running one of the .bat or .ps1 files in the condabin directory.

Alternatively, if conda is defined and you have a Python script (e.g., script.py), then you might be able to get away with using conda run, e.g.,

execCmd.push(
    `conda run -n ${CONDA_ENV_NAME} python script.py`
)

and that could potentially work cross-platform. Please note that conda run only recently added support for interactive I/O and it must be enabled with the --live-stream flag (see v4.9.0 Release Notes). Otherwise, it simply buffers everything hitting stdout/stderr and doesn't return it until the process exits.

merv
  • 67,214
  • 13
  • 180
  • 245
  • As I mentioned in my question, it seems to be working when I fire up `source $(conda info --root)/etc/profile.d/conda.sh` before running the `conda` commands. However, since the audience needs Windows support, that particular command won't cut it. `conda run` cannot be used, since as you mentioned that it buffers up everything, and as you can see from the code, this program runs for a really long time (around 15 minutes) and spits out a lot of logs which are relevant. Need to show the logs as they come. – Binaek Sarkar Dec 15 '19 at 07:55
  • @BinaekSarkar FYI, I updated the answer to point out that as of Conda v4.9.0, `conda run` now supports realtime I/O. – merv Oct 30 '20 at 20:20
  • 1
    I have moved away from this. We ultimately solved it by creating an executable out of the code and ran it without having to depend upon the environment. However, having said that, I do remember that using `conda run` worked across platforms - with the only hitch that it didn't have realtime IO. – Binaek Sarkar Nov 02 '20 at 08:15
2

The main problem is that your shell (CMD in this case) is not configured to handle conda. You have to add it to your system path by providing the Miniconda/Anaconda to the PATH enviroment variable.

Check this StackOverflow Question to know how to do it.

KingDarBoja
  • 1,033
  • 6
  • 12
  • That may not be the case. I say this because, if you notice, in the line before `conda activate`, I do issue a `conda init` which runs successfully. It's failing in `activate`. If PATH were the issue, then `init` wouldn't have run in the first place. – Binaek Sarkar Dec 14 '19 at 20:11
  • Have you tried running the conda init command on powershell then `conda --version` to checkout if conda is successfully initialized on it? The answer below is pointing out such behaviour as conda init couldn't set the proper configuration to run conda commands on powershell. – KingDarBoja Dec 15 '19 at 17:02
  • Disclaimer: I am developing on Linux. I have tried running the same command set on the shell (Terminal) and it's working! – Binaek Sarkar Dec 16 '19 at 07:05
0

Update:

I was referring the & for Windows platform and I also run conda by absolute path to conda.bat instead of call global installed conda. So it is easy for user to install it:

const execPath = path.dirname(process.execPath)
// silent install Miniconda3 to this path (electron-forge resources)
const condaPath = [execPath, "resources", "Miniconda3"].join(path.sep)
const conda = [condaPath, "condabin", "conda.bat"].join(path.sep)
cmd = `${condaPath} activate ${venvPath} & python ${filename} ${arg1} ${arg2}`

I was looking for this too and I got this working just by & as mentioned here

I have the same setup using electron and conda for UI talked to python back. This is in my node:

spawn(
  `conda activate ${name} & python ${filename} ${arg1} ${arg2}`,
  { shell: true }
);

I got it working to stream stdin/out too. Make sure the shell is true.

CallMeLaNN
  • 8,328
  • 7
  • 59
  • 74
  • The meaning of `&` vs `&&` are different in the `bash` context. `&` would essentially just run the `conda activate ${name}` *in the background* and proceed onto `python`, without any heed to whether the `activate` actually worked or not. This answer has a good explanation: https://stackoverflow.com/a/26770612/951243 In `bash`, Try `$ false && echo "something"`. This will return (or show) nothing. If you try `$ false & echo "something"`, you will get the `pid` of `false` and the output of `echo`. – Binaek Sarkar Jul 02 '20 at 06:31
  • While I notice the deferences (plus between other platforms), the concern is to solve this on Windows as your OP. So this is mainly for Windows. `&` means run after another similar to `;` in some unix shells. I tried and it works as I mentioned above. – CallMeLaNN Jul 02 '20 at 06:56
  • BTW, regarding `init` error. There is a difference between `conda init` and `${condaPath}\condabin\conda.bat activate ${condaPath}` before we can activate our environment. The earlier will install conda (add to %PATH% too) for the first time while the later is just to activate "base" so you will have "conda" cmd for current session. The later is useful to activate conda by path without messing or asking user to install it first. – CallMeLaNN Jul 02 '20 at 07:26