ImportError when calling python script from .net with Python 3.7

Question

I am trying to call a python script from a .net program. When I call the script directly from my console, everything works fine, but when I call it from a .net program it crashes with the following exception.

Traceback (most recent call last):
  File "run.py", line 1, in <module>
    import pika
  File "C:\Users\USERNAME\AppData\Local\Continuum\anaconda3\envs\py37\lib\site-packages\pika\__init__.py", line 10, in <module>
    from pika.connection import ConnectionParameters
  File "C:\Users\USERNAME\AppData\Local\Continuum\anaconda3\envs\py37\lib\site-packages\pika\connection.py", line 13, in <module>
    import ssl
  File "C:\Users\USERNAME\AppData\Local\Continuum\anaconda3\envs\py37\lib\ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: DLL load failed: The specified module could not be found.

The script I'm using is pretty long, but the bug is reproducible with something as simple as

import pika

(I'm using pika 1.1.0, but pika isn't the only package that fails, a lot do).

This isn't the first time I run into this bug. I ran into it with Pycharm before, and it seems to be a common issue Python 3.7 anaconda environment - import _ssl DLL load fail error . This only happens with Python 3.7, but I sadly can't go back to 3.6 or lower. The fix in Pycharm is to append the environment variables (more specifically, the PATH variable) explicitely. I tried doing the same in .net but the bug remained.

The .net script looks something like this

using System.Diagnostics;
using System.IO;
using NLog;
using System;

namespace ConsoleApp1
{
    class Program
    {
        static Logger logger = LogManager.GetCurrentClassLogger();

        static void Main(string[] args)
        {
            string cmd = "run.py";
            ProcessStartInfo start = new ProcessStartInfo();
            start.FileName = "PATH_TO_PYTHON_3.7_ENV.exe";
            start.Arguments = string.Format("\"{0}\"", cmd);
            start.UseShellExecute = false;
            start.CreateNoWindow = true;
            start.RedirectStandardOutput = true;
            start.RedirectStandardError = true; 
            using (Process process = Process.Start(start))
            {
                using (StreamReader reader = process.StandardOutput)
                {
                    string stderr = process.StandardError.ReadToEnd();
                    string result = reader.ReadToEnd();
                    Console.WriteLine(stderr);
                    Console.ReadKey();
                }
            }
        }
    }
}

Has anyone run into this problem before and found a way to fix it?

If you haven't already, try with upgrading to the latest Python version 3.7.5. — Darkonaut, Dec 05 '19 at 19:55
I did try that, however some other libraries I'm using are not compatible with 3.7.5 yet. So, this isn't an option for me... Or not until I become too desperate, at least — Skum, Dec 06 '19 at 12:42
AFAIK they changed the venv-script with 3.7.2, so if you can't find a better way, going back to 3.7.1 might be an option for now. — Darkonaut, Dec 06 '19 at 13:34
im not so sure but maybe your `openssl`package is corrupt or you dont have it. try installing this first and when you compile python you have to link it to the path where you install `openssl` package. — Siddhant Tandon, Dec 06 '19 at 23:59

amanb · Accepted Answer · 2019-12-07T21:21:52.600

In my attempt to reproduce the issue, I found that everything works when Python 3.6 is used as the env while it fails with the error for Python 3.7.4. A similar behavior is also observed in Pycharm(as you pointed out). While the underlying problem could be the way DLL search happens for Python 3.7.4 and the root cause will require further investigation, I noticed that this error in general could appear when Python is invoked in a "non-conda" cmd environment. If you invoke Python in a normal Windows cmd prompt, the following error should display(provided that "path_to_python.exe" is in PATH and pika is installed system-wide):

C:\Users\xxxxxx>python
Python 3.7.4 (default, Aug  9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32

Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated.  Libraries may fail to load.  To activate this environment
please see https://conda.io/activation

Type "help", "copyright", "credits" or "license" for more information.
>>> import pika
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\xxxxxx\Anaconda3\envs\py3.7.4\lib\site-packages\pika\__init__.py", line 10, in <module>
    from pika.connection import ConnectionParameters
  File "C:\Users\xxxxxx\Anaconda3\envs\py3.7.4\lib\site-packages\pika\connection.py", line 13, in <module>
    import ssl
  File "C:\Users\xxxxxx\Anaconda3\envs\py3.7.4\lib\ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: DLL load failed: The specified module could not be found.

So, when you invoke the Python script through C#, a similar behaviour is observed for Python 3.7.4 where the DLL is not loaded because the conda environment is not activated and the process is running in a normal cmd env.

Therefore, activating the conda env prior to running the script should resolve this issue in general for all versions of Python.

I used the following simple Python script to test:

import pika
print("Pika imported sucessfully!")

Credits to this answer, the above script ran successfully, displaying the message on the console when I activated the conda env first:

using System;
using System.Diagnostics;
using System.Threading;


namespace PyTest
{
    class Program
    {

        static void Main(string[] args)
        {
            string workingDirectory = @"C:\Test";
            var process = new Process
            {                
                StartInfo = new ProcessStartInfo
                {
                    FileName = "cmd.exe",
                    RedirectStandardInput = true,
                    UseShellExecute = false,
                    RedirectStandardOutput = true,
                    WorkingDirectory = workingDirectory
                }

            };
            process.Start();


            using (var sw = process.StandardInput)
            {
                if (sw.BaseStream.CanWrite)
                {
                    // Batch script to activate Anaconda
                    sw.WriteLine(@"C:\Users\xxxxx\Anaconda3\Scripts\activate.bat");
                    // Activate your environment
                    sw.WriteLine("conda activate py3.7.4");                   
                    // run your script. You can also pass in arguments
                    sw.WriteLine("py pika_test.py");
                }
            }
            // read multiple output lines
            while (!process.StandardOutput.EndOfStream)
            {
                var line = process.StandardOutput.ReadLine();
                Console.WriteLine(line);
                Thread.Sleep(500);
            }

        }
    }
}

Output

Pika imported sucessfully!

To list all conda envs, execute the following command from the Anaconda command line:

(base) C:\Users\xxxxx>conda env list
# conda environments:
#
base                  *  C:\Users\xxxxx\Anaconda3
py27                     C:\Users\xxxxx\Anaconda3\envs\py27
py3.5                    C:\Users\xxxxx\Anaconda3\envs\py3.5
py3.6                    C:\Users\xxxxx\Anaconda3\envs\py3.6
py3.7.4                  C:\Users\xxxxx\Anaconda3\envs\py3.7.4

ImportError when calling python script from .net with Python 3.7

1 Answers1