How to import Python modules provided by commercial software?

Question

I would like to test Python2 code that depends on objects provided by commercial software Abaqus. I can start the Python interpreter that is shipped with Abaqus via abaqus python:

Python 2.7.3 for Abaqus 2018 (default, Sep 9 2017, 03:28:08)

[GCC 4.9.2 20150212 (Red Hat 4.9.2-6)] on linux2

The problem is that when I try to import the module abaqus provided by Abaqus, I get the error:

abaqus module may only be imported in the Abaqus kernel process

I found the .pyc for the module under /some/path/ and tried to follow the answer described here to execute the compiled code:

import marshal

s = open('/some/path/abaqus.pyc', 'rb')
s.seek(8)  # go past first eight bytes
code_obj = marshal.load(s)
exec(code_obj)

But this yields the same error. Is there any way I can import the module outside the Abaqus kernel process?

UPD 1

I tried launching a subprocess, but to no avail. I created a file my_test_script.py:

import subprocess
subprocess.call( ["pytest", "--version"] )

Running python my_test_script.py produces:

This is pytest version 4.6.11, imported from /path/to/virtual/environment/pytest.pyc

Running the same script through Abaqus results in the following error:

$ abaqus cae nogui=my_test_script.py
Abaqus License Manager checked out the following license:
"cae" from Flexnet server licabaq1.lic
<39 out of 40 licenses remain available>.
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/apps/Common/Core/ABAQUS/2018-linux-x86_64/cae/linux_a64/tools/SMApy/python2.7/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha512
Traceback (most recent call last):
  File "/path/to/virtual/environment/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 65, in main
    config = _prepareconfig(args, plugins)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 214, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/path/to/virtual/environment/lib/python2.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/path/to/virtual/environment/lib/python2.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/helpconfig.py", line 94, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/path/to/virtual/environment/lib/python2.7/site-packages/pluggy/callers.py", line 81, in get_result
    _reraise(*ex)  # noqa
  File "/path/to/virtual/environment/lib/python2.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 789, in pytest_cmdline_parse
    self.parse(args)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 997, in parse
    self._preparse(args, addopts=addopts)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 938, in _preparse
    self._consider_importhook(args)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 886, in _consider_importhook
    self._mark_plugins_for_rewrite(hook)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 908, in _mark_plugins_for_rewrite
    for name in _iter_rewritable_modules(package_files):
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 662, in _iter_rewritable_modules
    package_files = list(package_files)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/_pytest/config/__init__.py", line 904, in <genexpr>
    if any(ep.group == "pytest11" for ep in dist.entry_points)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/importlib_metadata/__init__.py", line 289, in entry_points
    return EntryPoint._from_text(self.read_text('entry_points.txt'))
  File "/path/to/virtual/environment/lib/python2.7/site-packages/importlib_metadata/__init__.py", line 138, in _from_text
    config.read_string(text)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/backports/configparser/__init__.py", line 794, in read_string
    self.read_file(sfile, source)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/backports/configparser/__init__.py", line 789, in read_file
    self._read(f, source)
  File "/path/to/virtual/environment/lib/python2.7/site-packages/backports/configparser/__init__.py", line 1193, in _read
    raise e
backports.configparser.ParsingError: Source contains parsing errors: u'<string>'
    [line  2]: u'\U00660000\U00320000\U00700000\U00790000\U00200000\U003d0000\U00200000\U006e0000\U00750000\U006d0000\U00700000\U00790000\U002e0000\U00660000\U00320000\U00700000\U00790000\U002e0000\U00660000\U00320000\U00700000\U00790000\U00320000\U00650000\U003a0000\U006d0000\U00610000\U00690000\U006e0000\U000a0000\x00\x00\x00\x00w_MapString1DArrayListInt.'
    [line  3]: u'\U00660000\U00320000\U00700000\U00790000\U00320000\U00200000\U003d0000\U00200000\U006e0000\U00770000config\x00c'

The Abaqus replay file does not contain anything helpful for me:

# from driverUtils import executeOnCaeGraphicsStartup
# executeOnCaeGraphicsStartup()
#: Executing "onCaeGraphicsStartup()" in the site directory ...
from abaqus import *
from abaqusConstants import *
session.Viewport(name='Viewport: 1', origin=(1.36719, 1.36719), width=201.25, 
    height=135.625)
session.viewports['Viewport: 1'].makeCurrent()
from driverUtils import executeOnCaeStartup
executeOnCaeStartup()
execfile('my_test_script.py', __main__.__dict__)
print 'RT script done'
#: RT script done

Any takes on the output?

Why don't you use the CAE interpreter? I think Abaqus specifically protected their libraries so they can only be used alongside with the CAE licence — Roman Zh., Dec 24 '20 at 11:18
Do you mean that I run the tests from the interactive session? But then I face the problem that my `PyTest` is installed in the virtual environment. And something like `os.system('ml Python; source venv/bin/activate; python -m pytest unit_tests')` does not work. I work on the cluster, you see, and cannot install packages globally. — honey_badger, Dec 27 '20 at 23:47
If Abaqus is accessible from your virtual environment, then you can launch your test scripts using `subprocess` and the 'abaqus cae nogui=my_test_script.py'. `subprocess` would allow you retrieving log information and consequently use it in unit test I guess — Roman Zh., Dec 28 '20 at 11:35
@RomanZhuravlev it did not work. I updated the post with the error message I received — honey_badger, Jan 08 '21 at 17:29

score 2 · Accepted Answer · answered Jan 11 '21 at 09:07

I am not sure that finally understood all details of your problem. For example, what are you trying to achieve by executing the code frome a '.pyc' file (btw what does it contain?)?

Anyway, here I am sharing the routine which works for me.

Creating a script my_abaqus_script.py

Note: the list of imports can vary depending on your needs

from abaqus import *
from abaqusConstants import *
from part import *
from sketch import *

m = mdb.Model(name='myModel')
a = abq_model.rootAssembly

my_sketch = m.ConstrainedSketch(name='my_sketch', sheetSize=24.0)

pivots = [(0., 0.), (0., 1.), (1., 1.), (1., 0.)]
n = len(pivots)
for i in range(n):
    my_sketch.Line(point1=pivot_pts[i], point2=pivot_pts[(i+1) % n])

my_part = m.Part(dimensionality=THREE_D, name='MyPart', type=DEFORMABLE_BODY)
my_part.BaseSolidExtrude(sketch=my_sketch, depth=1.0)

# ...and so on

Launching the subprocess in your working directory (see the documentation of subprocess.Popen and how to execute abaqus job from the command line at the "Abaqus Analysis User's Guide -> Introduction, Spatial Modeling, and Execution -> Job Execution -> Execution procedures"). The following code you launch in any python interpreter.

import subprocess
my_cmd = 'abaqus job=my_abaqus_script analysis cpus=1 interactive'

proc = subprocess.Popen(
   my_cmd,
   cwd=my_working_dir,
   stdout='my_study.log',
   stderr='my_study.err',
   shell=True
)

If something in the code doesn't work, I am opening the Abaqus CAE and trying to launch the code "by line" in the Abaqus python interpreter (at the bottom of the window). You can investigate different objects there for a better understanding of how Abaqus environment is working.

Thank you for your answer! The `.pyc` file contains Abaqus objects (which are imported via `from abaqus import *` in your example). I need them to use in my unit test routines in PyTest. So my question is how do I marry Abaqus with PyTest? Your example shows how to call Abaqus as a subprocess. Do you imply that I should use the status of the `my_abaqus_script` in the `assert` statement of testing routines? — honey_badger, Jan 13 '21 at 12:35
Well, you could, yes. That this what I am doing: one test for model generation; one test for odb postprocess; etc — Roman Zh., Jan 13 '21 at 18:44
@honey_badger Anything you need from Abaqus has to go through the abaqus.exe process. I don't think you can import Abaqus' libraries from outside, and I don't see why you would want to do that. Furthermore, running unit tests on abaqus scripts (even for complex libraries) is not very productive because: (1) you can't fix API problems; (2) you can generally trust the API to work (except the assembly module, that's really bad); and (3) you can't really isolate the output of your functions because everything is big and time-consuming. — Mohammadreza Khoshbin, Jan 19 '21 at 01:31

How to import Python modules provided by commercial software?

1 Answers1