Is it possible to embed python without the standard library?

Question

I'm working with a cmake build for python 2.7.6 and I've got a basic embedded script running, like so:

#include <stdio.h>
#include <Python.h>

int main(int argc, char *argv[]) {

  /* Setup */
  Py_SetProgramName(argv[0]);
  Py_Initialize();

  /* Run the 'main' module */
  int rtn = Py_Main(argc, _argv);
  Py_Finalize();

  return rtn;
}

..but when I run it, I get:

ImportError: No module named site

If I setup a correct $PYTHONHOME, it works fine; but that's not what I'm trying to do. I'm trying to embed a copy of python in a stand alone application without the standard library.

I appreciate the use of it, but for this specific embedded environment I want something more like lua, (but with python syntax obviously), where only the specific libraries exposed by the parent application are available.

This has the added bonus of not caring about distributing (or building) the standard library with all it's cross linked dynamic libraries.

Is this possible as all? Or am I inevitably going to stumble into missing fundamental blocks of the language like sys.path, import, {}, [] or similar that are part of the standard library?

If it is possible, how would you go about doing it?

Why not provide your own minimal 'standard library'? Include just `site.py`, anything `site.py` needs to run, and anything your application requires? — Martijn Pieters, Jan 06 '14 at 14:09
But, yes, the CPython implementation does require certain modules to be present and importable. In Python 3, that includes the import machinery. `sys` is a built-in module, syntax is never imported but part of the language so `{}` and `[]` are not going to break. — Martijn Pieters, Jan 06 '14 at 14:19
I was thinking of this, but it's a little annoying to ship a binary with one .py file; I was thinking perhaps of hacking the import mechanism like kivy does for its kivy-ios project, and embedding the minimal required modules is raw text blobs, but I'm not exactly sure how to proceed down that path~ — Doug, Jan 06 '14 at 14:26
You can include the python files as a `.zip` file if you prefer. — Martijn Pieters, Jan 06 '14 at 14:27

Doug · Accepted Answer · 2014-01-10T05:33:56.340

11

Simple answer is yes you can.

int main(int argc, char *argv[]) {

  /* Setup */
  Py_NoSiteFlag = 1; // <--- This
  Py_SetProgramName(argv[0]);
  Py_Initialize();

  /* Run the 'main' module */
  int rtn = Py_Main(argc, argv);
  Py_Finalize();

  return rtn;
}

As far as I can tell, nothing breaks and you can continue to use everything (including the ability to import modules~) perfectly fine. If you try to import anything from the standard library that isn't bundled, you'll get:

>>> import os;
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named os

I'll leave a copy of the code up here for anyone who's interested;

https://github.com/shadowmint/cmake-python-embed-nostdlib

edited Jan 10 '14 at 05:33

answered Jan 07 '14 at 08:53

Doug

32,844
38
166
222

1

Importing `site.py` should be avoidable without modifying the interpreter, I think. The main executable avoids it when given the `-S` flag. You'd have to check whether the relevant flag (`Py_NoSiteFlag`) is internal or private though. Alternatively, add a dummy `site` module to your embedded build. Both options seem better than maintaining a separate patch, even if it's so simple and unlikely to break. – Jan 09 '14 at 19:43
@delnan you are completely correct! thanks! I've update my answer to reflect using the Py_NoSiteFlag, that's definitely a better solution. – Doug Jan 10 '14 at 05:35
1

This doesn't seem to be sufficient for Python3: http://stackoverflow.com/questions/34724057/embed-python3-without-standard-library – P i Jan 11 '16 at 14:36

score 1 · Answer 2 · answered Jan 06 '14 at 14:28

AFAIK, you can't. Without the standard library, Python won't even start, since it tries to find os.py (in 3.x; string.py in 2.x). On startup, it imports a number of modules, in particular site.py. You need at least pythonxy.dll, and probably the contents of the lib folder, plus the extension modules, i.e. the contents of the DLLs folder.

You can ship it with a file pythonXX.zip (where XX is the version number like 27 or 33) which should be a zipped copy of the standard library. You are free to strip the library from stuff that you don't need; there are various tools that compute dependencies statically (see modulefinder).

I guess that is why Lua is more popular as an embedded scripting language.

Is it possible to embed python without the standard library?

2 Answers2

Linked