3

I am looking at the option of embedding node.js into python to add node.js functionality to my existing python code. I know that it can be done the other way around, as described in this post. However, I want to keep the as much of the existing Python project intact as possible, which means allowing Python to manage execution.

PyV8 does just about everything I want except provide a node.js-like environment that would allow me to use node.js modules in PyV8, so it seems like a good starting point.

Does node.js provide an external API similar to that of V8 such that one could modify PyV8 to wrap node.js? If not, is there a way to load the node.js environment into PyV8 so I can use node.js modules?

Thanks!

Community
  • 1
  • 1
Toaster
  • 1,911
  • 2
  • 23
  • 43
  • 4
    what do you want to do that you can't just have a node service running side by side with your python script or just node by itself? – bitoiu Sep 17 '14 at 21:30
  • 1
    Instead of asking specifically about Python, you should first ask whether Node can be embedded into a C (or C++) application. If that's not possible, embedding it in Python (at least CPython, which is what you probably want) will not be possible; if it is, embedding it in Python will at worst be a matter of wrapping the C embedding in a Python C extension module or calling it via `ctypes` or `cffi` – abarnert Sep 17 '14 at 21:56
  • 1
    Meanwhile, last I checked (which was a while ago…), Node (unlike CPython… or, for that matter, V8) had no official embedding support, and no plans to add any, but there were various third-party wrappers, like [`tacnode`](https://github.com/ZECTBynmo/tacnode), that you can research. – abarnert Sep 17 '14 at 21:57
  • @bitoiu Good question. You may be right, a service could be the solution. However, given the choice I would prefer a tight integration with node.js for manageability and speed. wrt Speed, Node.js will almost certainly be processor bound and spinning up a private instance will help a bit with that. wrt Manageability, I would rather not add an additional interface to the system. – Toaster Sep 17 '14 at 21:57
  • 1
    it's an overhead but on the system design, contrary to the overhead you might be creating on maintaining a feature (embedding) which no one else will understand and that could give you further problems in the future. – bitoiu Sep 17 '14 at 22:02
  • @abarnert Thank you. I will look into tacnode. Quite right, asking if node.js can be embedded in Python is an indirect way of asking if there is a C API. – Toaster Sep 17 '14 at 22:03
  • Don't just look into `tacnode`; I found it by doing a web search and a forum search, and it was just one of the examples that came up. You want to do your own search and/or actually ask on the forums so you can find the best solution, not just one that I randomly found. – abarnert Sep 17 '14 at 22:04
  • Also, @bitoiu is right; the manageability overhead of, e.g., learning and using a trivial RPC system is nothing compared to the manageability overhead of learning and using an embedding API, especially once that's not official and not at 1.0 and probably only being used by a dozen people… – abarnert Sep 17 '14 at 22:06
  • @bitoiu I agree. I am weighing one option against the other. It depends on how much code we are talking about. My first impression was that the PyV8 code was reasonable, so PyNodeJs could be OK if it is the same order of magnitude. – Toaster Sep 17 '14 at 22:07

2 Answers2

4

What you want to do is not supported. Unlike, say, the CPython interpreter, or even the V8 JavaScript interpreter, Node.js was not designed to be embedded, has no interface for doing so, and no serious plan to change that.

I can't find any official documentation on that, but there are numerous threads like this one that give the relevant information.

But that doesn't mean it's impossible. The top layer of Node isn't that complicated, and really you'd just need to patch out a few parts of it to do different things. And, in fact, people have tried to do exactly that, in projects like tacnode. I don't know if any of them are ready for prime time or not, but it may be worth looking at them, especially if you're willing and able to contribute if they're incomplete.

Of course that only gets you embedding in C or C++; you still need to embed in Python. But wrapping a C shared library so you can use it in Python (well, CPython and PyPy) is a long-solved problem; Python has had extension modules since almost the beginning, as well as ctypes and cffi if you don't want to write any C code. And there are third-party projects like Cython to let you write almost-Python code that directly calls your shared library as if it were native C, and compiles to a Python extension module.

So, it's almost certainly doable, but it's probably not going to be trivial, or packaged and ready to go out of the box.

abarnert
  • 354,177
  • 51
  • 601
  • 671
2

Don't embed. Instead, have python and Node in two different processes and communicate between them. RabbitMQ as an example has clients for both Node and Python.

Michel Müller
  • 21
  • 1