19

I'm writing software that allows one to publish mathematical books as websites. It is based mostly on Python + Flask, but to deal with equations I'm using MathJax. MathJax can be used either client-side or server-side (through MathJax-node). In the latter case I have to use npm to install MathJax-node in some place accessible to my main Python script, then invoke it from the script. In the former case, I have to provide MathJax.js as an asset, available to client (currently I use Flask's send_from_directory function).

My question is: what is the best practice of dealing with such heterogenous dependencies in Python? My goal is to make installation process as simple as possible at least on unix-like systems (Linux or MacOS), provided that node and npm are already available.

I can just put all the javascript sources I need into my distribution itself, but maybe there's a better way to do it?

Ilya V. Schurov
  • 7,687
  • 2
  • 40
  • 78
  • softwareengineering.stackexchange.com might be a better fit for this question – WhiteHotLoveTiger May 25 '17 at 17:37
  • @WhiteHotLoveTiger when referring other sites, it is often helpful to point that [cross-posting is frowned upon](https://meta.stackexchange.com/tags/cross-posting/info) – gnat May 25 '17 at 18:09
  • @gnat Good to know. I was not aware of that. Would migrating be considered appropriate in this case? – WhiteHotLoveTiger May 25 '17 at 18:15
  • @WhiteHotLoveTiger, thanks, I didn't know about that site. How can I migrate my question there? It seems that *close → offtopic → this question belongs to another SE site" gives me some sites but not *softwareengineering* as possible migration targets. – Ilya V. Schurov May 25 '17 at 19:29
  • I have created and released the [calmjs](https://github.com/calmjs/calmjs) project for this specific use case, provided that `node` and `npm` are available, the integration packages (such as [`calmjs.webpack`](https://github.com/calmjs/calmjs.webpack)) should work. At the very least, with calmjs it is possible to declare a `package.json` that is persisted as part of a given Python package as its metadata for reuse by their dependants. – metatoaster Apr 26 '18 at 14:10

3 Answers3

7

My question is: what is the best practice of dealing with such heterogenous dependencies in Python?

In the case of Node dependencies, I would include a package.json file in the directory which specifies the Node dependencies needed. For other languages/package managers, I would also use whatever the conventional way of specifying dependencies is (e.g. add a Gemfile for Ruby dependencies).

Another common example of this that comes up with Python/Flask is using the Bower package manager for static frontend dependencies. In that case, the dependencies are specified in the bower.json file and are usually pulled into a bower folder in Flask's static directory.

I can just put all the javascript sources I need into my distribution itself, but maybe there's a better way to do it?

Once you've got the package.json with the dependencies specified, you can fetch and install all the Node dependencies needed by running npm install which, in my opinion, is a more elegant solution than including the javascript sources with the project.

Now that you've got multiple package managers (e.g. maybe you're using pip for the Python dependencies in addition to npm for the Node dependencies), you might want to make a Makefile or some deployment/build script to fetch/install using all of them (for example, if I were using Travis CI, I would update my .travis.yml to call npm install in addition to pip install -r).

Christopher Su
  • 373
  • 3
  • 10
  • Thanks! A couple of questions: 1. If I'm using `npm install` to install necessary node packages (like MathJax-node), how it is possible to make Python script aware of the location of installed packages? 2. Am I right that it is impossible to deal with this node dependencies if I want my package to be installable with only `pip install `? (I.e. it is impossible to invoke `npm install` from `setup.py`?) – Ilya V. Schurov May 25 '17 at 17:08
3

Using Node.js package.json would be the most optimal solution for dealing with JavaScript dependencies. As for executing executables from .py you can reference to this answer Running shell command from Python and capturing the output. Node dependencies are by default inside ./node_modules in the same directory as the location of your package.json file.

For installing new dependencies:

npm install --save npm-package-you-want-to-install

Once you have them prepared this command will have everything installed for you:

npm install

Node dependencies are definitely more elegant way of dealing with things since javascript is a constantly evolving world and it is much easier to take a look at a package.json than a lot of script tags / functions that simply invoke said scripts. If you want a automated system my suggestion would be to make a executable (.sh) which will run installment for both and you can use that in your future projects.

kochai
  • 49
  • 4
  • Thanks! A couple of questions: 1. If I'm using `npm install` to install necessary node packages (like MathJax-node), how it is possible to make Python script aware of the location of installed packages? 2. Am I right that it is impossible to deal with this node dependencies if I want my package to be installable with only `pip install `? (I.e. it is impossible to invoke `npm install` from `setup.py`?) – Ilya V. Schurov Jun 12 '17 at 23:38
  • node_modules are usually in the same folder as package.json. So in the case package.json is in root of your project, node_modules would be as well. For py scripts (npm install) there's a link in the comment above. – kochai Jun 12 '17 at 23:42
  • Yeah, I understand how to invoke `npm install` from python. What I am asking for is the following. `setup.py` is basically an invokation of `setuptools.setup` function that will automatically find the correct place to install package. However, I cannot execute (at least don't know how to) execute arbitrary python commands (like `subprocess.run(["npm", "install"])` in the directory where my scripts are putted by `setuptools.setup`. This is why I cannot understand how my Python scripts (after they are installed) can be aware of the location of JavaScript dependancies. – Ilya V. Schurov Jun 13 '17 at 11:57
1

I recommend to use webpack Webpack.js not Bowerjs. NPM and his package.json are very good for dependency updates but referencing libraries from node_modules is a little embarrasing.

Alex Alvarez Gárciga
  • 424
  • 5
  • 15