How to integrate a standalone Python script into a Rails application?

Question

I've got a program that has a small file structure going on and is then ran using

python do_work.py foo bar

I want my Rails users to press a button and have this happen for them, with the result either uploaded somewhere or just thrown to them as a download link or something of the sort - the output of do_work.py (say, it's result.txt)

I also want to clarify that the script results in the creation on the filesystem of 3 separate files, which are not text files (which shouldn't matter and isn't really the problem here)

What is the best way to go about it? Can rake run exec Python? More importantly, is this doable on heroku?

I have Python installed on my system but the provided answer by sockmonk doesn't seem to work - it returns nil. Mind you, other commands like ls seem to work.

Could it be a permissions problem?

def index
    value = %x( python --version )
    render :text => value
end

Incidentally, trying this in irb:

%x(python)

Brings up the Python terminal INSIDE of irb. It will not take params for whatever reason however.

I think this is more of a Ruby thing than Rails. There seems to be several ways to achieve this, find them here - http://stackoverflow.com/questions/2232/calling-bash-commands-from-ruby — Bibhas Debnath, May 05 '13 at 06:08

score 27 · Accepted Answer · answered May 05 '13 at 04:33

Your index method does not work because python --version outputs its version to STDERR, not STDOUT. If you don't need to separate these streams, you may just redirect STDERR to STDOUT:

value = %x(python --version 2>&1)

This call is synchronous, so after running the script (python do_work.py foo bar 2>&1), you should be able to read the files produced by it.

If the script is not able to create the files for some reason, you will now see the exception in the value variable because error messages are usually sent to STDERR.

If you want to separate STDERR from STDOUT, use the Open3 module.

Beware that the script takes some time to run, so the calls may overlap. I would use a queue here to prevent this.

And don't forget to check the data the user enters. Never pass it directly to the script.

can you comment on how to tie this in with the ethereal /tmp/filesystem so that the files are saved there? — dsp_099, May 05 '13 at 07:36
Do you mean the *ephemeral writeable filesystem*? See this question: [How to use Heroku's ephemeral filesystem?](http://stackoverflow.com/questions/12416738/how-to-use-herokus-ephemeral-filesystem). You just need to process the files during the request as they may be deleted after the request is processed. — utapyngo, May 05 '13 at 13:05

score 4 · Answer 2 · answered Apr 30 '13 at 18:01

4

It partly depends on the format of the data. If it's not too long and can be rendered directly in the browser, you can just do something like this in a rails controller:

result = `python do_work.py foo bar`
render :text => result

And assuming that result is plain ASCII text, the result will go straight to their browser. If the params to do_work.py come from the user you MUST validate them first though, so you don't wind up creating a nasty vulnerability for yourself. Using the system() call would probably be safer in that case.

If you want to send the results back as a file, look at ruby's Tempfile class for creating the file (in a way that won't stick around forever), and rails' send_file and send_data commands for some different options to send back the results that way.

answered Apr 30 '13 at 18:01

sockmonk

4,195
24
40

Also I can't to seem to get the example to work on a naked rails app – dsp_099 May 05 '13 at 01:50
Why would `system()` be safer? The backtick is just an alias for `system()`. – Linuxios May 07 '13 at 22:52
The backtick does string interpolation of whatever is in backticks, so if it contained user input, bad input similar to a SQL injection attack could slip in. By using system with multiple args, you're being much more explicit about what the command is, and what the args are and the boundaries between them, so that sort of attack doesn't work as well. On the downside, system() returns the exit status instead of the output like backticks does, so the output would have to be stored somewhere to render back to the user. – sockmonk May 08 '13 at 15:04

score 3 · Answer 3 · answered May 07 '13 at 20:22

The answer from utapyngo is cover almost all you need to know. I'll answer this part:

incidentally, trying this in irb: %x(python) Brings up the python terminal INSIDE of irb. It will not take params for whatever reason however.

To pass parameters to your python script, simply pass it. Example:

[fotanus@thing ~]$ python a.py 
args:
['a.py']
[fotanus@thing ~]$ irb
1.8.7 :001 > %x(python a.py foo bar)
 => "args:\n['a.py', 'foo', 'bar']\n"

This works on ruby 1.8, 1.9 and 2.0.

score 1 · Answer 4 · answered May 10 '13 at 00:44

It depends how deeply you want to integrate the python script. There are ways to actually call python modules directly from ruby.

http://www.goto.info.waseda.ac.jp/~fukusima/ruby/python-e.html

This would give you the benefit of getting the output directly from your python script instead of going over the I/O device.

Josnidhin · Answer 5 · 2013-05-10T02:23:12.533

I would do something like the following.

Asynchronously execute this task in the background. And once the result is ready report it to the user.

One way you can achieve this will by using Open3 and delayed_job gem.

Take a look the popen3 method in Open3 module.

Open3.popen3([env,] cmd... [, opts]) {|stdin, stdout, stderr, wait_thr|
  pid = wait_thr.pid # pid of the started process.
  ...
  exit_status = wait_thr.value # Process::Status object returned.
}

In your case you can change the Open3.popen3 statement to something like the following

Open3.popen3("python do_work.py foo bar"){
  ...
  # mechanism for reporting like setting a flag in database system
  # or queue system
}

Note: you should give the full path to your python script

And then use delayed_job gem to run this as a background task.

You should also have a polling mechanism which will poll the system to see if the flag is set which would mean the result is ready and then serve it to the user.

How to integrate a standalone Python script into a Rails application?

5 Answers5

Linked