61

I want to call up an editor in a python script to solicit input from the user, much like crontab e or git commit does.

Here's a snippet from what I have running so far. (In the future, I might use $EDITOR instead of vim so that folks can customize to their liking.)

tmp_file = '/tmp/up.'+''.join(random.choice(string.ascii_uppercase + string.digits) for x in range(6))
edit_call = [ "vim",tmp_file]
edit = subprocess.Popen(edit_call,stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True )   

My problem is that by using Popen, it seems to keep my i/o with the python script from going into the running copy of vim, and I can't find a way to just pass the i/o through to vim. I get the following error.

Vim: Warning: Output is not to a terminal
Vim: Warning: Input is not from a terminal

What's the best way to call a CLI program from python, hand control over to it, and then pass it back once you're finished with it?

Nikolai K.
  • 1,221
  • 1
  • 10
  • 14
sam
  • 763
  • 1
  • 5
  • 7

6 Answers6

100

Calling up $EDITOR is easy. I've written this kind of code to call up editor:

import sys, tempfile, os
from subprocess import call

EDITOR = os.environ.get('EDITOR', 'vim')  # that easy!

initial_message = ''  # if you want to set up the file somehow

with tempfile.NamedTemporaryFile(suffix=".tmp") as tf:
  tf.write(initial_message)
  tf.flush()
  call([EDITOR, tf.name])

  # do the parsing with `tf` using regular File operations.
  # for instance:
  tf.seek(0)
  edited_message = tf.read()

The good thing here is the libraries handle creating and removing the temporary file.

Michael
  • 8,362
  • 6
  • 61
  • 88
mike3996
  • 17,047
  • 9
  • 64
  • 80
  • Brilliant! My one modification is to add a fallback in case EDITOR is not set: `EDITOR = os.environ.get('EDITOR') if os.environ.get('EDITOR') else 'vim'`. I've submitted this as an edit to your, if you wish to accept it. – sam Jun 10 '11 at 19:25
  • Thanks @sam and @unutbu for suggestions. I didn't know you could get away with an unset `$EDITOR` :) – mike3996 Jun 10 '11 at 20:40
  • +1, even though for some reason EDITOR is not set on my Ubuntu. Strange(r) `$ editor` launchs Nano where as `crontab -e` launches vim... – FMCorz Apr 09 '14 at 12:34
  • @Roun: your suggested edit is pretty good. Shame it got shot down. – mike3996 Oct 02 '15 at 09:43
  • @chishaku: the file is already open, no need to reopen a handle to it. Especially no need to leave the handle open and hanging :) – mike3996 Mar 17 '16 at 09:10
  • 12
    I had to re-open the file with `with open(tf.name)` to get at the updated file contents. Otherwise, I was just getting the same contents as `initial_message` – Paul May 16 '16 at 05:52
  • @Paul: interesting. What system, editor you used? It is probably due to the fact that editors (vim included I think) can sometimes write to an entirely new file that is just renamed afterwards over the old name. It would cause funniness to the open file handles... – mike3996 May 16 '16 at 05:55
  • 1
    @progo Mac OSX El Capitan, VIM. Interesting – Paul May 16 '16 at 20:05
  • Same result as @Paul, I have to reopen the file (and tell it not to be deleted). Linux 64bit. – astrojuanlu Oct 27 '16 at 16:06
  • 5
    I ran into the same problem of reading back the old contents of the file, and its true that in many cases, vim moves the old file aside and writes the new file, leaving our tf file descriptor still open on the now-backup file. The command line option "+set backupcopy=yes" added before the file name prevents this problem, and I'm now happily able to read the new contents of the file. call([EDITOR, '+set backupcopy=yes', tf.name]) – Chuck Buche Nov 22 '16 at 06:56
  • @ChuckBuche: I see. I haven't set `backupcopy` anyhow in my vimrc and it's set `auto`. Never experienced the problem, maybe my distro has patched this issue away or something. The argument passing is not very viable because it only applies to vim. The poor sods who use nano as their editors are in for a surprise! – mike3996 Nov 22 '16 at 07:26
  • On Mac 10.13.6 I get this error `IOError: [Errno 2] No such file or directory: '/var/folders/67/sh_h1jd16jd8jt79rgcyryy80000gn/T/tmplW22o_.tmp'` even when I implement `'+set backupcopy=yes'` and `with open(tf.name):` – Thisisstackoverflow Jul 12 '18 at 21:47
  • It's probably old but a modification I made was to remove the `tf.seek(0)`. So now when I enter vim it shows my initial message but once I exit, it prints out only what I wrote without the initial message. – dearprudence Sep 28 '20 at 05:23
  • @dearprudence you do what your code or problem demands. I am not sure how the file will be read if the user removes some of the pre-message so it's safest to read and parse the file from the beginning. – mike3996 Sep 28 '20 at 09:45
  • @nperson325681 your answer helped me a ton nonetheless. Thank you dear stranger. – dearprudence Sep 29 '20 at 03:02
10

In python3: 'str' does not support the buffer interface

$ python3 editor.py
Traceback (most recent call last):
  File "editor.py", line 9, in <module>
    tf.write(initial_message)
  File "/usr/lib/python3.4/tempfile.py", line 399, in func_wrapper
    return func(*args, **kwargs)
TypeError: 'str' does not support the buffer interface

For python3, use initial_message = b"" to declare the buffered string.

Then use edited_message.decode("utf-8") to decode the buffer into a string.

import sys, tempfile, os
from subprocess import call

EDITOR = os.environ.get('EDITOR','vim') #that easy!

initial_message = b"" # if you want to set up the file somehow

with tempfile.NamedTemporaryFile(suffix=".tmp") as tf:
    tf.write(initial_message)
    tf.flush()
    call([EDITOR, tf.name])

    # do the parsing with `tf` using regular File operations.
    # for instance:
    tf.seek(0)
    edited_message = tf.read()
    print (edited_message.decode("utf-8"))

Result:

$ python3 editor.py
look a string
modle13
  • 1,242
  • 2
  • 16
  • 16
  • 1
    receiving blank chars back. Trying instead `print(edited_message)` results in `b''` returned. This is with Python 3.5.2 via OS X Terminal – ljs.dev Dec 08 '16 at 17:00
8

Package python-editor:

$ pip install python-editor
$ python
>>> import editor
>>> result = editor.edit(contents="text to put in editor\n")

More details here: https://github.com/fmoo/python-editor

Mathieu Longtin
  • 15,922
  • 6
  • 30
  • 40
4

click is a great library for command line processing and it has some utilities, click.edit() is portable and uses the EDITOR environment variable. I typed the line, stuff, into the editor. Notice it is returned as a string. Nice.

(venv) /tmp/editor $ export EDITOR='=mvim -f'
(venv) /tmp/editor $ python
>>> import click
>>> click.edit()
'stuff\n'

Check out the docs https://click.palletsprojects.com/en/7.x/utils/#launching-editors My entire experience:

/tmp $ mkdir editor
/tmp $ cd editor
/tmp/editor $ python3 -m venv venv
/tmp/editor $ source venv/bin/activate
(venv) /tmp/editor $ pip install click
Collecting click
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Installing collected packages: click
Successfully installed click-7.0
You are using pip version 19.0.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(venv) /tmp/editor $ export EDITOR='=mvim -f'
(venv) /tmp/editor $ python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import click
>>> click.edit()
'stuff\n'
>>>
Powell Quiring
  • 2,349
  • 2
  • 11
  • 9
3

The PIPE is the problem. VIM is an application that depends on the fact that the stdin/stdout channels are terminals and not files or pipes. Removing the stdin/stdout paramters worked for me.

I would avoid using os.system as it should be replaced by the subprocess module.

dmeister
  • 34,704
  • 19
  • 73
  • 95
  • Thanks dmeister. However it doesn't work for me. I have the following code. Is this what you meant? `edit_call = [ "vim",tmp_file]; edit = subprocess.Popen(edit_call)` – sam Jun 10 '11 at 17:05
  • @sam Yes, that is what I meant – dmeister Jun 10 '11 at 17:08
  • 1
    After running that code, it brings up vim, but I can't interact with it. After typing a couple characters, vim disappears, and I'm left with `Vim: Error reading input, exiting...` and then `Vim: Finished.`. This leaves me still inside a process. After typing a space or enter (no prompt, yet), I get the following and then returned to the command-prompt: `-bash: 1: command not found` – sam Jun 10 '11 at 18:25
  • For future users, and maybe for @sam you just need a `.wait()` on the end of it. – TheSchwa Jul 31 '19 at 16:09
2

The accepted answer does not work for me. edited_message stays the same as initial_message. As explained in the comments, this is caused by vim saving strategy.

There are possible workarounds, but they are not portable to other editors. Instead, I strongly recommend to use click.edit function. With it, your code will look like this:

import click

initial_message = "edit me!"
edited_message = click.edit(initial_message)
print(edited_message)

Click is a third-party library, but you probably should use it anyway if you are writing a console script. click to argparse is the same as requests to urllib.

Nikolai K.
  • 1,221
  • 1
  • 10
  • 14