How to write code to autocomplete words and sentences?

Question

I'd like to write code that does autocompletion in the Linux terminal. The code should work as follows.

It has a list of strings (e.g. "hello, "hi", "how are you", "goodbye", "great", ...).

In the terminal, the user will start typing and when there is some match possibility, he gets the hint for possible strings, from which he can choose (similarly as in vim editor or google incremental search).

e.g. he starts typing "h", and he gets the hint

h"ello"

_ "i"

_"ow are you"

And better yet would be if it would complete words not only from the beginning, but from an arbitrary part of the string.

What's the context? Are you running Python application in the terminal which receives input from the user? I don't get it... — Felix Kling, Oct 19 '11 at 13:01

score 58 · Accepted Answer · edited Aug 01 '16 at 20:40

(I'm aware this isn't exactly what you're asking for, but) If you're happy with the auto-completion/suggestions appearing on TAB (as used in many shells), then you can quickly get up and running using the readline module.

Here's a quick example based on Doug Hellmann's PyMOTW writeup on readline.

import readline

class MyCompleter(object):  # Custom completer

    def __init__(self, options):
        self.options = sorted(options)

    def complete(self, text, state):
        if state == 0:  # on first trigger, build possible matches
            if text:  # cache matches (entries that start with entered text)
                self.matches = [s for s in self.options 
                                    if s and s.startswith(text)]
            else:  # no text entered, all matches possible
                self.matches = self.options[:]

        # return match indexed by state
        try: 
            return self.matches[state]
        except IndexError:
            return None

completer = MyCompleter(["hello", "hi", "how are you", "goodbye", "great"])
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')

input = raw_input("Input: ")
print "You entered", input

This results in the following behaviour (<TAB> representing a the tab key being pressed):

Input: <TAB><TAB>
goodbye      great        hello        hi           how are you

Input: h<TAB><TAB>
hello        hi           how are you

Input: ho<TAB>ow are you

In the last line (HOTAB entered), there is only one possible match and the whole sentence "how are you" is auto completed.

Check out the linked articles for more information on readline.

"And better yet would be if it would complete words not only from the beginning ... completion from arbitrary part of the string."

This can be achieved by simply modifying the match criteria in the completer function, ie. from:

self.matches = [s for s in self.options 
                   if s and s.startswith(text)]

to something like:

self.matches = [s for s in self.options 
                   if text in s]

This will give you the following behaviour:

Input: <TAB><TAB>
goodbye      great        hello        hi           how are you

Input: o<TAB><TAB>
goodbye      hello        how are you

Updates: using the history buffer (as mentioned in comments)

A simple way to create a pseudo-menu for scrolling/searching is to load the keywords into the history buffer. You will then be able to scroll through the entries using the up/down arrow keys as well as use Ctrl+R to perform a reverse-search.

To try this out, make the following changes:

keywords = ["hello", "hi", "how are you", "goodbye", "great"]
completer = MyCompleter(keywords)
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')
for kw in keywords:
    readline.add_history(kw)

input = raw_input("Input: ")
print "You entered", input

When you run the script, try typing Ctrl+r followed by a. That will return the first match that contains "a". Enter Ctrl+r again for the next match. To select an entry, press ENTER.

Also try using the UP/DOWN keys to scroll through the keywords.

Thank you, I tested the code. It only has missing some features I'd like to have. `automatic autocomplation while typing, without the need to press tab key` possibility to complete from `arbitrary part of the strings`, possibility of `choosing between matches with rows, or other keyboard shortcuts`. — xralf, Oct 19 '11 at 14:17
I forgot that this feature is in google, so it's possible to try search something with google. — xralf, Oct 19 '11 at 14:24
AFAIK there isn't a canned solution to do what you want. That level of interactivity in a console app would require a lot more work. As @jro commented in the other answer, curses would be a good start. I'm merely pointing out the more common approach to cmd-line-based autocompletion, i.e. using readline. Apologies for not being of more help. Good luck with your app. — Shawn Chin, Oct 19 '11 at 14:32
*"I forgot that this feature is in google"*... huh? What king of application are you trying to write? — Shawn Chin, Oct 19 '11 at 14:34
Just ordinary app to save me some typing. I hoped there are some libraries that could do this easily. Your answer is acceptable, but I will wait for some time yet if it can be improved yet. Thank you — xralf, Oct 19 '11 at 14:44
Command line apps work quite different from GUIs/webpages. For menu-based interfaces as the ones in your linked screenshots, you'll have to write your app as a [TUI](http://en.wikipedia.org/wiki/Text_user_interface) using libraries such as [ncurses](http://en.wikipedia.org/wiki/Ncurses) (used by vim). The [curses](http://docs.python.org/library/curses.html) module provides an interface to the curses library, but you'll quickly find that its a completely different can of worms to your standard cmd-line app. — Shawn Chin, Oct 19 '11 at 15:04
But is `ncurses` neccessary for completion without the tab key? `ncurses` is needed I think for the feature with browsing through the list of suggestions, this is the feature I give up or leave for later. — xralf, Oct 19 '11 at 15:09
That would not require `ncurses`, but I doubt if *completion every keypress* is doable using `readline` (quite possible for good reasons). The closest solution I can think of to *"browsing through the list of suggestions"* is to load the keywords into the `readline` history buffer which allows you to navigate through it using your up/down arrow keys as well as using the `Ctrl-r` search shortcuts. I'll update the answer with an example. — Shawn Chin, Oct 19 '11 at 15:30

score 9 · Answer 2 · answered Sep 23 '16 at 19:50

9

To enable autocomplete in a Python shell, type this:

import rlcompleter, readline
readline.parse_and_bind('tab:complete')

(thanks to http://blog.e-shell.org/221)

answered Sep 23 '16 at 19:50

Nicolas S

91
1
2

score 9 · Answer 3 · answered Jul 01 '20 at 17:39

9

You may want to checkout fast-autocomplete: https://github.com/seperman/fast-autocomplete

It has a demo mode that you can type and get results as you type: https://zepworks.com/posts/you-autocomplete-me/#part-6-demo

It is very easy to use:

>>> from fast_autocomplete import AutoComplete
>>> words = {'book': {}, 'burrito': {}, 'pizza': {}, 'pasta':{}}
>>> autocomplete = AutoComplete(words=words)
>>> autocomplete.search(word='b', max_cost=3, size=3)
[['book'], ['burrito']]
>>> autocomplete.search(word='bu', max_cost=3, size=3)
[['burrito']]
>>> autocomplete.search(word='barrito', max_cost=3, size=3)  # mis-spelling
[['burrito']]

Disclaimer: I wrote fast-autocomplete.

answered Jul 01 '20 at 17:39

Seperman

4,254
1
28
27

This is actually very handy. Thank you sir! – WannabeSmith Jul 28 '20 at 20:59
1

Sure! Happy to hear you found it handy! – Seperman Jul 30 '20 at 16:57
Just one doubt, is it scalable. I mean if my dictionary is large enough containing like some million or billion words, would it be able to return the results this fast ? – coderina May 29 '21 at 07:57
1

@coderina I have not had a chance to optimize it. If somebody has time to optimize it via Cython, that would be great. My use case at the time of creation did not require huge dictionaries. You can read more here at the bottom: https://github.com/seperman/fast-autocomplete/issues/10 – Seperman May 31 '21 at 19:18
I'm afraid this answer is not useful. The question asks about an auto-completer for the shell, not about an auto-completer in general. – mara004 Nov 11 '21 at 18:35
@mara004 If you look at the docs, it does offer shell autocomplete: https://github.com/seperman/fast-autocomplete#demo – Seperman Nov 12 '21 at 19:10
1

Oh, right, I overlooked that. Thanks! – mara004 Nov 13 '21 at 21:22

score 7 · Answer 4 · edited Apr 03 '13 at 07:44

I guess you will need to get a key pressed by the user.

You can achieve it (without pressing enter) with a method like this:

import termios, os, sys

def getkey():
    fd = sys.stdin.fileno()
    old = termios.tcgetattr(fd)
    new = termios.tcgetattr(fd)
    new[3] = new[3] & ~termios.ICANON & ~termios.ECHO
    new[6][termios.VMIN] = 1
    new[6][termios.VTIME] = 0
    termios.tcsetattr(fd, termios.TCSANOW, new)
    c = None
    try:
        c = os.read(fd, 1)
    finally:
        termios.tcsetattr(fd, termios.TCSAFLUSH, old)
    return c

Then, if this key is a tab key (for example, that's something you need to implement), then display all possibilities to user. If that's any other key, print it on stdout.

Oh, of course you will need to have getkey() looped in a while, as long as the user hits enter. You can also get a method like raw_input, that will get the whole word sign by sign, or display all the possibilities, when you hit a tab.

At least that's the item, you can start with. If you achieve any other problems, than write about them.

EDIT 1:

The get_word method can look like this:

def get_word():
    s = ""
    while True:
        a = getkey()
        if a == "\n":
            break
        elif a == "\t":
            print "all possibilities"
        else:
            s += a

    return s

word = get_word()
print word

The issue I'm occuring right now is the way to display a sign, you have just entered without any enteres and spaces, what both print a and print a, does.

To display the suggestions, you could then use the [curses](http://docs.python.org/library/curses.html) module to be able to mimic `vi`'s way of displaying the suggestions. — jro, Oct 19 '11 at 13:08
Oh, didn't hear about this one before. Looks interesting, thanks :) — Gandi, Oct 19 '11 at 13:15
@Gandi thank you. There is only the core part missing `incremental matching` and `showing hints while typing` — xralf, Oct 19 '11 at 14:11

score 4 · Answer 5 · edited May 02 '17 at 09:30

4

Steps:

Create a file .pythonrc in home directory by this command: vi .pythonrc

Enter this content:

import rlcompleter, readline  
readline.parse_and_bind('tab:complete')

Close the file
Now run

echo "export PYTHONSTARTUP=~/.pythonrc" >> ~/.bashrc
Restart the terminal

edited May 02 '17 at 09:30

jps

20,041
15
75
79

answered May 02 '17 at 08:31

AMAN JAIN

41
3

score 1 · Answer 6 · edited Feb 22 '18 at 12:27

1

For those (like me) that end up here searching for autocomplete in the interpreter:

https://web.archive.org/web/20140214003802/http://conjurecode.com/enable-auto-complete-in-python-interpreter/

This involves creating a file .pythonrc, modifying .bashrc and an import sys you have to import every time you launch the Python interpreter.

I wonder if the latter can be automated for even more win.

edited Feb 22 '18 at 12:27

styrofoam fly

578
2
9
25

answered Dec 17 '13 at 06:22

Mark Jeronimus

9,278
3
37
50

How to write code to autocomplete words and sentences?

6 Answers6

Updates: using the history buffer (as mentioned in comments)

Linked

Related