Pygments syntax highlighter in python tkinter text widget

Question

I have been trying to incorporate syntax highlighting with the tkinter text widget. However, using the code found on this post, I cannot get it to work. There are no errors, but the text is not highlighted and a line is skipped after each character. If there is a better way to incorporate syntax highlighting with the tkinter text widget, I would be happy to hear it. Here is the smallest code I could find that replicates the issue:

import Tkinter
import ScrolledText
from pygments import lex
from pygments.lexers import PythonLexer

root = Tkinter.Tk(className=" How do I put an end to this behavior?")
textPad = ScrolledText.ScrolledText(root, width=100, height=80)
textPad.tag_configure("Token.Comment", foreground="#b21111")
code = textPad.get("1.0", "end-1c")
# Parse the code and insert into the widget
def syn(event=None):
    for token, content in lex(code, PythonLexer()):
        textPad.insert("end", content, str(token))
textPad.pack()
root.bind("<Key>", syn)
root.mainloop()

So far, I have not found a solution to this problem (otherwise I would not be posting here). Any help regarding syntax highlighting a tkinter text widget would be appreciated.

Note: This is on python 2.7 with Windows 7.

Answer 1

To highlight certain words you can do this :

textarea.tag_remove("tagname","1.0",tkinter.END)
first = "1.0"
while(True):
    first = textarea.search("word_you_are_looking_for", first, nocase=False, stopindex=tkinter.END)
    if not first:
        break
    last = first+"+"+str(len("word_you_are_looking_for"))+"c"
    textarea.tag_add("tagname", first, last)
    first = last
textarea.tag_config("tagname", foreground="#00FF00")

Answer 2

The code in the question you linked to was designed more for highlighting already existing text, whereas it looks like you're trying to highlight it as you type.

I can give some suggestions to get you started, though I've never done this and don't know what the most efficient solution is. The solution in this answer is only a starting point, there's no guarantee it is actually suited to your problem.

The short synopis is this: don't set up a binding that inserts anything. Instead, just highlight what was inserted by the default bindings.

To do this, the first step is to bind on <KeyRelease> rather than <Key>. The difference is that <KeyRelease> will happen after a character has been inserted whereas <Key> happens before a character is inserted.

Second, you need to get tokens from the lexer, and apply tags to the text for each token. To do that you need to keep track where in the document the lexer is, and then use the length of the token to determine the end of the token.

In the following solution I create a mark ("range_start") to designate the current location in the file where the pygments lexer is, and then compute the mark "range_end" based on the start, and the length of the token returned by pygments. I don't know how robust this is in the face of multi-byte characters. For now, lets assume single byte characters.

def syn(event=None):
    textPad.mark_set("range_start", "1.0")
    data = textPad.get("1.0", "end-1c")
    for token, content in lex(data, PythonLexer()):
        textPad.mark_set("range_end", "range_start + %dc" % len(content))
        textPad.tag_add(str(token), "range_start", "range_end")
        textPad.mark_set("range_start", "range_end")

This is crazy inefficient since it re-applies the highlighting to the whole document on every keypress. There are ways to minimize that, such as only highlighting after each word, or when the GUI goes idle, or some other sort of trigger.