How to create a simple custom language colorization to VS Code

Question

I'm trying to create a simple colorization for log files, now that it's possible include custom languages in Code (I'm on 0.9.2). I have created a simple .tmLanguage file for colorizing the letter 'q', just for starting up, but have been unsuccessful.

My new language, log, is associated correctly with the file extension and I can also select it manually from inside Code, but no coloring takes places. I have a feeling it has to do with what "scope" I associate my pattern with, but I'm not sure. Is there a list of valid scope to choose from? Initially I thought I'd use something general, such as "comment" to get some color, but it doesn't seem to work.

Here's my .tmLanguage file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>scopeName</key>
        <string>text.log</string>

        <key>fileTypes</key>
        <array>
            <string>log</string>
        </array>

        <key>name</key>
        <string>Log file</string>

        <key>patterns</key>
        <array>
            <dict>
                <key>match</key>
                <string>q</string>
                <key>name</key>
                <string>comment</string>
            </dict>
        </array>
    </dict>
</plist>

I'm probably misunderstanding something here, so any help is very appreciated :-)

Wosi · Accepted Answer · 2015-10-29T12:22:44.763

You need to use regular expressions instead of static strings to describe the pattern:

<key>match</key>
<string>q</string>  <- This needs to be a regular expression
<key>name</key>
<string>comment</string>

I provide a more useful example for a log file highlighter. It colors numbers, hints, warnings and errors in different colors. The rules to identify these keywords and numbers are based on regular expression.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>scopeName</key>
        <string>text.log</string>

        <key>fileTypes</key>
        <array>
            <string>log</string>
        </array>

        <key>name</key>
        <string>Log file</string>

        <key>patterns</key>
        <array>
            <dict>
                <key>match</key>
                <string>\b(?i:(hint|info|information))\b</string>
                <key>name</key>

                <string>info-token</string>
            </dict>                
            <dict>
                <key>match</key>
                <string>\b(?i:(warning|warn))\b</string>
                <key>name</key>
                <string>warn-token</string>
            </dict>
            <dict>
                <key>match</key>
                <string>\b(?i:(Error|Failure|Fail))\b</string>
                <key>name</key>
                <string>error-token</string>
            </dict>
            <dict>
                <key>match</key>
                <string>\b((0(x|X)[0-9a-fA-F]*)|(([0-9]+\.?[0-9]*)|(\.[0-9]+))((e|E)(\+|-)?[0-9]+)?)(L|l|UL|ul|u|U|F|f|ll|LL|ull|ULL)?\b</string>
                <key>name</key>
                <string>constant.numeric</string>
            </dict>                                
        </array>
        <key>uuid</key>
        <string>FF0550E0-3A29-11E3-AA6E-0800200C9A77</string>
    </dict>
</plist>

The highlighter gives a result like this (using the default theme):

I didn't find an official documentation about the available tokens (like error-token, constant.numeric etc). But there is a file located in %VSCODE_INSTALLATION%\resources\app\out\vs\languages\markdown\common\tokens.css. It seems to list all available tokens etc. Use it as a reference when you create the .tmLanguage file.

But pay attention: Some themes are using only the basic tokens. And some other themes are using the same color for many different tokens. So you should test the highlighter frequently against the most common themes to see whether the result looks good or not.

You should definitely visit this page about Language Grammars to learn more.

Many thanks for your useful sample and tips about the language tokens, I really appreciate it! Your sample worked right out of the box so I compared it to my more basic file and it turns out that I was missing the **uuid** part, and that caused the problems. It's working just fine now :-) — emilast, Oct 29 '15 at 20:40
BTW, `q` is actually a valid regex, albeit a very short one... It was just for testing, I don't really have a need for identifying q:s in my log files :-) — emilast, Oct 29 '15 at 20:43
I extracted a list of valid language tokens, it's available here: https://gist.github.com/vivainio/b89bd60a3f2c7bbb31f7e149d6cb8806 — vivainio, May 14 '16 at 19:57
Path for markdown is now: ``C:\Program Files\Microsoft VS Code\resources\app\extensions\markdown`` — Ben DeMott, Nov 06 '17 at 21:33
@Wosi can you please answer this? https://stackoverflow.com/questions/74076898/custom-language-vscode-extension-syntax-highlighting-for-variables-and-function — usmanharoon, Oct 17 '22 at 11:36

score 8 · Answer 2 · answered Jul 29 '16 at 20:01

We just released a language extension that brings colorization to the Output panel. Basically, it does the same thing as the approved answer on this thread, and adds the text/x-code-output mime type, which is used by the Output panel.

Get it free here: https://marketplace.visualstudio.com/items?itemName=IBM.output-colorizer

Source here: https://github.com/IBM-Bluemix/vscode-log-output-colorizer Please help contribute! Bugs, feature requests, contributions all welcome.

Here are some screenshots of it working:

score 1 · Answer 3 · edited May 23 '17 at 11:54

1

As @Woshi said, you need regular expressions.

As for the scopes that generally work with most color themes, I'll link you to this answer. Keep in mind that for scope to be picked up, it needs to be in dictionary with key "name".

edited May 23 '17 at 11:54

Community

1
1

answered Oct 29 '15 at 17:00

Taugeshtu

51
5

Thank you, I found that page yesterday. Useful reading, but it turned out my problem was that I was missing the **uuid** data. My scope `comment` turned out to be valid. – emilast Oct 29 '15 at 20:46

How to create a simple custom language colorization to VS Code

3 Answers3

Linked