tools for testing vim plugins

Question

I'm looking for some tools for testing vim scripts. Either vim scripts that

do unit/functional testing, or
classes for some other library (eg Python's unittest module) that make it convenient to
- run vim with parameters that cause it to do some tests on its environment, and
- determine from the output whether or not a given test passed.

I'm aware of a couple of vim scripts that do unit testing, but they're sort of vaguely documented and may or may not actually be useful:

vim-unit:

purports "To provide vim scripts with a simple unit testing framework and tools"
first and only version (v0.1) was released in 2004
documentation doesn't mention whether or not it works reliably, other than to state that it is "fare [sic] from finished".

unit-test.vim:

This one also seems pretty experimental, and may not be particularly reliable.
May have been abandoned or back-shelved: last commit was in 2009-11 (> 6 months ago)
No tagged revisions have been created (ie no releases)

So information from people who are using one of those two existent modules, and/or links to other, more clearly usable, options, are very welcome.

Justin M. Keyes · Answer 1 · 2013-11-02T02:39:31.197

19

vader.vim is easy, and amazing. It has no external dependencies (doesn't require ruby/rake), it's a pure vimscript plugin. Here's a fully specified test:

Given (description of test):
  foo bar baz

Do (move around, insert some text):
  2Wiab\<Enter>c

Expect:
  foo bar ab
  cbaz

If you have the test file open, you can run it like this:

:Vader %

Or you can point to the file path:

:Vader ./test.vader

edited Nov 02 '13 at 02:39

answered Nov 01 '13 at 02:59

Justin M. Keyes

6,679
3
33
60

5

On top of that the author replied quickly to issues. – Ciro Santilli OurBigBook.com Sep 07 '14 at 13:07

score 12 · Answer 2 · answered Apr 24 '12 at 07:13

I've had success using Andrew Radev's Vimrunner in conjunction with RSpec to both test Vim plugins and set them up on a continuous integration server.

In brief, Vimrunner uses Vim's client-server functionality to fire up a Vim server and then send remote commands so that you can inspect (and verify) the outcome. It's a Ruby gem so you'll need at least some familiarity with Ruby but if you put the time in then you get the full power of RSpec in order to write your tests.

For example, a file called spec/runspec.vim_spec.rb:

require "vimrunner"
require "fileutils"

describe "runspec.vim" do
  before(:suite) do
    VIM = Vimrunner.start_gui_vim
    VIM.add_plugin(File.expand_path('../..', __FILE__), 'plugin/runspec.vim')
  end

  after(:all) do
    VIM.kill
  end

  it "returns the current path if it ends in _test.rb" do
    VIM.echo('runspec#SpecPath("foo_test.rb")').should == "foo_test.rb"
    VIM.echo('runspec#SpecPath("bar/foo_test.rb")').should == "bar/foo_test.rb"
  end

  context "with a spec directory" do
    before do
      FileUtils.mkdir("spec")
    end

    after do
      FileUtils.remove_entry_secure("spec")
    end

    it "finds a spec with the same name" do
      FileUtils.touch("spec/foo_spec.rb")
      VIM.echo('runspec#SpecPath("foo.rb")').should == "spec/foo_spec.rb"
    end
  end
end

I've written about it at length in "Testing Vim Plugins on Travis CI with RSpec and Vimrunner" if you want more detail.

Luc Hermitte · Answer 3 · 2021-03-07T13:11:56.377

There is another (pure Vimscript) UT plugin that I'm maintaining.

It is documented, it comes with several examples, and it is also used by my other plugins.

It aims at testing function results and buffer contents, and displaying the failures in the quickfix window. Exception callstacks are also decoded. AFAIK, it's the only plugin so far (or at least the first) that's meant to fill the quickfix window. Since then, I've added helper scripts to produce test results with rspec (+Vimrunner)

Since v2.0 (May 2020), the plugin can also test buffer content -- after it has been altered with mappings/snippets/.... Up until then I've been using other plugins. For instance, I used to test my C++ snippets (from lh-cpp) on travis with VimRunner+RSpec.

Regarding the syntax, for instance the following

Assert 1 > 2
Assert 1 > 0
Assert s:foo > s:Bar(g:var + 28) / strlen("foobar")

debug AssertTxt (s:foo > s:Bar(g:var+28)
      \, s:foo." isn't bigger than s:Bar(".g:var."+28)")
AssertEquals!('a', 'a')
AssertDiffers('a', 'a')
let dict = {}
AssertIs(dict, dict)
AssertIsNot(dict, dict)
AssertMatch('abc', 'a')
AssertRelation(1, '<', 2)
AssertThrows 0 + [0]

would produce:

tests/lh/README.vim|| SUITE <[lh#UT] Demonstrate assertions in README>
tests/lh/README.vim|27 error| assertion failed: 1 > 2
tests/lh/README.vim|31 error| assertion failed: s:foo > s:Bar(g:var + 28) / strlen("foobar")
tests/lh/README.vim|33 error| assertion failed: -1 isn't bigger than s:Bar(5+28)
tests/lh/README.vim|37 error| assertion failed: 'a' is not different from 'a'
tests/lh/README.vim|40 error| assertion failed: {} is not identical to {}

Or, if we want to test buffer contents

silent! call lh#window#create_window_with('new') " work around possible E36
try
    " :SetBufferContent a/file/name.txt 
    " or
    SetBufferContent << trim EOF
    1
    3
    2
    EOF

    %sort

    " AssertBufferMatch a/file/NAME.txt
    " or
    AssertBufferMatch << trim EOF
    1
    4
    3
    EOF
finally
    silent bw!
endtry

which results into

tests/lh/README.vim|78 error| assertion failed: Observed buffer does not match Expected reference:
|| ---
|| +++
|| @@ -1,3 +1,3 @@
||  1
|| -4
|| +2
||  3

(hitting D in the quickfix window will open the produced result alongside the expected result in diff mode in a new tab)

yay documentation! Thanks, I'll check this out. Also noticed that there are a couple of other similar tools listed in the links section there. — intuited, Jun 14 '10 at 13:12
The link in the above post does not work. I think this ( https://github.com/LucHermitte/vim-UT ) is where it should go. — Forbesmyester, Feb 25 '16 at 10:40
Yes indeed. I forgot to update it after google code shut down. Thanks! — Luc Hermitte, Feb 25 '16 at 11:33

score 3 · Answer 4 · answered Jun 13 '10 at 16:32

I've used vim-unit before. At the very least it means you don't have to write your own AssertEquals and AssertTrue functions. It also has a nice feature that lets you run the current function, if it begins with "Test", by placing the cursor within the function body and typing :call VUAutoRun().

The documentation is a bit iffy and unfinished, but if you have experience with other XUnit testing libraries it won't be unfamiliar to you.

Neither of the script mentioned have ways to check for vim specific features - you can't change buffers and then check expectations on the result - so you will have to write your vimscript in a testable way. For example, pass strings into functions rather than pulling them out of buffers with getline() inside the function itself, return strings instead of using setline(), that sort of thing.

Any support for configuring vim startup? For example, can I specify that only a specific plugin should be loaded? — intuited, Jun 13 '10 at 22:00
Nope, but you can always fake that with --noplugin and -u if it's an issue. — richq, Jun 14 '10 at 06:28

score 1 · Answer 5 · answered Jun 15 '17 at 13:44

There is vim-vspec. Your tests are written in vimscript and you can write them using a BDD-style (describe, it, expect, ...)

runtime! plugin/sandwich/function.vim
describe 'Adding Quotes'
  it 'should insert "" in an empty buffer'
    put! = ''
    call SmartQuotes("'")

    Expect getline(1) == "''"
    Expect col('.') == 2
  end
end

The GitHub has links to a video and an article to get you started:

A tutorial to use vim-vspec by Vimcasts.org [the video]

Introduce unit testing to Vim plugin development with vim-vspec [the article]

score 0 · Answer 6 · edited Apr 02 '16 at 23:26

0

Another few candidates:

VimBot - Similar to VimRunner in that it's written in Ruby and allows you to control a vim instance remotely. Is built to be used with the unit testing framework RSpec.

VimDriver - Same as VimBot except done in Python instead of Ruby (started as a direct port from VimBot) so you can use Python's unit testing framework if you're more familiar with that.

edited Apr 02 '16 at 23:26

Michael Mior

28,107
9
89
113

answered Sep 07 '14 at 19:36

Steve Vermeulen

1,406
1
19
25

score 0 · Answer 7 · answered Feb 11 '15 at 02:09

For functional testing, there's a tool called vroom. It has some limitations and can take seconds-to-minutes to get through thorough tests for a good size project, but it has a nice literate testing / documentation format with vim syntax highlighting support.

It's used to test the codefmt plugin and a few similar projects. You can check out the vroom/ dir there for examples.

tools for testing vim plugins

7 Answers7