4

I want' to create a protractor test adapter for Visual Studio. I have found I can create it with implementing ITestExecutor and ITestDiscoverer.
I follow the Marcel De Vries documentation (https://xpirit.com/wp-content/uploads/2016/03/Xpirit-magazine-2-Protractor-UI-testing.pdf) but I can't found how to add my extension in Visual Studio.

  • I have tried to add my assembly directly in the \Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions folder.
  • I have tried to create a Vsix and install it. (with UnitTestExtension as Asset)
  • I have tried to download Marcel's project (https://github.com/XpiritBV/ProtractorAdapter) compile and install it.

It's not work, the command "vstest.console.exe /listdiscoverers /UseVsixExtensions:true" doesn't list my discoverer. And JS tests don't found by VS.

What i'm doing wrong ?

Cedric
  • 672
  • 7
  • 17
  • Read up on creating VSIX projects. – JWP Jun 30 '16 at 14:35
  • Ok, i have found why vstest don't work. A very ugly mistake of my. I have been in the wrong visual studio folder. I'll continue to understand the Microsoft Test implementation. – Cedric Jul 04 '16 at 10:06
  • 2
    If anyone struggles with VS 2017, the correct path to put your adapter in is _Common7\IDE\Extensions\TestPlatform\Extensions_. – Grx70 Oct 17 '18 at 02:28

3 Answers3

15

The documentation is very poor for creating test adapter for VS. There are many URL that can help to understand how implement UTE with JS automated test :

In summary :
Step 1 :

  • we need to implement ITestExecutor to launch test with protractor prompt command and return the result.
  • we need to implement ITestDiscoverer to parse file in argument to find all tests inside.

You can test this primary version without VSix : Go to Visual studio Test extension folder (C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions for VS 2013) and add your DLL and dependencies. Try to launch tests with commands like that in windows command prompt :

cd C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow  
vstest.console.exe "c:\test\test.js"

Step 2 :
By default Visual Studio only parse .dll and .exe files. If you want to parse another file type (.lua, .js, ...) you need to create a parser who explicitly browse source files and find all tests.
You need to create implement two Interfaces :

  • ITestContainerDiscoverer
  • ITestContainer

This project (https://github.com/XpiritBV/ProtractorAdapter) will be useful to understand how implement that and how adapt your code to subscribe on any changes in your project (Add file, Edit file, Project Load, etc.).

Important : You need to create a VSix project to test that ! If not, Visual Studio don't load your TestContaineDiscoverer.

Important 2 : When you create a VSix file, you need to explain two Assets :

  • Type : Microsoft.VisualStudio.MefComponent / Path : |YourProject|
  • Type : UnitTestExtension / Path |YourProject|

Hope it's help somebody.

Cedric
  • 672
  • 7
  • 17
  • How do you debug any issues that maybe there in a test adapter? – resp78 Jan 18 '17 at 03:01
  • 2
    You can lauchn VS in debug mode (http://stackoverflow.com/questions/9281662/how-to-debug-visual-studio-extensions). But I have encountered difficulties with that, so I have used the old and ugliest way : Write logs in a file. – Cedric Jan 18 '17 at 08:51
6

If you need to run the test adapter with vstest.console.exe but don't need visual studio integration you will need to do the following:

  • Create a project that is a class library
  • Very important: The assembly name must end with .TestAdapter or vstest.console.exe will not recognize your test adapter. You can change this setting in the properties of your project.
  • Import Microsoft.VisualStudio.TestPlatform.ObjectModel which for me was located at C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll
  • You only need to implement ITestDiscoverer and ITestExecutor
  • If your test adapter is for .dll or .exe files the default test adapter will run first. I could not find a way to disable this.

The above accepted answer and code was very useful for me when figuring out how to create a test adapter. Be aware that if you are creating a VSIX then the asset type UnitTestExtension is not available in the list but seems to work and can be added in the .vsixmanifest file manually.

Additional useful links:

Thomas927
  • 853
  • 1
  • 11
  • 19
2

Late to the party, but as of today (2022) I'd suggest to read through the docs mentioned in official MS Visual Studio Test Platform readme.

Also, watching this Exploring the Visual Studio Test Platform episode, altough dated, might help.

superjos
  • 12,189
  • 6
  • 89
  • 134
  • The content you are pointing to I don't think even existed when this question was asked. The top answer should be marked as obsolete. – PaulG Jun 13 '22 at 05:35
  • Definitely did not exist, dev experience for this piece of VS changed a nr of times in the past. I don't know about SO policies about obsolete contents. If you're right and the answer is *that* useful, eventually will receive so many upvotes to become the most voted (I don't think it'll happen, BTW) – superjos Jun 14 '22 at 08:23
  • Its been known to happen: https://stackoverflow.com/a/42966511/6438779 :thinking: Unfortunately, I would be surprised if enough of us were writing custom test adapters though. They don't mention anything about enabling the green play button in the IDE. I can ask a separate question - but do you happen to know anything about that? My adapter reads dlls, I thought it should #justwork, but while tests will run in the IDE - the play button does not appear. – PaulG Jun 14 '22 at 10:56
  • @PaulG sorry for late reply, I completely missed that. I've been far for too long from test adapter area. At the time, "it just worked": when I was in VS, inside a test project file, and there was an adapter installed for that type of test project, I could push the play button in the test runner panel at the bottom. I remember I could also switch some .Net framework version or something in the top menu bar (as I had issues with that and asked here on SO) – superjos Sep 15 '22 at 16:40
  • BTW in the python question you referenced, there is a more recent, most voted answer yet the accepted answer is still the older one :) (that's what I was trying to say before) – superjos Sep 15 '22 at 16:42
  • 1
    For future travellers - I ended up working out how to accomplish this custom test adapter - its now implemented over in https://github.com/paulegradie/Sailfish – PaulG Apr 13 '23 at 03:27