3

I have a solution build that produces outputs among which are a unit test assembly.

I have tried both the VsTest and the dotnet test pipeline tasks to run tests in this assembly. Every combination of arguments I can think of yields (paths anonymised):

Unable to find MyProject\MyTests.deps.json. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk".

It does have a reference to Microsoft.NET.Test.Sdk. What do I have to do to make it work?

I have tried:

  • dotnet test only accepts .csproj and doesn't recognise dlls at all
  • dotnet vstest produces the error message mentioned above. I've tried the /Framework:.NETCoreApp,Version=v2.1 command line argument, matching the version of the assembly, which doesn't make any difference
  • VsTest task with and without the above argument. Again no difference.

Edit: I realised I had neglected to include the *.deps.json files in the build artefacts. Supplying these has produced a new error:

Unable to find MyProject\testhost.dll. Please publish your test project and retry.

What?

Edit #2: I've published the .csproj in the related build pipeline and run a subsequent step to copy everything from the resulting folder to the build artefacts. This doesn't make any difference.

How can this be so difficult?

Edit #3: To explain more clearly, what I am trying to achieve is to run SpecFlow tests, written in a .net core assembly, during my release pipeline. I can't run them in a build pipeline because they are integration tests that exercise deployed artefacts, and those artefacts haven't been deployed until the end of the release pipeline. I don't think I can do the pattern explained here because the release artefacts are compiled project outputs and the code isn't in scope anymore.

Why can't I just run tests from an assembly like Visual Studio does? These tests work fine in VS2017 and VS2019.

Tom W
  • 5,108
  • 4
  • 30
  • 52
  • What kind of agent are you using here? Linux? Windows? 2015, 2017, 2019? Hosted? Private? – jessehouwing Aug 22 '19 at 14:30
  • The agent specification is 'vs2017-win2016'. Does that answer the question? – Tom W Aug 22 '19 at 14:34
  • Tried the 2019 agent? – jessehouwing Aug 22 '19 at 15:46
  • @jessehouwing won't work for most of the other artefacts. Why would this make a difference? – Tom W Aug 22 '19 at 15:47
  • Newer version of vstest. As it ships with 2019. – jessehouwing Aug 22 '19 at 16:03
  • @jessehouwing tried it, doesn't make a difference. – Tom W Aug 22 '19 at 16:23
  • Are you using the test platform installer task? https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/tool/vstest-platform-tool-installer – jessehouwing Aug 22 '19 at 17:02
  • @jessehouwing have tried this in conjunction with the vstest task which presents the error message mentioned. Doesn't make any difference – Tom W Aug 23 '19 at 01:28
  • Did dotnet vstest work locally on your machine? Are you intending to run test assemblies on a different host from where they were built? – Levi Lu-MSFT Aug 23 '19 at 09:24
  • @LeviLu-MSFT the tests run; they fail for a reason related to the test code itself but I don't think that is relevant to the issue here - the issue is that dotnet vstest doesn't see the right things somehow to be able to attempt to run tests. – Tom W Aug 23 '19 at 10:46
  • That is "the tests run locally" and "dotnet vstest in the release pipeline" – Tom W Aug 23 '19 at 10:51

2 Answers2

0

Can you find <project>.runtimeconfig.dev.json in your specflow test artifacts. The existence of this file can cause testhost.dll not found error. As the ProbingPaths defined in this file point to the locations of build host machine. And the tests are run in release host machine, as descripted in this page, You can remove this file from your artifacts and have a try.

However I tried on my demo project and it did not work for me, for the dependency packages did not exist in release machine.

I run my demo specflow test successfully by following below steps. Hope it could be helpful to you.

1,I added dotnet push task to publish demo project.

2,And use Publish build Artifacts task to publish the result artifacts to release pipeline.

3,In my release pipeline. I add Vstest task to run the test assembly. The demo test ran successfully

enter image description here

enter image description here

enter image description here

Levi Lu-MSFT
  • 27,483
  • 2
  • 31
  • 43
0

As usual, PEBKAC (Problem Exists Between Keyboard And Chair).

I had mistakenly taken the bin/Debug folder's contents, not the contents of the /publish subfolder including all the required framework assemblies.

This exposed a further problem:

An assembly specified in the application dependencies manifest (MyProject.deps.json) was not found: 2019-08-27T16:12:11.5536355Z package: 'System.Private.ServiceModel', version: '4.5.3'

This was solved by passing the argument -r win_x64 to dotnet publish. System.ServiceModel is platform-dependent so the particular platform dll to support ServiceModel is not copied to the output folder unless the publish is bound to a specific platform.

Tom W
  • 5,108
  • 4
  • 30
  • 52