VSTS - Test Recording files empty

Question

For the VSTS (or TFS) experts among us...

I'm having an issue where recording files generated by the Microsoft Expression Encoder SDK show up empty. A small description of the context:

We're working in a VSTS environment with a build-definition and multiple release-definitions set up. This flow functions correctly. A check-in is done, build triggers and finishes, releases start and finish, up to the last release where UI tests run and end with the correct outcome. We're working with a build-agent and a seperate test-agent (both Windows 10 OS), which are in the same domain and can connect to each other without trouble.

I'm trying to include recordings in the test scenario's. I installed Microsoft Expression Encoder SP 2 on my laptop (which also functions as the build-agent) and on the test-agent. On both systems, I can manually make recordings without problems. When doing a test run locally (from Visual Studio itself), a recording is created accordingly.

Now the problem: when the test-agent starts the test runs, recording files are created, but they all end up as 4.93kb files without any content. I can open them, but they have 0:00 runtime. No errors show up in the build logs or in the Event Viewer: all I get are the empty files.

Some relevant code for background:

    [SetUp]
    public void Initialize()
    {
        testName = TestContext.CurrentContext.Test.Name;

        var path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + @"\Recordings";
        Directory.CreateDirectory(path);
        recorder = new ScreenCaptureJob
        {
            OutputScreenCaptureFileName = path + @"\" + testName + @".xesc",
            CaptureRectangle = new Rectangle(0, 0, 1024, 768),
            ShowFlashingBoundary = false,
            CaptureMouseCursor = false,
            CaptureFollowCursor = false,
            CaptureLargeMouseCursor = false,
            CaptureLayeredWindow = false,
            ShowCountdown = false
        };

        Logger.Logger.Log(recorder.OutputScreenCaptureFileName); 

        if (File.Exists(recorder.OutputScreenCaptureFileName))
        {
            File.Delete(recorder.OutputScreenCaptureFileName);
        }

        recorder.Start();

        /* Navigation to baseUrl */
        Driver.Navigate().GoToUrl(startUrl);
        WaitForUrl(startUrl);
        recorder.Stop();
    }

Does anyone have a clue to what (lack of) magic is causing this issue? If more info is needed for the analysis, then I'll be happy to provide it.

Thanks in advance!

-EDIT- In response to Wouter: I should note that the build-agent uses the Administrator account of the test-agent to login, so permissions should (probably?) not be an issue.

-EDIT 2- I added some logging towards the whole process, and this shows something interesting:

2017-12-14 10:51:29.322 C:\TestDrop*censored*\drop\bin\Debug\Recordings_LoadHomePage_ClickMenuQuestions_NavigatedToJobs.xesc

2017-12-14 10:51:29.322 posX  : 0
2017-12-14 10:51:29.337 posY  : 0
2017-12-14 10:51:29.337 width : 1024
2017-12-14 10:51:29.337 height: 768
2017-12-14 10:51:29.353 Size before starting: 0
2017-12-14 10:51:29.353 Status before starting: NotStarted
2017-12-14 10:51:29.353 Duration before starting: 00:00:00
2017-12-14 10:51:29.369 Framecount before starting: 0
2017-12-14 10:51:29.400 Size started        : 0
2017-12-14 10:51:29.415 Status started        : Running
2017-12-14 10:51:29.415 Duration started        : 00:00:00.0111608
2017-12-14 10:51:29.415 Framecount started        : 1
2017-12-14 10:51:44.315 Size close to ending: 0
2017-12-14 10:51:44.320 Status close to ending: Running
2017-12-14 10:51:44.324 Duration close to ending: 00:00:14.9117567
2017-12-14 10:51:44.332 Framecount close to ending: 190
2017-12-14 10:51:44.344 Size stopped        : 0
2017-12-14 10:51:44.349 Status stopped        : NotStarted
2017-12-14 10:51:44.354 Duration stopped        : 00:00:00
2017-12-14 10:51:44.359 Framecount stopped        : 0

Apparently, the recording is actually running. Right before recorder.stop(), the framecount is 190, but the file size remains 0. When running the test locally, I get the following:

2017-12-14 10:48:31.861 posX  : 0
2017-12-14 10:48:31.867 posY  : 0
2017-12-14 10:48:31.872 width : 1024
2017-12-14 10:48:31.877 height: 768
2017-12-14 10:48:31.882 Size before starting: 0
2017-12-14 10:48:31.888 Status before starting: NotStarted
2017-12-14 10:48:31.893 Duration before starting: 00:00:00
2017-12-14 10:48:31.898 Framecount before starting: 0
2017-12-14 10:48:31.948 Size started        : 0
2017-12-14 10:48:31.952 Status started        : Running
2017-12-14 10:48:31.956 Duration started        : 00:00:00.0090997
2017-12-14 10:48:31.963 Framecount started        : 0
2017-12-14 10:48:53.630 Size close to ending: 9998272
2017-12-14 10:48:53.634 Status close to ending: Running
2017-12-14 10:48:53.640 Duration close to ending: 00:00:21.6936283
2017-12-14 10:48:53.647 Framecount close to ending: 322
2017-12-14 10:48:53.669 Size stopped        : 0
2017-12-14 10:48:53.675 Status stopped        : NotStarted
2017-12-14 10:48:53.681 Duration stopped        : 00:00:00
2017-12-14 10:48:53.685 Framecount stopped        : 0

The only difference here is file size, which is 9998272 right before end. I assume this indicates a permissions issue indeed, although I'm still baffled as to why a video can be created, but not written...

Have you tried running the recording software manually logged in as the build agent? it could be a permissions issue — Wouter de Kort, Dec 13 '17 at 14:39
Valid idea, but the build-agent already logs in using the Administrator account of the test-agent. I will add it to the info: thanks for the note! — Tybs, Dec 13 '17 at 14:53
Could it be a timing issue where the video software hasn't started recording yet and the test is already finished? — Wouter de Kort, Dec 13 '17 at 15:44
Try to run the test through command line manually and check whether it works. Set/add system.debug variable to true, then queue build (and trigger release), then post the detail log here (remove personal information). You can compare the command. — starian chen-MSFT, Dec 14 '17 at 06:56
@WouterdeKort When I initially ran the tests, the recorder.stop command was set to the end of the test. As the test takes roughly 15 seconds to complete, at least some part of it should have been recorded if timing was the problem. — Tybs, Dec 14 '17 at 08:34
@starianchen-MSFT How would I run the test in command line on the remote machine? I've never run a test in that way so far. — Tybs, Dec 14 '17 at 08:36
Just run the test in command line on local machine and check the result. — starian chen-MSFT, Dec 14 '17 at 08:42
@starianchen-MSFT Thx for the input, I updated the post after some logging on both local and test-agent. — Tybs, Dec 14 '17 at 10:04
Do you use Run Functional Test task to run test? How do you configure it? Can you share the whole log here? — starian chen-MSFT, Dec 15 '17 at 01:30
@starianchen-MSFT We found a different solution to the problem, and we'll continue with that approach because of the time this has already taken: therefore, I'm afraid there won't be any more logs added for the question itself. Still, thanks a lot for your input! — Tybs, Dec 15 '17 at 10:15

Tybs · Accepted Answer · 2017-12-19T14:17:12.907

A work-around has been found. Since Expression Encoder itself works fine on the Test-Agent, I wrote a small application that runs on the Test-Agent itself and is listening for calls from other devices on the network. When receiving simple commands like "start" or "stop", the application uses the Expression Encoder SDK locally (so, on the Test-Agent) to start or stop the recordings. Since the application is run locally, any possible permission issues are prevented. Eventually, we will convert the application into a windows service to make the usage a bit easier.

Although this is technically not an answer to the issue, at least we have something we can work with. Thanks to everyone for your responses!

EDIT: While working with the solution mentioned, we discovered what the actual issue is: the recording process did not find place in an interactive session. It was running as a windows service, therefore there was no actual screen to record.

Apparently VSTS has an easy solution for this. The task "Visual Studio Test Agent Deployment" actually has a checkbox that's literally called "Run UI tests". For some reason I missed it before, I think it was called "Run as interactive session" in TFS2015.

Check this button, and (in my case) Expression Encoder has no problem saving the recordings.

VSTS - Test Recording files empty

1 Answers1