0

I'll start by saying I don't know if this is possible, but I hope it is.

I have several jobs in Azure Devops running the "Windows Machine File Copy" task. Documentation here.

At the end of the task logs, there's an output that displays the total number of files...

2019-12-06T20:42:33.2991058Z ------------------------------------------------------------------------------
2019-12-06T20:42:33.2992182Z 
2019-12-06T20:42:33.2994274Z                Total    Copied   Skipped  Mismatch    FAILED    Extras
2019-12-06T20:42:33.2995919Z     Dirs :        29        28         0         0         0         0
2019-12-06T20:42:33.2997487Z    Files :       361       361         0         0         0         0
2019-12-06T20:42:33.2999056Z    Bytes :   89.96 m   89.96 m         0         0         0         0
2019-12-06T20:42:33.3000612Z    Times :   0:00:03   0:00:02                       0:00:00   0:00:00
2019-12-06T20:42:33.3000997Z 
2019-12-06T20:42:33.3002124Z 
2019-12-06T20:42:33.3003761Z    Speed :            31614545 Bytes/sec.
2019-12-06T20:42:33.3005534Z    Speed :            1808.998 MegaBytes/min.
2019-12-06T20:42:33.3006054Z    Ended : Friday, December 6, 2019 2:42:33 PM
2019-12-06T20:42:33.3006989Z 
2019-12-06T20:42:33.3049176Z Copying recursively from...

Is this value stored as a variable anywhere? Has anyone been able to work with logs like this to parse information from them programmatically?

nuprap
  • 385
  • 5
  • 22
  • What are you hoping to do with this information? Why is it important? – Daniel Mann Dec 12 '19 at 15:14
  • Someone with spiky hair wants it as a metric in a report. As build/deploy of apps are automated and moved into the cloud, they want to stare and compare. – nuprap Dec 12 '19 at 15:17
  • The number of files deployed is totally meaningless as any kind of metric. That needs to be communicated so that they can decide on a metric that actually measures something useful. – Daniel Mann Dec 12 '19 at 15:20
  • Yes. It's meaningless to engineers. However, management and auditors currently disagree. – nuprap Dec 12 '19 at 15:26

2 Answers2

1

You could probably write something that uses the REST APIs to retrieve the release logs. Or you could just look at the number of files in $(System.DefaultWorkingDirectory).

But I'd push back hard on this metric, as it is completely meaningless. Not just to developers. To everyone. Auditors, managers, developers, QA testers; pick a role and it's meaningless to someone in that role. Whatever thing they think this is measuring, it's not measuring that thing.

If the idea is that it's going to gauge the "size" or risk behind a release, that's totally wrong; a big change can involve very few files (or even remove files), and a small change can still cover a large number of files.

And in fact, what you'll most likely see is that every file is updated every time you run a release due to timestamps being different; robocopy does not operate on CRCs/hashes, it operates on timestamps. So all you're really finding out is if the number of files has increased or decreased. Which is equally meaningless and also measures nothing valuable.

Daniel Mann
  • 57,011
  • 13
  • 100
  • 120
  • I completely agree. But sometimes I have to pick my battles when companies are transitioning. This is one that they've dug their heels in on for now. Thanks for the info. I will check this out. Thanks Daniel! – nuprap Dec 12 '19 at 17:11
1

Agree with Daniel's appointment, this seems meaningleass for developer. But, if someone want this data, of course it can be parsed.

Firstly, we need know what does this task compiled based on. We have open the source code of this task in github: WindowsMachineFileCopyV2. You can see that this task is using RoboCopy to achieve the files copied.

So, now, the puzzle of yours could be considered into how to read and store the Robocopy output as variables.

I think you have known that it seems no way to do any operation directly on this task output. So, firstly, you can specify the /log:{file path} into the argument of this Windows Machine Copy files task.

The log look like this:

enter image description here

At present, it is very easier to parse out the exactly value and store it into variable. Just write a script to read out the value and store it. I ever provide a similar solution which about how to compile out the value from shell output.

Also, you can check this script for more reference.

## Create Powershell object to tally Robocopy results
$row = "" |select COPIED, MISMATCH, FAILED, EXTRAS
$row.COPIED = [int](($robo_arr[1] -split "\s+")[4]) + [int](($robo_arr[2] -split "\s+")[4])
$row.MISMATCH = [int](($robo_arr[1] -split "\s+")[6]) + [int](($robo_arr[2] -split "\s+")[6])
$row.FAILED = [int](($robo_arr[1] -split "\s+")[7]) + [int](($robo_arr[2] -split "\s+")[7])
$row.EXTRAS = [int](($robo_arr[1] -split "\s+")[8]) + [int](($robo_arr[2] -split "\s+")[8])
Mengdi Liang
  • 17,577
  • 2
  • 28
  • 35