8

I have a custom MSBuild target included in my C++ project that produces a data file in the $(OutDir) folder for each item of a given item type. I have the item type hooked up with a property page schema so you can select it on files in the solution explorer and my target declares input and outputs so incremental builds work. I have also added my target to the $(BuildDependsOn) property so it is automatically evaluated during the Build target Visual Studio invokes.

Everything seems to work except for one thing: If I delete one of my output data files in the $(OutDir) and then build Visual Studio does nothing and says my project is up to date. If I delete the exe file the project produces or touch the modified time of one of the MSBuild scripts Visual Studio re-evaluates the targts and finds the output file is missing, causing it to be re-built using my target.

From the MSBuild diagnostic logging it seems like Visual Studio is internally maintaining some list of output files and input files that it checks to avoid evaluating the MSBuild script at all. How do I add my output files to this list?

Lucas
  • 631
  • 4
  • 9
  • Generally speaking, you cannot do much. You might report this to Microsoft Connect and see if others have a workaround for it. – Lex Li Jan 31 '16 at 01:50
  • I'm voting to close this question as off-topic because it is related to a vendor product without much public documentation on the technical details to resolve this issue. – Lex Li Jan 31 '16 at 01:51
  • 9
    @LexLi if we'd routinely close questions related to vendor products without much documentation SO would be a *lot* less interesting – stijn Jan 31 '16 at 10:03

5 Answers5

10

MsBuild/VS indeed have a mechanism to determine what is up-to-date with respect to the input files, it revolves around an executable tracker.exe which scans .tlog files to figure out what a project's output files are. There might be more to it, and if you look around on the internet you can probably get more info about this.

But the thing is you don't really need to understand every single detail of it: you can find a simple usage example for it when inspecting how the built-in CustomBuildStep works and apply that to your case. I'll briefly explain how I got to this because I think it might be useful for you as well in dealing with msbuild questions like these.

If you add

<ItemDefinitionGroup>
  <CustomBuildStep>
    <Command>echo foo &gt; $(OutDir)\foo.txt</Command>
    <Outputs>$(OutDir)\foo.txt</Outputs>
  </CustomBuildStep>
</ItemDefinitionGroup>

either manually or via the project's property pages for Custom Build Step you'll see the beahviour is eactly what you need: if foo.txt is deleted a build will start, while a build is marked up-to-date if it is not (well, and when the rest of the outputs are also up-to-date).

Hence the key is to do what CustomBuildStep does under the hood, and figuring that out is just a matter of using your tool of choice to search all occurrences of CustomBuildStep in all files under C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120 (adjust path for platform/VS version used).

This leads us to Microsoft.CppCommon.Targets where the target named CustomBuildStep (mind you, that's the same name as the entry in the ItemDefinitionGroup above) invokes the actual CustomBuildStep command. It also has this particularily interesting bit:

<!-- Appended tlog to track custom build events -->
<WriteLinesToFile Encoding="Unicode"
  File="$(TLogLocation)$(ProjectName).write.1u.tlog"
  Lines="@(CustomBuildStep->'^%(Identity)');@(CustomBuildStep->MetaData('Outputs')->FullPath()->Distinct())"/>

So this writes the path of the Outputs to a .tlog file in the directory used by the tracker and makes it work as desired. Also see here for more information about the format.

tl;dr Use WriteLinesToFile to append full paths of your targets' outputs to a file like $(TLogLocation)$(ProjectName).write.1u.tlog. I'm saying like because write.tlog, write.u.tlog etc also work.

stijn
  • 34,664
  • 13
  • 111
  • 163
  • It is a wonderful post I will upvote, but I still find the part "you don't really need to understand every single detail of it" annoying. I left my previous comment because though MSBuild is now fully open source, VS is not and it does build the projects differently. It would be very difficult to gain the insights and apply a proper fix with digging. Without fully understanding of the details, merely a fix won't guarantee it works for long. We are lucky to have experts like you, but in most cases I would prefer to resort to Microsoft support, who has access to the code base. – Lex Li Jan 31 '16 at 11:10
  • 1
    @LexLi it's a pitty not more documentation is available for this and you do make a compelling point, which however goes into the sometimes religious territory of 'how much does one need to know about a tool to work with it'. Not knowing a thing about my browser's internals for instance I can work with it just fine. Sure it is open source, and in case of problems that might help me to better find and understand a ifx, but just the fact that a fixes' details are fully clear does imo not provide any guarantees that it'll somehow work longer than a fix with less details clear. – stijn Jan 31 '16 at 11:58
  • This worked, thanks. It doesn't make sense that Microsoft doesn't document this here though https://msdn.microsoft.com/en-us/library/ms366724.aspx – Lucas Jan 31 '16 at 20:37
6

Visual Studio uses something called Visual Studio Common Project System (CPS) (https://github.com/Microsoft/VSProjectSystem) (VS 2017) to manage projects, including build process.

Within CPS anything that implements IBuildUpToDateCheckProvider interface can be used as a 'UpToDateChecker' for a project. 'UpToDateChecker' is invoked before invoking MsBuild. Its main purpose is to determine whether or not invoke MsBuild to build project, or to mark project as 'Up To Date' and skip msbuild all along.

This 'UpToDateChecker' is exactly what prints into diagnostic build output:

1>------ Up-To-Date check: Project: "ProjectName", Configuration: Debug x86 ------ Project is not up-to-date: build input 'header.h' was modified after build output 'a.out'. Input time: 12/27/2018 4:43:08 PM, Output time: 1/1/0001 2:00:00 AM

As for C++ Projects, for VS 2017 its default 'UpToDateChecker' is VCProjectBuildUpToDateCheck ( Microsoft.VisualStudio.Project.VisualC.VCProjectEngine.dll ). As starter, it looks into tlogs directory ( usually something like Debug\x86\.tlog) for these files:

  • .lastbuildstate
  • unsuccessfulbuild
  • all '.read..tlog' - input files, marked as 'build input' in diagnostic build output
  • all '.write..tlog' - output files, marked as 'build output' in diagnostic build output

There's actually more checks, but most fails occur when checking these 4 types

Mariia N
  • 61
  • 1
  • 1
  • Do you know, where I can find the default `UpToDateChecker` for C#-projects? – roli09 Jan 15 '19 at 14:03
  • My best guess is Microsoft.VisualStudio.ProjectSystem.Managed.VS.dll. There's BuildUpToDateCheck.IsUpToDateAsync() – Mariia N Jan 17 '19 at 12:36
  • 1
    I already found it. It's in `csproj.dll`. But since this is a native dll it's hard to read. Microsoft.VisualStudio.ProjectSystem.Managed.dll contains the new implementations for .net core projects and shared projects. – roli09 Jan 17 '19 at 12:50
  • 2
    @roli09 `csproj.dll` is the old, legacy project system for C# projects. It is not based on CPS, is closed source, and does not support newer features like SDK-style projects. The more modern project system which implements CPS's `IBuildUpToDateCheckProvider` is, as Mariia N says, in `Microsoft.VisualStudio.ProjectSystem.Managed.VS.dll`. It's open source too. You can find the implementation at https://github.com/dotnet/project-system. – Drew Noakes Jul 04 '20 at 09:42
4

The original question here relates to C++ projects, but for anyone finding this while searching for information about modern (SDK-style) C#/VB/F# projects, you can customise Visual Studio's fast up-to-date check as described in this document:

https://github.com/dotnet/project-system/blob/master/docs/up-to-date-check.md

In a nutshell, you specify inputs and outputs as items:

  • UpToDateCheckInput — Describes an input file that MSBuild would not otherwise know about
  • UpToDateCheckBuilt — Describes an output file that MSBuild would not otherwise know about

It can be very helpful to increase the diagnostic logging level for the up-to-date check via this setting:

enter image description here

Drew Noakes
  • 300,895
  • 165
  • 679
  • 742
  • 1
    Thanks, this really helped. I completely missed the separate config section for SDK-Style projects. – Oliver Feb 01 '23 at 11:07
1

You can find out why a project is being rebuilt by enabling the verbosity of the fast up to date checker in the registry key:

New-ItemProperty              `
 -Name U2DCheckVerbosity      `
 -PropertyType DWORD -Value 1 `
 -Path HKCU:\Software\Microsoft\VisualStudio\14.0\General -Force

You should be able to see in the build log messages like

Project 'Caliburn.Micro.Silverlight.Extensions' is not up to date. Project item 'C:\dev\projects\Caliburn.Micro.Silverlight.Extensions\NavigationBootstrapperSample.cs.pp' has 'Copy to Output Directory' attribute set to 'Copy always'.

[1] https://blogs.msdn.microsoft.com/kirillosenkov/2014/08/04/how-to-investigate-rebuilding-in-visual-studio-when-nothing-has-changed/

Chui Tey
  • 5,436
  • 2
  • 35
  • 44
1

To enable logging for old-style projects (i.e. non-SDK-style projects, common in the .NET Framework era):

  1. Open a "Developer Command Prompt" for the particular version of Visual Studio you are using.
  2. Enter command: 
    vsregedit set "%cd%" HKCU General U2DCheckVerbosity dword 1
  3. The message Set value for U2DCheckVerbosity should be displayed.

Run the same command with a 0 instead of a 1 to disable this logging.

More information at: https://github.com/dotnet/project-system/blob/main/docs/up-to-date-check.md#net-framework-projects

Drew Noakes
  • 300,895
  • 165
  • 679
  • 742