Using a PyTorch model in C++ application on Windows

Question

Following the official PyTorch tutorial, I created the model in Python, converted it to Torch Script via tracing, and saved a script module to a .pt file. The C++ code loading the model and CMakeLists are identical to those from the tutorial.

I downloaded LibTorch 1.3 (stable, Windows, no CUDA, release) and extracted it, so my directory structure is:

│
├───artifact
│       traced_resnet_model.pt
│       
├───cmakeapp
│   │   CMakeLists.txt
│   │   example-app.cpp
│   │   
├───libtorch
│   │   build-hash   
│   ├───bin
│   ├───cmake
│   ├───include
│   ├───lib
│   ├───share
│   └───test

I have Visual Studio 2019 with CMake installed as a component, so I ran Developer Command Prompt for VS2019 and cd to the project directory (cmakeapp).

According to the guidelines, I ran the following commands to build the application:

mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=..\libtorch ..
make

CMake seemed to succeed, except with some warnings:

CMake Warning (dev) at D:/dox/projects/AI/torchscript/libtorch/share/cmake/Caffe
2/public/utils.cmake:57 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  D:/dox/projects/AI/torchscript/libtorch/share/cmake/Caffe2/Caffe2Config.cmake:
121 (caffe2_interface_library)
  D:/dox/projects/AI/torchscript/libtorch/share/cmake/Torch/TorchConfig.cmake:40
 (find_package)
  CMakeLists.txt:4 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at D:/dox/projects/AI/torchscript/libtorch/share/cmake/Torch
/TorchConfig.cmake:90 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  CMakeLists.txt:4 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

Now the first issue, neither make nor nmake work:

'make' is not recognized as an internal or external command, operable program or batch file.

D:\dox\projects\AI\torchscript\cmakeapp\build>nmake

Microsoft (R) Program Maintenance Utility Version 14.23.28107.0 Copyright (C) Microsoft Corporation.  All rights reserved.

NMAKE : fatal error U1064: MAKEFILE not found and no target specified Stop.

Am I missing something?

Second, I found the generated custom_ops.sln file, so opened it in Visual Studio. The project offers 4 different configurations: Debug, MinSizeRel, Release, and RelWithDebInfo. Building anything except Release fails:

LINK : fatal error LNK1181: cannot open input file 'torch-NOTFOUND.obj'
2>Done building project "example-app.vcxproj" -- FAILED.

I am quite surprised with this error since the libtorch path was specified and CMake succeeded to find it.

Third, building Release succeeds, but it skips ALL_BUILD project:

3>------ Skipped Build: Project: ALL_BUILD, Configuration: Release x64 ------
3>Project not selected to build for this solution configuration 
========== Build: 2 succeeded, 0 failed, 0 up-to-date, 1 skipped ==========

Not sure what solution configuration should have been selected to build all of it.

I'll be grateful for clarification regarding these confusing points.

Answer 1

The instructions on the linked site are Linux-centric, and appear to assume the user is operating in a Linux environment. On Linux, the last command make would work without any issue, but you are likely building with Visual Studio, not make. You should instead take the cross-platform approach, and tell CMake to build with whatever build tool it found during configuration; Try using cmake build . for the last command instead, as you can see is used in this other tutorial:

mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=/absolute/path/to/libtorch ..
cmake --build .

However, mentioned in that tutorial is the following:

On Windows, debug and release builds are not ABI-compatible. If you plan to build your project in debug mode, we recommend building PyTorch from source.

This indicates that the Release configuration should work, while you will need to download the source code from Github to build in Debug mode. As MSVC builds Debug by default, you should modify this last command to indicate the Release configuration:

cmake --build . --config Release

Also, when building on Windows with MSVC, their installation tutorial suggests appending the following lines to your CMake file to avoid errors discussed in this issue thread, which may also help the issues you are seeing:

if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $<TARGET_FILE_DIR:example-app>)
endif (MSVC)

Answer 2

I cannot answer directly to the previous answers. I am not sure I fully understand what is happening but I found a way to avoid the error torch-NOTFOUND.obj and have my project compile in RelWithDebugInfo. The error seems to be an IMPORTED_LOCATION error issue IMPORTED_LOCATION and -NOTFOUND. If you go in libtorch\share\cmake\Caffe2\Caffe2Targets-release.cmake towards the end (line 51-53 as of the 1.5.0 version) you find:

set_target_properties(torch PROPERTIES
  IMPORTED_IMPLIB_RELEASE "${_IMPORT_PREFIX}/lib/torch.lib"
  IMPORTED_LOCATION_RELEASE "${_IMPORT_PREFIX}/lib/torch.dll"
  )

Which I replaced by:

set_target_properties(torch PROPERTIES
  IMPORTED_IMPLIB "${_IMPORT_PREFIX}/lib/torch.lib"
  IMPORTED_LOCATION "${_IMPORT_PREFIX}/lib/torch.dll"
  )

And I could compile successfully.