1

newbie here, first message :)

I need tensorflow with CUDA, AVX and SSE on a windows machine. As far as I understood I need to build it myself. I first tried with Anaconda, but it was a mess, so I uninstalled anything related to python and I started following step by step the official guide

I used several commands to build, for instance:

bazel build -c opt --copt=-march=native --copt=-mfpmath=both --config=cuda -k //tensorflow/tools/pip_package:build_pip_package

bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda -k //tensorflow/tools/pip_package:build_pip_package

bazel build --config=opt --config=cuda --define=no_tensorflow_py_deps=true //tensorflow/tools/pip_package:build_pip_package

The first two commands came from here and might be old.

The building always fails with this message:

ERROR: missing input file 'external/llvm-project/mlir/include/mlir/Interfaces/SideEffectInterfaces.h', owner: '@llvm-project//mlir:include/mlir/Interfaces/SideEffectInterfaces.h'

Does anybody understand what is going on here?

Also, what is the best command to build among the one I used? Is there any way to install it in Anaconda on windows (with CUDA, avx and SSE capabilities)?

talonmies
  • 70,661
  • 34
  • 192
  • 269
giorgio pulvirenti
  • 13
  • 2

1 Answers1

0

Building tensorflow on windows can be tough, there are many ways for it to fail. The procedure will vary with the version you are compiling. For the most part, the instructions on the tensorflow site are correct if followed verbatim. I think the trickiest part is matching the version of tools used to compile with version of tensorflow you are working with.

My suggestion would be to lock into a particular version of tensorflow and stick with that until you succeed. If you git clone the source from github, I would suggest git checkout r2.2. This will put you into the most recent version.

I would avoid anaconda as that presents complications with the python version you are working with. I have had good results with python 3.6.8., but it may be possible to use 3.7.

You will need a specific version of Bazel as well, 2.0.0 is what works with tensorflow r2.2. Be mindful that you need to configure the BAZEL_VC environment variable before you start compiling. It should look something like C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC. You can do a bazel clean --expunge after the variable is set to avoid some confusion.

The r2.2 tensorflow also requires MSVC 2019, it will not compile with other versions. You will need the build tools for this version as well.

The last bazel build command you showed is the correct one. Don't forget to python ./configure.py before starting a fresh compile.

If my guess is correct, the error message you are getting is from using an older version of MSVC on the later tensorflow source code, but that's just a guess.

sr996
  • 36
  • 2
  • 5
  • First of all, thanks a lot! I do not understand what is the BAZEL_VC environment and how to change it... so probably your guess is right. How do I configure it? I don't find it on the instructions... – giorgio pulvirenti May 21 '20 at 18:56
  • just found it here https://docs.bazel.build/versions/master/windows.html , I'll try it... thanks again :)) – giorgio pulvirenti May 21 '20 at 19:04
  • Hi sr996, You helped a lot - Thanks! That error doesn't appear anymore, but the process still fails. Firstly I want to find a better command to buid, because I have many "Command line warning D9002 : ignoring unknown option". Depending on the command I use to build, I have this set of ignored options: {'-march=native' '-mfpmath=both' '/std:c++14' '-fno-strict-aliasing' '-fexceptions'}, Or this other ones: {'-mavx' '-mavx2' '-mfma' '-mfpmath=both' '-msse4.2' '/std:c++14'}. The problem is that these are the most important commands for me! Do you know how to fix this? – giorgio pulvirenti May 22 '20 at 06:53
  • I don't know much about AVX and SSE, but my understanding is that these are related to CPU calculations, and are not important if CUDA is used as the GPU card will perform the math. There is a pretty extensive post on the subject here https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions?noredirect=1&lq=1 I would suggest putting those to the side at first and see if you can get tensorflow to compile completely, then go back and add those options – sr996 May 22 '20 at 10:25