3

I am writing code for NIST FRVT. NIST wants the program to run at max 2 threads(Only CPU, No GPU). I am using TensorFlow in my code but it always spawns much more than 2 threads. I tried this solution. It decreased the number of threads, but not up to 2

I'm getting this warning

[WARNING] We've detected that your software may be threading or using other multiprocessing techniques during template creation. The number of threads detected was 9 and it should be 2. Per the API document, implementations must run single-threaded. In the test environment, there is no advantage to threading, because NIST will distribute workload across multiple blades and multiple processes. We highly recommend that you fix this issue prior to submission.

NIST is calculating threads by top -H -b -n1 | grep validate11 | wc -l

Is there any way to force TensorFlow to use at max 2 threads?

Is there any TensorFlow version that will run on 2 threads?

(It is because of TensorFlow, I checked by removing TensorFlow part from the code)

tensorflow version 1.8.0

opencv version 3.4.1

g++ version 4.8.5

g++ -std=c++11

Alessio
  • 3,404
  • 19
  • 35
  • 48
Nawal Singh
  • 31
  • 1
  • 3
  • Highly parallel work is the whole point of tensorflow, is it not? It just sounds like you're using the wrong tool for the job. – sweenish Feb 13 '20 at 11:51
  • Don't know exact details but I highly doubt that's it is possible. In fact, it is meaningless to begin with. What is normally desired, when one wants to limit cpu usage, is to limit the number of threads that run concurrently. In general, the topic of executors isn't solved thus there is little chance of globaly effective solution. – ALX23z Feb 13 '20 at 11:52
  • 2
    It seems a bit complicated, as TF uses threads in different ways. Looking at [`process_util.cc`](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/core/common_runtime/process_util.cc#L93-L113), there are at least three relevant env vars, `TF_NUM_INTEROP_THREADS`, `TF_NUM_INTRAOP_THREADS` and `OMP_NUM_THREADS`. – jdehesa Feb 13 '20 at 13:41
  • 1
    You can also look at [`NumSchedulableCPUs`](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/core/platform/default/port.cc#L64-L81) to see how default parallelism is computed (here [Windows](https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/core/platform/windows/port.cc#L52-L56)). In Linux, for example, maybe you can change the affinity of the process so it is elegible to run on only two cores. – jdehesa Feb 13 '20 at 13:42
  • 1
    @jdehesa I tried building tensorflow after hard-coding NumSchedulableCPUs's return value to 0 and 1, still warning is there. NIST uses Linux(Centos 7) and they want one shared file(.so file), we have no control over process. – Nawal Singh Feb 14 '20 at 06:29
  • By setting the thread affinity from my C++ code, I can divert all processing to any specific core of the CPU but the number of threads are still equal to the number of cores. During execution (createTemplate function) only one thread is Running and the rest are in Sleeping mode. – fisakhan Aug 23 '20 at 12:22
  • @Nawal The reason you get 9 threads is that you have 8 cores. The validate11.cpp generates two processes using fork() where only one process runs your implementation (by calling createTempleate()). The process running your implementation uses TensorFlow that generates at least 1 thread per core (you have 8 cores and thus 8 threads). The other process doesn't use TensorFlow and generates only 1 thread. Thus in total 9 threads. I'm still struggling to force TensorFlow C API or C++ API not to generate more than 1 threads. – fisakhan Aug 24 '20 at 08:26

1 Answers1

1

With reference to the following sources, it looks like there is no possibility to run TensorFlow on 1 or 2 threads.

I submitted my implementation of NIST-FRVT having more than 6 threads, and NIST accepted that. Only 1 out of 6 threads were running and the rest of the threads were in sleeping mode.

fisakhan
  • 704
  • 1
  • 9
  • 27