Is it possible to load/read shape_predictor_68_face_landmarks.dat at compile time?

Question

I am trying to build a C++ application in Visual Studio using DLIB's face_landmark_detection_ex.cpp. The build application run from command promt and trained model and image file is passed as arguments.

face_landmark_detection_ex.exe shape_predictor_68_face_landmarks.dat image.jpg

this shape_predictor_68_face_landmarks.dat is the trained model for 68 landmarks to perform detection on input image and needs to load at run-time every time to perform any detection. I am trying to do following things.

• Load this shape_predictor_68_face_landmarks.dat at building the application or compile time.
• Read this shape_predictor_68_face_landmarks.dat inside the code so that every time my application strarts its execution, it will not take more amount of memory.

Is there any way to pack this file inside my application so that it will take less physical memory to run.

Update:

How can I store this shape_predictor_68_face_landmarks.dat file in a static buffer so that every time shape_predictor can read from this buffer.

Answer 1

Yes, its possible, but depends on Visual Studio and not cross-platform

1. You should create resource file and include hape_predictor_68_face_landmarks.dat into your project. See https://msdn.microsoft.com/ru-ru/library/7zxb70x7.aspx for details. This will make compiler to put this file into your exe/dll

2. Open resoure at runtime and get memory pointer https://msdn.microsoft.com/en-us/library/windows/desktop/ee719660(v=vs.85).aspx

3. Create memory stream (std::istream) from pointer.

   4. deserialize from this stream with dlib::deserialize

Here is minimal example, but without resource reading:

#include <string>
#include <iostream>
#include <dlib/image_processing/shape_predictor.h>


struct membuf : std::streambuf {
    membuf(char const* base, size_t size) {
        char* p(const_cast<char*>(base));
        this->setg(p, p, p + size);
    }
};
struct imemstream : virtual membuf, std::istream {
    imemstream(char const* base, size_t size)
            : membuf(base, size)
            , std::istream(static_cast<std::streambuf*>(this)) {
    }
};
using namespace dlib; //its important to use namespace dlib for deserialize work correctly
using namespace std;
int main(int argc, const char* argv[])
{
    const char* file_name = "shape_predictor_68_face_landmarks.dat";
    ifstream fs(file_name, ios::binary | ios::ate);
    streamsize size = fs.tellg();
    fs.seekg(0, ios::beg);
    std::vector<char> buffer(size);
    if (fs.read(buffer.data(), size))
    {
        cout << "Successfully read " << size << " bytes from " << file_name << " into buffer" << endl;
        imemstream stream(&buffer.front(), size); // here we are loading from memory buffer. you can change this line to use pointer from Resource
        shape_predictor sp;
        deserialize(sp, stream);
        cout << "Deserialized shape_predictor" << endl;
    }
    else cout << "Failed to read " << file_name << " into buffer" << endl;
    return 0;
}

And about memory usage.

First of all you should know that shape_predictor::operator() is const, and the documentation says that is safe to use one shape_predictor for different threads.

So, you can create one shape_predictor at the start of program and use it many times, even from different threads

Next, putting shape predictor inside resource will make it be loaded into RAM when program starts, but deserializing it from resource will make copy of this memory, and this will lead to RAM usage overhead. If you need minimal possible RAM usage - you should load it from file

And the last your question - how to initialize it by compiler. There is no ready-to-use solution for it, but you can use the code from shape_predictor.h/deserialize function and load it manually. I think, this is bad solution, because you will not get less RAM usage compared to loading file

So my recommendation is to load one shape_predictor from file and use it globally for all threads

Answer 2

I know this is an old question, but a Visual Studio only solution would not have worked in my case since I am using dlib in Linux/macOS. Here is a Unix compatible solution that I came up with.

What I did was to use the xxd tool to convert the model file into the unsigned char [] representation of the file contents, write that into a custom header file, and use that inside deserialize (rather than read in the file during execution).

The following command would generate the header file for shape_predictor_68_face_landmarks.dat:

xxd -i shape_predictor_68_face_landmarks.dat > shape_predictor_68_face_landmarks.hpp

If you look inside shape_predictor_68_face_landmarks.hpp, there will be 2 variables: shape_predictor_68_face_landmarks_dat of type unsigned char [] containing the contents of the model file and shape_predictor_68_face_landmarks_dat_len of type unsigned int.

Inside your dlib driver code, you would do the following

...
#include "shape_predictor_68_face_landmarks.hpp"
...

shape_predictor sp;
std::stringstream landmarksstream;
landmarksstream.write((const char*)shape_predictor_68_face_landmarks_dat, shape_predictor_68_face_landmarks_dat_len);
deserialize(sp, landmarksstream);

A word of warning: be careful about opening files generated by xxd because they can be quite large and cause your text editor to crash.

I can't answer to the efficiency of this method, but it does allow for the model file to be "read in" at compile time rather than execution time.