How to restore base64 encoded opencv java matrix to C++ cv::Mat?

Question

I'm extracting ORB descriptors of an image in android using opencv which is then sent to a C++ server using a socket. To send it via the socket I'm encoding the opencv mat of descriptors to base64 using the below java code.

    public static String matToJson(Mat mat){        
       JsonObject obj = new JsonObject();



          if(mat.isContinuous()){
               int cols = mat.cols();
               int rows = mat.rows();
               int elemSize = (int) mat.elemSize();    

               byte[] data = new byte[cols * rows * elemSize];

               mat.get(0, 0, data);

               obj.addProperty("rows", mat.rows()); 
               obj.addProperty("cols", mat.cols()); 
               obj.addProperty("type", mat.type());

               // We cannot set binary data to a json object, so:
               // Encoding data byte array to Base64.
               String dataString = new String(Base64.encode(data, Base64.DEFAULT));

               obj.addProperty("data", dataString);            

               Gson gson = new Gson();
               String json = gson.toJson(obj);

               return json;
           } else {
               Log.e(TAG, "Mat not continuous.");
           }

public static Mat matFromJson(String json){
    JsonParser parser = new JsonParser();
    JsonObject JsonObject = parser.parse(json).getAsJsonObject();

    int rows = JsonObject.get("rows").getAsInt();
    int cols = JsonObject.get("cols").getAsInt();
    int type = JsonObject.get("type").getAsInt();

    String dataString = JsonObject.get("data").getAsString();       
    byte[] data = Base64.decode(dataString.getBytes(), Base64.DEFAULT); 

    Mat mat = new Mat(rows, cols, type);
    mat.put(0, 0, data);

    return mat;
}

The output from matToJson() looks like below:

{"rows":26,"cols":32,"type":0,"data":"aSbt5TQabzJ9qv7s3ymQchrEfSyp8OWO5v8nkG6oUiFAMLFkAExBEGCgJggSF0AyAFRoCAlgsADJ\niAUwQCBGImQwuWSATEEQYrAiCNY3QToBVGgYAWDxAMmIRTTAYEYibDS4YqltU5o7v68Y3/9JOpHX\n/RAaZvsiy991f8JzxnNJJ33lNB5vMn3ofkzfP5BwlkR/LKnw4c7n/yOQTuhTcUAwOWAATFEQYCAg\nCBYXADCAFFAIAWCwAMGJADBAAEYyYCAxQABNQQBgpCQIFhdAMgBUQAgAYLAAwYgBMEACRiJgMLFk\nAEhBAGKkJAgWF0AyAVRIGABgsADJiAUwwABGImwwuXW4bVGaM/+mGJ+3QTqV3v0YK3b5As/fNX3A\n4EZzQTCRZABMRQBgoiIIVjdAMgBUaBgBYLEAyagFMMBgRiJJPj33NJ57N33ofky/O5Bwhi5+DKm4\n4e7n/zKQSojDKXw0vHW4b1Gas7+2GNc3QzuF33wYKGT5AMvbdTTA8EZzQCCRZABIAQBgoCIIFhdA\nMgBUaBgBYKAAiYAFMEBgRiBpLv33NI9pN30p/1z/KZBwgqZ8DLn44S7G/zKQSoDDKUAhEUAQSEEA\nYAAgCBYXADAQVEBAASCgACGIABBAAEYgdDC8dahsURLiv7YYlzdAOofWfRiLRfsAy8t1NODgRnNA\nIJFEAEgAAGCgIAgWE0AyAFRIAABgoACJgAUwQCBGIEAikEQgCAACYKggaBITwDIAVEwAAGCAAImA\nJTBAIAIhYDC5dGhtURJzvzYYl7dAOofWfRgLePMA34t1cODARnNAIBFAEEgAAGAAIAgSEwAwEFxA\nYEEggAwhiAAQQAJGIHEy1WQoDEEQ4KiyTJcTwDIExGwMiWHRAMCpIbBogEIhcapb+iytSRfzGbNY\nt4PQeEuubByba8Ix2vkwWOiIwil4oLhwie1RjqK3rhiXtUIagdbsEBpr4wDb5XV54EDGYnAwuXSI\nbVEb4rekGBe3QDoB1mwYGm/6EtundT3g4MZicaBb+qytSZb+GbFct8PYeKGObBybbdox2ekyePiB\nwij1oBl6rKxJm/IRoVi300g4gYpsHDtr2jPZqzR42MDCIA\u003d\u003d\n"}

the encoded base64 string in the "data" key is the Matrix data.

The above json string is sent to the server via socket.

Now, I need to convert the "data" value of the json string back to cv::Mat in C++, so how do i get back the matrix from the above base64 string ?

'The output encoded matrix looks like below:'. ? Your matrix? I would think it is the content of 'String json'. — greenapps, Oct 30 '15 at 08:49
Could you first show how you on Android would construct back a matrix from that json text? — greenapps, Oct 30 '15 at 08:52
'so how do i get back the matrix from the above base64 string ?'. Strange question. I would think you would ask: how do i get back the matrix from the above json string ? — greenapps, Oct 30 '15 at 08:54
@greenapps - the encoded matrix is in the json string "data". and I need to get back the encoded base64 string to matrix in C++. — pallavi, Oct 30 '15 at 09:05
You did not react on: Show how you would do it in Android. There are several steps you have to do. Just as you did when constructing this json text. So do it step by step. Make clear on which step you need help exactly. — greenapps, Oct 30 '15 at 09:12
@greenapps - the function `matToJson(mat)` in android converts the matrix to json. the matrix is the descriptors of the image. and i have posted the code for `matToJson(mat)`. — pallavi, Oct 30 '15 at 09:17
Yes i know. You only repeat yourself. I asked you to show your JsonToMat(jsontext) code. — greenapps, Oct 30 '15 at 09:19
@greenapps - what i need is converting the encoded string to matrix in C++ and not in android. FYI you can find the `matFromJson(jsontext)` in [http://answers.opencv.org/question/8873/best-way-to-store-a-mat-object-in-android/] — pallavi, Oct 30 '15 at 09:23
I will not follow that link. Post the Android/java code here and then ask for help to convert it in C++. Remove the 'sockets' tag from this post. — greenapps, Oct 30 '15 at 09:27
@greenapps - you can now find the `matFromJson(jsontext)` in the question. — pallavi, Oct 30 '15 at 09:31
Ok. Finally. So now you can tell us what your problem is to convert this java code to C++. — greenapps, Oct 30 '15 at 10:17
Possible duplicate of [Convert the Base64(String or Byte array) to mat(image) in c++(opencv)](http://stackoverflow.com/questions/32264709/convert-the-base64string-or-byte-array-to-matimage-in-copencv) — Miki, Oct 30 '15 at 10:18
@Miki - it is for converting to mat(image) in C++ opencv which uses imencode and imdecode functions which cannot be used for descriptors. — pallavi, Oct 30 '15 at 10:24
Well, still you see how to decode base64, that's seems the point of your question. — Miki, Oct 30 '15 at 10:27
If the problem is reading json data, just use a C++ library to retrieve data, and then decode the string. May I ask why you don't use XML/YAML format? — Miki, Oct 30 '15 at 10:29
The problem is not reading json data. I need to pass a opencv matrix from android to a server which runs on C++. Android uses java implementation of opencv matrix which can not directly be serialized in java and then deserialized to C++. So what I'm trying to do is, base64 encode the raw bytes of Java matrix and then decode the same in C++ and restore it to cv::Mat. OpenCV has methods for XML/YML, but these methods will store the matrix in disk. So, if I'm calling these functions on every frame, disk I/O will slow down the application. — pallavi, Oct 30 '15 at 10:39
You can use `FileStorage::MEMORY` to work in memory. See [here](http://docs.opencv.org/master/d6/d03/filestorage_8cpp-example.html#gsc.tab=0) — Miki, Oct 30 '15 at 10:45
btw, I retracted close vote for duplicate question. Still I'll leave the link that may be useful to decode base64 data. — Miki, Oct 30 '15 at 10:46

score 1 · Accepted Answer · edited May 23 '17 at 11:51

Assuming that you read the data from the JSON, you can create the Mat like this:

int rows = 26;
int cols = 32;
int type = 0;
string data = "aSbt5TQabz...";
string decoded_data = base64_decode(data);  
Mat m(rows, cols, type, (void*)decoded_data.data());

Code for base64_decode is from here

NOTE

I needed to manually convert the characted \u003D to = to make it work. You can avoid to escape HTML characters using Gson gson = new GsonBuilder().disableHtmlEscaping().create();. See here.

Code:

#include <opencv2\opencv.hpp>
#include <vector>
#include <string>
using namespace std;
using namespace cv;

// Code from: http://www.adp-gmbh.ch/cpp/common/base64.html

static const std::string base64_chars =
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz"
"0123456789+/";


static inline bool is_base64(unsigned char c) {
    return (isalnum(c) || (c == '+') || (c == '/'));
}

std::string base64_decode(std::string const& encoded_string) {
    int in_len = encoded_string.size();
    int i = 0;
    int j = 0;
    int in_ = 0;
    unsigned char char_array_4[4], char_array_3[3];
    std::string ret;

    while (in_len-- && (encoded_string[in_] != '=') && is_base64(encoded_string[in_])) {
        char_array_4[i++] = encoded_string[in_]; in_++;
        if (i == 4) {
            for (i = 0; i < 4; i++)
                char_array_4[i] = base64_chars.find(char_array_4[i]);

            char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
            char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
            char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];

            for (i = 0; (i < 3); i++)
                ret += char_array_3[i];
            i = 0;
        }
    }

    if (i) {
        for (j = i; j < 4; j++)
            char_array_4[j] = 0;

        for (j = 0; j < 4; j++)
            char_array_4[j] = base64_chars.find(char_array_4[j]);

        char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
        char_array_3[1] = ((char_array_4[1] & 0xf) << 4) + ((char_array_4[2] & 0x3c) >> 2);
        char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];

        for (j = 0; (j < i - 1); j++) ret += char_array_3[j];
    }

    return ret;
}

int main()
{
    // Read your JSON here...

    int rows = 26;
    int cols = 32;
    int type = 0;
    //string data(L"aSbt5TQabzJ9qv7s3ymQchrEfSyp8OWO5v8nkG6oUiFAMLFkAExBEGCgJggSF0AyAFRoCAlgsADJ\niAUwQCBGImQwuWSATEEQYrAiCNY3QToBVGgYAWDxAMmIRTTAYEYibDS4YqltU5o7v68Y3/9JOpHX\n/RAaZvsiy991f8JzxnNJJ33lNB5vMn3ofkzfP5BwlkR/LKnw4c7n/yOQTuhTcUAwOWAATFEQYCAg\nCBYXADCAFFAIAWCwAMGJADBAAEYyYCAxQABNQQBgpCQIFhdAMgBUQAgAYLAAwYgBMEACRiJgMLFk\nAEhBAGKkJAgWF0AyAVRIGABgsADJiAUwwABGImwwuXW4bVGaM/+mGJ+3QTqV3v0YK3b5As/fNX3A\n4EZzQTCRZABMRQBgoiIIVjdAMgBUaBgBYLEAyagFMMBgRiJJPj33NJ57N33ofky/O5Bwhi5+DKm4\n4e7n/zKQSojDKXw0vHW4b1Gas7+2GNc3QzuF33wYKGT5AMvbdTTA8EZzQCCRZABIAQBgoCIIFhdA\nMgBUaBgBYKAAiYAFMEBgRiBpLv33NI9pN30p/1z/KZBwgqZ8DLn44S7G/zKQSoDDKUAhEUAQSEEA\nYAAgCBYXADAQVEBAASCgACGIABBAAEYgdDC8dahsURLiv7YYlzdAOofWfRiLRfsAy8t1NODgRnNA\nIJFEAEgAAGCgIAgWE0AyAFRIAABgoACJgAUwQCBGIEAikEQgCAACYKggaBITwDIAVEwAAGCAAImA\nJTBAIAIhYDC5dGhtURJzvzYYl7dAOofWfRgLePMA34t1cODARnNAIBFAEEgAAGAAIAgSEwAwEFxA\nYEEggAwhiAAQQAJGIHEy1WQoDEEQ4KiyTJcTwDIExGwMiWHRAMCpIbBogEIhcapb+iytSRfzGbNY\nt4PQeEuubByba8Ix2vkwWOiIwil4oLhwie1RjqK3rhiXtUIagdbsEBpr4wDb5XV54EDGYnAwuXSI\nbVEb4rekGBe3QDoB1mwYGm/6EtundT3g4MZicaBb+qytSZb+GbFct8PYeKGObBybbdox2ekyePiB\nwij1oBl6rKxJm/IRoVi300g4gYpsHDtr2jPZqzR42MDCIA\u003d\u003d\n");
    string data = "aSbt5TQabzJ9qv7s3ymQchrEfSyp8OWO5v8nkG6oUiFAMLFkAExBEGCgJggSF0AyAFRoCAlgsADJ\niAUwQCBGImQwuWSATEEQYrAiCNY3QToBVGgYAWDxAMmIRTTAYEYibDS4YqltU5o7v68Y3/9JOpHX\n/RAaZvsiy991f8JzxnNJJ33lNB5vMn3ofkzfP5BwlkR/LKnw4c7n/yOQTuhTcUAwOWAATFEQYCAg\nCBYXADCAFFAIAWCwAMGJADBAAEYyYCAxQABNQQBgpCQIFhdAMgBUQAgAYLAAwYgBMEACRiJgMLFk\nAEhBAGKkJAgWF0AyAVRIGABgsADJiAUwwABGImwwuXW4bVGaM/+mGJ+3QTqV3v0YK3b5As/fNX3A\n4EZzQTCRZABMRQBgoiIIVjdAMgBUaBgBYLEAyagFMMBgRiJJPj33NJ57N33ofky/O5Bwhi5+DKm4\n4e7n/zKQSojDKXw0vHW4b1Gas7+2GNc3QzuF33wYKGT5AMvbdTTA8EZzQCCRZABIAQBgoCIIFhdA\nMgBUaBgBYKAAiYAFMEBgRiBpLv33NI9pN30p/1z/KZBwgqZ8DLn44S7G/zKQSoDDKUAhEUAQSEEA\nYAAgCBYXADAQVEBAASCgACGIABBAAEYgdDC8dahsURLiv7YYlzdAOofWfRiLRfsAy8t1NODgRnNA\nIJFEAEgAAGCgIAgWE0AyAFRIAABgoACJgAUwQCBGIEAikEQgCAACYKggaBITwDIAVEwAAGCAAImA\nJTBAIAIhYDC5dGhtURJzvzYYl7dAOofWfRgLePMA34t1cODARnNAIBFAEEgAAGAAIAgSEwAwEFxA\nYEEggAwhiAAQQAJGIHEy1WQoDEEQ4KiyTJcTwDIExGwMiWHRAMCpIbBogEIhcapb+iytSRfzGbNY\nt4PQeEuubByba8Ix2vkwWOiIwil4oLhwie1RjqK3rhiXtUIagdbsEBpr4wDb5XV54EDGYnAwuXSI\nbVEb4rekGBe3QDoB1mwYGm/6EtundT3g4MZicaBb+qytSZb+GbFct8PYeKGObBybbdox2ekyePiB\nwij1oBl6rKxJm/IRoVi300g4gYpsHDtr2jPZqzR42MDCIA==\n";

    string decoded_data = base64_decode(data);

    Mat m(rows, cols, type, (void*)decoded_data.data());

    cout << m << endl;

    return 0;
}

How to restore base64 encoded opencv java matrix to C++ cv::Mat?

1 Answers1