Convert Texture2D to OpenCV Mat?

Question

There is a post about converting OpenCV cv::Mat to Texture2D in Unity and I provided an answer which works well. Now, I am trying to do the opposite but have been stuck on this for few hours now.

I want to convert Unity's Texture2D to OpenCV cv::Mat so that I can process the Texture on the C++ side.

Here is the original Texture2D in my Unity project that I want to convert to cv:Mat:

Here is what it looks like after converting it into cv:Mat:

It looks so washed out. I am not worried about the rotation of the image. I can fix that. Just wondering why it looks so washed out. Also used cv::imwrite to save the image for testing purposes but the issue in also in the saved image.

C# code:

[DllImport("TextureConverter")]
private static extern float TextureToCVMat(IntPtr texData, int width, int height);

unsafe void TextureToCVMat(Texture2D texData)
{
    Color32[] texDataColor = texData.GetPixels32();
    //Pin Memory
    fixed (Color32* p = texDataColor)
    {
        TextureToCVMat((IntPtr)p, texData.width, texData.height);
    }
}

public Texture2D tex;

void Start()
{
    TextureToCVMat(tex);
}

C++ code:

DLLExport void TextureToCVMat(unsigned char*  texData, int width, int height)
{
    Mat texture(height, width, CV_8UC4, texData);

    cvNamedWindow("Unity Texture", CV_WINDOW_NORMAL);
    //cvResizeWindow("Unity Texture", 200, 200);
    cv::imshow("Unity Texture", texture);
    cv::imwrite("Inno Image.jpg", texture);
}

I also tried creating a struct on the C++ side to hold the pixel information instead of using unsigned char* but the result is still the-same:

struct Color32
{
    uchar r;
    uchar g;
    uchar b;
    uchar a;
};


DLLExport void TextureToCVMat(Color32* texData, int width, int height)
{
    Mat texture(height, width, CV_8UC4, texData);

    cvNamedWindow("Unity Texture", CV_WINDOW_NORMAL);
    cvResizeWindow("Unity Texture", 200, 200);
    cv::imshow("Unity Texture", texture);
}

Why does the image look so so washed out and how do you fix this?

The only thing that comes to my mind would be passing a set of params to `imwrite` like `std::vector params; params.push_back(CV_IMWRITE_JPEG_QUALITY); params.push_back(100);` and then call `imwrite`. — mutantkeyboard, Jun 19 '18 at 10:15
@mutantkeyboard Note that I don't care about `imwrite`. I am just using it to debug the image and figure out while it is displaying like that. If `cv::imshow` failed too, `cv::imwrite` should also fail. I just tried it and the problem is still there but nice suggestion — Programmer, Jun 19 '18 at 10:22
OpenCV creates the image as `BGRA` by default. `Color32` is RGBA. Use `cv::cvtColor` to convert it to `RGBA`. — zindarod, Jun 19 '18 at 10:27
@zindarod I printed [`Texture2D.format`](https://docs.unity3d.com/ScriptReference/Texture2D-format.html) on the Unity side and it says ["RGB24"](https://docs.unity3d.com/ScriptReference/TextureFormat.RGB24.html). Also, I have been trying `cv::cvtColor` with different format even before making this post but those didn't work — Programmer, Jun 19 '18 at 10:33
@zindarod but he already has `CV_8UC4` which is a 4 channel 8-bit matrix. I think it's RGBA. I don't know if you tried with gamma correction @Programmer — mutantkeyboard, Jun 19 '18 at 10:34
In your C++ code, do: `cv::cvtColor(texture,texture,cv::COLOR_BGRA2RGBA)`. As you said the `Texture2D.format` gives `RGB24`, so in your case the alpha channel doesn't matter. — zindarod, Jun 19 '18 at 10:38
@zindarod Beautiful. Please leave an answer. There are so many formats and I guess I skipped `COLOR_BGRA2RGBA` during the test. — Programmer, Jun 19 '18 at 10:42

score 7 · Accepted Answer · answered Jun 19 '18 at 10:58

OpenCV creates images as BGR by default, whereas Color32 stores pixels as RGBA. However since OP mentioned in the comments that the Texture2D.format gives texture format as RGB24, we can ignore the alpha channel altogether.

DLLExport void TextureToCVMat(unsigned char*  texData, int width, int height)
{
    Mat texture(height, width, CV_8UC4, texData);
    cv::cvtColor(texture,texture,cv::COLOR_BGRA2RGB);

    cv::imshow("Unity Texture", texture);
    cv::waitKey(0);
    cv::destroyAllWindows();

}

Convert Texture2D to OpenCV Mat?

1 Answers1

Linked