1

I've created a dll to pass two images from unity to opencv and return a struct. But I'm getting a run time c++ error. This is the code I used in C++:

    struct rigidTransform
    {
        rigidTransform(double eular_angle_x, double eular_angle_y, double eular_angle_z, double eular_dis_x, double eular_dis_y, double eular_dis_z) : Eular_angle_x(eular_angle_x), Eular_angle_y(eular_angle_y), Eular_angle_z(eular_angle_z), Eular_dis_x(eular_dis_x), Eular_dis_y(eular_dis_y), Eular_dis_z(eular_dis_z) {}
        double Eular_angle_x;
        double Eular_angle_y;
        double Eular_angle_z;

        double Eular_dis_x;
        double Eular_dis_y;
        double Eular_dis_z;
    };
        struct Color32
    {
        uchar r;
        uchar g;
        uchar b;
        uchar a;
    };

    extern "C" rigidTransform __declspec(dllexport) __stdcall findPose(Color32* img1, Color32* img2,  int width, int height)
    {
    Mat Img1(height, width, CV_8UC4, img1), Img2(height, width, CV_8UC4, img2);
    //my pose estimation code here

    return Tr;
    }

And my C# code used in unity is:

    using System.Collections;
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using UnityEngine;

    public struct regidTransform
    {
        public double eular_angle_x;
        public double eular_angle_y;
        public double eular_angle_z;
        public double eular_dis_x;
        public double eular_dis_y;
        public double eular_dis_z;
    }



public class UI : MonoBehaviour
{
    [DllImport("test2", EntryPoint = "findPose", CallingConvention = CallingConvention.Cdecl)]
    public unsafe static extern regidTransform findPose(Color32[] img1, Color32[] img2, int width, int height);

    public regidTransform results_;


    public WebCamTexture webcamTexture;
    public Color32[] img1;
    public Color32[] img2;

    void Start()
    {

        results_ = new regidTransform();




        webcamTexture = new WebCamTexture();
        webPlane.GetComponent<MeshRenderer>().material.mainTexture = webcamTexture;
        webcamTexture.Play();

        if(webcamTexture.isPlaying)
        {
            img1 = webcamTexture.GetPixels32();
            System.Array.Reverse(img1);
            img2 = webcamTexture.GetPixels32();
            System.Array.Reverse(img2);


            unsafe
            {

                    results_ = findPose(img1, img2, webcamTexture.width, webcamTexture.height);


            }
        }




    }


    void Update()
    {


    }
}

When I run this code, it throws an error saying:

Runtime Error!

Program:

This Application has requested the Runtime to terminate it in an unusual way. Please contact the application support team.

Kindly assist me on what wrong I've made in this code.

Thanking You in advance!

UPDATED:

With the help of @Programmer I found that the code is getting crashed when calling calcOpticalFlowPyrLK(). And when checked if I'm getting the image correctly using imshow() function, I found it results a complete black image.

Althaf
  • 143
  • 2
  • 11
  • 1
    You need to start commenting stuff on the C# and C++ function. Make both functions to be empty functions then see if problem exist. If the problem is still there, make the `findPose` function from C++ to be a void function that doesn't return anything then check if the crash is still happening. You can use this to pinpoint what's really causing the issue. – Programmer Nov 19 '17 at 21:00
  • When commented out. It works perfectly, returning 0 as defined in the constructor. – Althaf Nov 19 '17 at 21:21
  • 1
    Keep going. continue uncommenting them one by one until there is a crash. Comment that line then uncomment it again to make sure it's the problem then post that line of code that's causing the crash. – Programmer Nov 19 '17 at 22:10
  • Thanks for assisting me on this. It is getting crashed when calling calcOpticalFlowPyrLK() function. It seems, the Mat type I'm using is not valid to use this function. Is there an alternative? Thank You very much for this extend. – Althaf Nov 19 '17 at 22:46
  • 1
    I don't know how you are calling and what you passing to the `calcOpticalFlowPyrLK` function. That part of code is not even in your question so I think it is not possible to help without that code but I suspeect you are passing null to it – Programmer Nov 19 '17 at 22:59
  • When I locally run it by PC, it works perfectly. And even when I call VideoCapture it worked. Thats the reason I'm suspecting. I'll update the function that call calcOpticalFlowPyrLK() code in my question. Thank You Again – Althaf Nov 19 '17 at 23:05
  • @Programmer you are right. When I simply passed an imshow() function, it result with a black picture. I've done that image passing process by referring this article (http://amin-ahmadi.com/2017/05/24/how-to-use-opencv-in-unity-example-project/) – Althaf Nov 20 '17 at 00:06
  • 1
    Well, check for null for all the variables you are passing to that function. Use my [`Debug.Log`](https://stackoverflow.com/questions/43732825/use-debug-log-from-c) plugin to save your self time. You Should be able to see the C++ variables in the Editor with that. For example `if(img_1 == null){Debug::Log("img_1 is null", Color::Red);}`. If you are still getting errors, comment out `calcOpticalFlowPyrLK` then use the `Debug::Log` above to make sure that everything is fine. – Programmer Nov 20 '17 at 00:12
  • I don't think, that needs to be checked. When I call imshow("", Img1) function immediately inside the dll. It shows the black picture. I think, I'vent implemented a way to pass the image correctly. But couldn't identified yet. I'm pretty sure about rest of the C++ code. – Althaf Nov 20 '17 at 00:34

1 Answers1

0

Finally I figured it out.

Yes that code worked well. and there is no error in my C++ code or C# code. The reason that resulted a complete black image for me is, I call my dll fuction in side strat() function of unity, which only execute once. So I found in my case, that the typical 1st image acquired to dll will mostly a black one, this was proved when I tested with imshow() function inside the dll. The reason it throws error is because I'm passing that black image to calcOpticalFlowPyrLK() function, which doesn't have any feature points to track.

So I created another function inside dll to check if the true image is acquired before passing it to my code by checking for feature points.

Something like:

featureDetection(frame_1, points1);

if (points1.size() > MIN_NUM_FEATURES)
{
    //Pass the frame to my code.
}

Now things will perfectly work. My sincere thanks to the @Programmer who helped me to figure this out.

Althaf
  • 143
  • 2
  • 11