1

This is a followup question from this one,
I'm trying to do a simple resize using opencv which ends up crashing!
This is the example code which the access violation occurs:

void Classifier::Preprocess(const cv::Mat& img, std::vector<cv::Mat>* input_channels)
{
    /* Convert the input image to the input image format of the network. */
    cv::Mat sample;
    if (img.channels() == 3 && num_channels_ == 1)
        cv::cvtColor(img, sample, cv::COLOR_BGR2GRAY);
    else if (img.channels() == 4 && num_channels_ == 1)
        cv::cvtColor(img, sample, cv::COLOR_BGRA2GRAY);
    else if (img.channels() == 4 && num_channels_ == 3)
        cv::cvtColor(img, sample, cv::COLOR_BGRA2BGR);
    else if (img.channels() == 1 && num_channels_ == 3)
        cv::cvtColor(img, sample, cv::COLOR_GRAY2BGR);
    else
        sample = img;

    //resize image according to the input
    cv::Mat sample_resized;
    Size size(input_geometry_.width,input_geometry_.height );

    if (sample.size() != input_geometry_)
        cv::resize(sample, sample_resized, size);
    else
        sample_resized = sample;

    //...
}

This is what I get in my C# solution when debugging :

Exception thrown at 0x00007FF8C8D9AA90 (opencv_imgproc310d.dll) in Classification Using dotNet.exe: 0xC0000005: Access violation reading location 0x0000018B000F2000.

If there is a handler for this exception, the program may be safely continued.

When I debug into the opencv code, I can see that sz is a null struct(<struct at NULL>) :

template<typename _Tp> inline
Size_<_Tp>::Size_(const Size_& sz)
    : width(sz.width), height(sz.height) {}

and this clearly causes the access violation!.

What is wrong here and what should I do ?

Update :
More information :
sample.size() is as exactly as img.size() since the last else clause is executed and thus sample = img.
sample.size() = {width=256 height=378 } cv::Size_<int>
num_channels_ is 3
img.channels() is 3 so is sample.channels()
When I use imshow() to display either img or sample, the access violation exception is issued. creating a simple black image and displaying it in imshow() is fine though.

Hossein
  • 24,202
  • 35
  • 119
  • 224
  • Have you checked the values of `input_geometry_.width` and `input_geometry_.height`? And also what is the value of `sample.size()`? – fedepad Aug 16 '17 at 06:46
  • And also about `if (sample.size() != input_geometry_)`: what kind of object is `input_geometry_`? – fedepad Aug 16 '17 at 06:55
  • @fedepad: input_geometry_ is of type `cv::Size()` just like the `size` variable. at first I thought maybe its because of input_geometry, then I used a new variable and got the same exception. the input size is `382x256` a color image. the height and width are `224x224` respectively. – Hossein Aug 16 '17 at 06:59
  • Why you created two questions highlighting the same problem? – fedepad Aug 16 '17 at 07:08
  • @fedepad: They are different questions, That question about the proper way of interacting between C# and a dll . and I explained my way of doing it. This is a separate question about the resize issue. if this is rectified, the previous question is still valid. – Hossein Aug 16 '17 at 07:11
  • Sorry but since you said `input_geometry_` is an object of type Size, in Size `size(input_geometry_.width,input_geometry_.height );` why don't you pass directly the `input_geometry_` object instead of passing the two data members? Because you show the constructor which takes an object and from that it extracts the two fields, instead you're passing the two fields (unless there is another Size constructor which is not showed here!) – fedepad Aug 16 '17 at 07:19
  • OK, checked the doc. There is such a constructor, so what you wrote should be ok. So you confirm that `input_geometry_.width = 224` and `input_geometry_.height = 224` and if you try to print to the console `input_geometry_.width` and `input_geometry_.height` just before calling `Size size(input_geometry_.width,input_geometry_.height );` it prints out those two values? – fedepad Aug 16 '17 at 07:28
  • @fedepad: yes, I even went further and hardcoded 224 in the size variable (i.e Size size(224,224)) yet the same issue persists! – Hossein Aug 16 '17 at 07:38
  • @fedepad, input_geometry_ is a field defined as `cv::Size input_geometry_;` in the header file. I was using the whole variable at first, then as I mentioned earlier, I thought maybe this is the culprit, so I defined a new variable name size and did what is shown in the snippet posted. – Hossein Aug 16 '17 at 07:41
  • And you guarantee that `sample.size()` != 0 and sample.size() height and weight are the right ones (382x256)? Could you verify that using assert on the height and weight of the object sample right before `if (sample.size() != input_geometry_)`? And after you test that, can you try to comment out the if else condition `if (sample.size() != input_geometry_)` and just let execute resize()? – fedepad Aug 16 '17 at 07:58
  • @fedepad: Yes I guarantee `sample.size() !=0` and has the right values. and yes I used assert as well, no issues here. and yes resize regardless of using the if statement, throws an access violation exception just like before. – Hossein Aug 16 '17 at 08:19
  • I found out the exception is caused because of the `sample`, when I created a new image (`cv::Mat ccc(240, 320, CV_8UC3, Scalar(0, 0, 0));` and used instead of sample, everything worked just fine.! now I'm back to the where I started! what is wrong with my way of creating the new image from C# in my c++ code!! – Hossein Aug 16 '17 at 08:56
  • can you print sample.size after your cvtColor block? Can you find out, whether the right if clause is chosen? Can you imshow sample? Can you tell us the value of `num_channels_` and the value of `img.channels()` in your sample? – Micka Aug 16 '17 at 10:11
  • can you tell us in which line your code crashed? Are you sure it is within this part of code? I would recommend you to use `sample = img.clone();` and `sample_resized = sample.clone();` for testing, since there might be some unhandled side effects on sharing the pixel memory on multiple images, but within this code snippet it looks quite safe (if none of the input images and sizes is empty). – Micka Aug 16 '17 at 10:15
  • @Micka: `sample.size()` is as exactly as `img.size()` since the last `else` clause is executed and thus `sample = img`. `sample.size() = {width=256 height=378 } cv::Size_` `num_channels_ ` is 3 `img.channels()` is 3 so is `sample.channels()` When I use imshow() to display either img or sample, the access violation exception is issued. creating a simple black image and displaying it in imshow() is fine though. – Hossein Aug 16 '17 at 10:26
  • @Micka: Yes, I'm using the debug build and it crashes exactly at the resize line. (using imshow, it crashes at that line respectively, so there must be a problem with the img itself!) – Hossein Aug 16 '17 at 10:29
  • so you can't imshow the `img` (even before any other operation)? Looks like your input image is broken or your environment is broken. Are you compiling in debug or in release mode? Does it work if you send a newly created black image (like the one that is working if you create it locally) to your ::Preprocess function instead of the image you are currently using for testing? – Micka Aug 16 '17 at 10:30
  • ok, probably your problem is in calling the function from C#, right? Since you don't have an answer on https://stackoverflow.com/questions/45697930/how-do-i-send-an-image-from-c-sharp-to-a-dll-accepting-opencv-mat-properly yet, you'll probably have to fix that first. Maybe you should reduce the test setting to just call a function that tries to imshow the image. Did you search and find topics like this? https://stackoverflow.com/questions/29290428/how-to-pass-managed-bitmap-from-c-sharp-to-wrapped-unmanaged-opencvs-mat – Micka Aug 16 '17 at 10:35
  • @Micka: yes it does. I'm compiling in Debug mode. Yes if I send a black image it would work. I mean if I create a black image using `cv::Mat ccc(240, 320, CV_8UC3, Scalar(0, 0, 0));` and feed it to the Classify method it works – Hossein Aug 16 '17 at 10:36
  • ok, that is strange. Can you output the previous "broken" image in the calling C# function? – Micka Aug 16 '17 at 10:37
  • @Micka: The image is fine, I have two methods, one of them read the image using its path which works just fine, and the other one recieves the image which is problematic. I believe the way I'm creating a new Mat image is the issue not the C# part sending the image. since I'm sending it in the form of unsigned char (or byte[] array in C#) – Hossein Aug 16 '17 at 10:39
  • can you add the code on how you send/receive the images and how you call the function from C# side? – Micka Aug 16 '17 at 10:55
  • 1
    @Micka: Check out my answer, your initial guess was right . I took a different path in the C# side and everything started to work. However, I really would like to know why my initial approach failed! Thank you very much for your time and help . I really appreciate it – Hossein Aug 16 '17 at 16:21

1 Answers1

2

Thanks to @Micka's suggestions in the comments, I found out the cause was first due to the image that was sent to the preprocess method and not the resize method by itself.
Further debugging proved @Micka's suspicion about the faulty code on C# side. I was originally wrote in the C# side :

[DllImport(@"CDll2.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
static extern void Classify_image(byte[] img, uint height, uint width, byte[] out_result, out int out_result_length, int top_n_results = 2);

private string Classify(Bitmap img, int top_n_results)
{
    byte[] result = new byte[200];
    int len;
    var img_byte = (byte[])(new ImageConverter()).ConvertTo(img, typeof(byte[]));

    Classify_image(img_byte, (uint)img.Height, (uint)img.Width,res, out len, top_n_results);

    return  ASCIIEncoding.ASCII.GetString(result);
}

and this was the code on the dll part :

CDLL2_API void Classify_image(unsigned char* img_pointer, unsigned int height, unsigned int width, char* out_result, int* length_of_out_result, int top_n_results)
    {
        auto classifier = reinterpret_cast<Classifier*>(GetHandle());
        cv::Mat img = cv::Mat(height, width, CV_32FC3, (void*)img_pointer);

        std::vector<Prediction> result = classifier->Classify(img, top_n_results);

        //misc code...
        *length_of_out_result = ss.str().length();
    }

This proved to be plain wrong, or at the very least buggy. Using this code, the image on the C++ side seemed to have been initialized properly, the number of channels, the height, and width all seemed fine.
But the moment you attempt to use the image, either by resizing it or even showing it using imshow(), it would crash the application and be giving an access violation exception, the very same error that happened when resizing and is posted in the question.
Looking at this answer, I changed the C# code responsible for handing the image to the dll. the new code is as follows :

//Dll import 
[DllImport(@"CDll2.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
static extern void Classify_Image(IntPtr img, uint height, uint width, int step, byte[] out_result, out int out_result_length, int top_n_results = 2);

//...
//main code 

Bitmap img = new Bitmap(txtImagePath.Text);
BitmapData bmpData = img.LockBits(new Rectangle(0, 0, img.Width, img.Height),  ImageLockMode.ReadWrite,  img.PixelFormat);
result = Classify_Image(bmpData.Scan0, (uint)bmpData.Height, (uint)bmpData.Width, bmpData.Stride, res, out len, top_n_results);
img.UnlockBits(bmpData); //Remember to unlock!!!

and the C++ code in the DLL :

CDLL2_API void Classify_Image(unsigned char* img_pointer, unsigned int height, unsigned int width, int step, char* out_result, int* length_of_out_result, int top_n_results)
    {
        auto classifier = reinterpret_cast<Classifier*>(GetHandle());

        cv::Mat img = cv::Mat(height, width, CV_8UC3, (void*)img_pointer, step);

        std::vector<Prediction> result = classifier->Classify(img, top_n_results);

        //...
        *length_of_out_result = ss.str().length();
    }

and doing so rectified all access violations I was getting previously.

JK.
  • 21,477
  • 35
  • 135
  • 214
Hossein
  • 24,202
  • 35
  • 119
  • 224